[tex-live] General test suite for TeX-Live

[Zdenek Wagner]

Hi all,

the idea of automated tested was here more than a year ago and I explained
that I consider it impossible. Nobody knows which set of packages will be
used in a document created by each user. It is only possible to create
subsets of test files. It happened to my zwpagelayout package, it did not
work for one user at all. After the bug report I found conflicting
packages, therefore I added code to cope with it. I still cannot guarantee
that it will work with every package (it does not work with xcolor and the
support of RTL is very limited). Without loading such packages you cannot
find the problem and it took a few years before somebody did it. In this
case the error appeared when the package was loaded, but for instance if
you have path.sty together with any package loading url.sty (for instance
hyperref), the packages wil load without errors but it will fail when
typesetting the URL.

If you load a lot of packages into a single document, it is more probable
that new clashes will be detected but then it might be difficult to find
the reason. Moreover, it sometimes depends on the order of packages.


Zdeněk Wagner
http://ttsm.icpf.cas.cz/team/wagner.shtml
http://icebearsoft.euweb.cz

2016-06-19 4:12 GMT+02:00 Reinhard Kotucha <reinhard.kotucha at web.de>:

> On 2016-06-18 at 14:59:18 +0900, Norbert Preining wrote:
>
>  > Hi Uwe,
>  >
>  > I think this is a nice idea, but needs good thinking. I don't want
>  > just a bunch of examples - all must be automatized and best
>  > included in our daily checks.
>
> Hi Uwe and Norbert,
> maybe it's worthwhile to know how Thomas Esser tested all the packages
> before they were included in teTeX.
>
> Thomas said that his tests were never supposed to detect all problems
> but a package should at least be able to compile its own
> documentation.  Thus he created a Makefile for each and every package
> he supported.
>
> Even this was an enormous amount of work.  teTeX was a small subset of
> TeX Live, hence it was feasible.  But TeX Live was always much bigger
> and grew rapidly since then.
>
> Compiling package documentation is a good starting point.  If a
> package comes with a .dtx file, the process can probably be
> automatized, but I fear that most packages require manual
> intervention.
>
> An other point comes to my mind.  Occasionally people report that some
> dependencies in texlive.tlpdb are incorrect.  Of course, it's
> difficult to get everything correct.  But if there is a halfway
> comprehensive test suite we can probably determine dependencies from
> the log files.  Another approach is to run TeX with the --recorder
> option on these files and evaluate the .fls files afterwards.
>
> I don't think that the dependencies in texlive.tlpdb can be determined
> automatically but I'm convinced that a halfway comprehensive test
> suite can provide useful information.
>
> Uwe's idea is, as far as I understand, to create files which load many
> packages at once.  This reduces the mumber of required tests
> significantly.  The caveat is that some packages are overlooked
> because Uwe doesn't use them.
>
> After all, I like Uwe's idea.  In TeX parlance:
>
>   \let\uwe\proceed
>
> Regards,
>   Reinhard
>
> --
> ------------------------------------------------------------------
> Reinhard Kotucha                            Phone: +49-511-3373112
> Marschnerstr. 25
> D-30167 Hannover                    mailto:reinhard.kotucha at web.de
> ------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/tex-live/attachments/20160619/e84b5e9e/attachment-0001.html>