[tldoc] suggestions for the doc section 3.4.5 context lmtx and mkiv

[Hans Hagen]

On 2/24/2023 1:52 PM, Carlos wrote:
> On 2/23/23 17:37, Karl Berry wrote:
>> Hi Carlos,
>>
>>      I had some issues with the the way by which namely, context, and the
>>      scripts used therein handles any call with a file.
>>
>> I admit I'm not sure at what level the problem you're discussing is. 
> 
> I'm not sure either. I'm not. I thought of TeX Live documentation, 
> mainly  (believe it not), after spending quite a while frustrated while 
> trying to keep up with a recurring issue, and it was through the manual 
> indeed which seemed to resolve it But the problem shouldn't have been 
> there in the first place.
> If
>> some specific change to the TL manual that would help, please draft
>> something and let me know.
>>
>> In general, we cannot document detailed/exact ConTeXt (or anything else)
>> behavior in the TL manual, especially wrt what envvars are used or not
>> used, how they conflict, etc. It has to be left to the documentation of
>> the formats/programs. If it's not documented there, we can't solve that.
> 
> I wish I could draft just an excerpt for this. I wish. A suggestion for 
> a solution that is, that would tackle the source, and bypassing envvars
>>
>> If you think ConTeXt should behave differently, the change needs to be
>> sent to the ConTeXt developers. --best, karl.
>>
> 
> Sorry Karl. But I'm not here to tell them what they must do. I'm more 
> concerned with the tl guide than anything else, really. Don't forget 
> that it was solely by going over the tl guide  by which the issue was 
> resolved in my end. I had given up with this.
> 
> But let's go over the following.
> 
> If you were to go over this fragment on the tl guide that says
> 
> \dirname{TEXMFVAR} is the location for storing most cached runtime data
> specific to each user.  \dirname{TEXMFCACHE} is the variable name used
> for that purpose by Lua\LaTeX\ and \ConTeXt\ MkIV (see
> section~\ref{sec:context-mkiv}, \p.\pageref{sec:context-mkiv}); its
> default value is \dirname{TEXMFSYSVAR}, or (if that's not writable),
> \dirname{TEXMFVAR}.
> 
> 
> Both LuaLaTeX and ConTeXt are included. Correct?
> 
> And it clearly says that the default value for ConTeXt is TEXMFSYSVAR
> 
> It would not be surprising at all then, to find out that LuaLaTeX simply 
> does not seem affected at all by this issue, does it? and for a good 
> reason: presumably because of the kpathsea library.
> 
> But I genuinely can't remember if it ever happened with LuaTeX, and if 
> it did, I missed it, but with ConTeXt (whether standalone or not) this 
> is a recurring event. Give or take a handful of years, add or subtract a 
> couple of them in between. It's a cycle Karl. A cycle.
> 
> Anyway. Here's a possible draft for the doc, and from what I've seen 
> after breaking down my own system with this very same issue.
> 
> diff --git a/texlive-en.tex b/texlive-en.tex
> index 6c7c572..09b2529 100644
> --- a/home/ce/Downloads/texlive-en.tex
> +++ b/texlive-en.tex
> @@ -1187,6 +1187,10 @@ for LMTX, and
>   \begin{sverbatim}
>   context --luatex --generate
>   \end{sverbatim}
> +or
> +\begin{sverbatim}
> +mtxrun --generate
> +\end{sverbatim}
>   for MkIV, to refresh the \ConTeXt{} disk cache data.
>   The resulting files are stored under \code{TEXMFCACHE},
>   whose default value in \TL\ is \verb+TEXMFSYSVAR;TEXMFVAR+.
> @@ -1196,6 +1200,18 @@ all paths mentioned in \verb+TEXMFCACHE+, and 
> write to the first
>   path that is writable. While reading, the last found match will
>   take precedence in the case of duplicated cache data.
> 
> +It is recommended for the stored cache data on \ConTeXt\ to be
> +written under \code{TEXMFVAR} preferably over \code{TEXMFSYSVAR}.
> +
> +Duplicate cache data on both \code{TEXMFVAR} and \code{TEXMFSYSVAR}
> +could create a conflict while generating and updating the formats. This
> +may result in an error such as:
> +
> +\verb+mtxrun | unknown script `context.lua' or `mtx-context.lua'+
> +
> +The \dirname{luatex-cache} directories under both \code{TEXMFVAR} and
> +\code{TEXMFSYSVAR} would normally hold the duplicate runtime data cache.
> +
>   For more information, see \url{https://wiki.contextgarden.net/LMTX}\\
>   and \url{https://wiki.contextgarden.net/Running_Mark_IV}.

fwiw:

the caching code hasn't changed in over a decade; normally context 
itself will generate the file database and format (when it notices a 
change in engine or context version)

the context cache paths are hashed in a way that also stores the path so 
  conflicts are unlikely; there are dedicated subpaths for 'luatex' and 
'luametatex' and even dedicated suffixes in some cases (luajittex is 
still supported for mkiv but eventually it might get dropped)

when lmtx / luametatex is installed, the 'runner' is actually luametatex 
the choice of the engine --luatex flag to the 'context' runner (which is 
a copy of or symlink to luametatex as is mtxrun)

in an editor one can configure

   mtxrun --autogenerate --script context  <filename>

which will generate the file database and format if needed (and use it 
in runs after that)

so, there are three binaries (two are actually stubs, so the old runner 
- runscript or so, i'm not sure what was used in texllive - is now 
obsolete, which is actually good because that one occasionally got 
adapted out of our sight)

luametatex
mtxrun -> luametatex
context -> luametatex

and

two lua files (in the bin path):

mtxrun.lua      (also resolves --script specs)
context.lua     (eventually loads mtx-context.lua)

it's the same on all platforms.

the only complication i can imagine is that when the bin path is not 
relative to the rest the auto* stuff might not work as expected but 
there's not much i can do about that (it's then also controlled by env 
vars)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------