[tex-live] Technical showstoppers for TL2003

[Vladimir Volovich]

"HH" == Hans Hagen writes:

 >> in my opinion, macro packages should write to files in encoding
 >> independent form (e.g. like the LaTeX does) - that's really
 >> portable.

 HH> say that (in context) one says:

 HH>    \startbuffer
 HH>    abcd\eacute <8bitchar representing \eacute> ^^ef
 HH>    \stopbuffer

 HH> (the buffer is written to file) and then later on:

 HH>    \typebuffer

 HH> (which reads the file) now, what is supposed to end up typeset?
 HH> The literal code between \startbuffer and stopbuffer, so \eacute
 HH> (as string) the o bit char, and ^^ef (literal). if the <8bitchar>
 HH> was expanded to \eacute or ^^ef then that would be typeset
 HH> verbatim since there is no way (unless one starts adding
 HH> additional code to trigger things in which case some kind of
 HH> escape character would be needed).

exactly - verbatim environments are a special case compared to
"regular" writes which are supposed to be read back by TeX and by
writes supposed to be read by third-party applications.

in the above example, the portable way to write to file would be:

abcd\textbackslash eacute \'e \textasciicircum \textasciicircum ef

where \textbackslash typesets the `\' character and \textasciicircum
typesets the `^' character. when this is read back by TeX, it will
typeset verbatim what was between the \startbuffer and \stopbuffer.
and it is not hard to do that at the macro level...
(yes, here we use one escape character - `\').

and this is portable since it's independent on any input and font
encodings which might be arbitrary...

 HH> This in itself would present other problems, think of:

 HH> \startbuffer <tag>some text including 8 bit chars</tag>
 HH> \stopbuffer

 HH> here context can handle the buffer as verbatim text, tex text,
 HH> xml ...

writing files which are intended to be read by "3-rd party programs"
(not by TeX) is another case which could also be programmed at the
macro level - it's possible to make all characters expand "as is" for
such applications (then it will depend on cp8bit TCX to be able to
output raw 8-bit chars)... i.e., there are two kinds of writes - the
ones which are supposed to be read back by TeX (which should use
"portable" representation), and ones for "3-rd party programs" which
may/should use the 1-to-1 representation.

 HH> or think of

 HH> \startMPcode draw btex whatever kind of french accented text you
 HH> want etex ; \stopMPcode

well, btex ... etex is supposed to be processed by TeX, so the
"portable" encoding as in the first example should work just fine.

 HH> or

 HH> \startJScode str = "something in pdfdoc encoding with 8 but
 HH> chars" \stopJScode

thus us again something for 3rd party apps; actually, hyperref package
for LaTeX contains very good code which converts any string from LICR
to pdfdoc encoding.

 HH> so, control over what gets 'expanded' or 'translated' is up to
 HH> the application (macro package) and it really depends on what one
 HH> is dealing with

sure.

 HH> (internally context uses named glyphs for typeset material) and
 HH> for this purpose 8 bit in is 8 bit out. Otherwise we would force
 HH> all other (non tex) applications to think like tex when reading
 HH> data (with ^^'s) which i think is no real option -)

e.g. xindy can deal with the LICR... :)

Best,
v.