lualatex vs pdflatex- char encoding issue not fixed by suggestions found in CTAN etc

[Mike Marchywka]

On Thu, Feb 17, 2022 at 10:55:18AM +0100, Ulrike Fischer wrote:
> Am Wed, 16 Feb 2022 18:29:02 -0500 schrieb Mike Marchywka:
> 
> > I recently changed my default builds from pdflatex to lualatex
> > and now the foreign chars are creating cryptic failure modes.
> 
> Well it would help if you would mention the actual error (and would
> show a concrete example)

yes, I just thought this was a well known problem and it
had "A{n}" accepted solution...

> 
> > When I finally figured it out, neither of these suggestions
> > seemed to help.
> > 
> > https://www.latex-project.org/news/latex2e-news/ltnews26.pdf
> 
> > \usepackage[OT1]{fontenc}
> > 
> > \usepackage{luaninputenc}
> 
> Do not use fontenc (OT1 or T1) with lualatex. 
> Do not use luainputenc.
>

I thought these were supposed to make lualatex act like pdflatex
which was my only immediate objective.
 
> 
> Encode your files in utf8 or use pure ascii.
>
I was trying to avoid the issue but since everything seemed to work
I did not not worry about it. I have been changing my bibtex
downloader, TooBib, I guess i could have checked the download
dates of the offending entries but since it worked on pdflatex
I didn't think it was a factor. 

In essence I did make everything ASCII although I never took a
char histogram looking for high bit set.
However, instead of converting offenders to latex I just picked
a close ASCII char. 
 
> With a current latex this here should work out of the box with an
> utf8 encoded file: 
> 
> \documentclass{article}
> \begin{document}
> Grüße
> \end{document}
> 
Which it appears to do, see output at end. IIRC some of the chars were
showing up in vi as "<90>" but i can try to "od" them if I get more.
I guess the question is what happens if that is in a bibtex
entry... 

> 
> If you are using bibtex: be aware that it isn't utf8 aware and that
> won't change if you use lualatex. It can't for example create

AFAICT the only non-ASCII I had was in the bibtex entries so
I was surprised that lualatex mattered but I had to dig
into the log to find it choking on the bbl file. But the bbl
file worked with pdflatex.   


> initials from names starting with cyrillic or accented chars. 
> If you want to use lots of utf8 inside bib files use biblatex +
> biber.

Thanks, I seem to recall seeing something about biber and I can look
at those. But all of this was a distraction that came at a confusing
time because I wast just trying to verify a project was still intact
after I wrote out a custom bbl file from new code I wrote - so
I at first thoguth my new code corrupted the project or bib files :)
This is on top of non-ASCII already being a distraction but I think
I now accomodates this mostly. 

My new c++ bbl writer is just a skeleton to let me implement
arbitrary logic easily to cite less traditional sources in
arbitrary ways ( notably "bomtex" or a bill of materials ).
Do you knoww offhand if there is any facility or interest
regarding document specific modifications to citation
url's that identify the document responsible for the
click?




> 
> -- 
> Ulrike Fischer 
> http://www.troubleshooting-tex.de/
> 



pdflatex ulrike.tex 
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./ulrike.tex
LaTeX2e <2016/02/01>
Babel <3.9q> and hyphenation patterns for 12 language(s) loaded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
No file ulrike.aux.
[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] (./ulrike.aux) )
(see the transcript file for additional information)</usr/share/texlive/texmf-d
ist/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on ulrike.pdf (1 page, 10656 bytes).
Transcript written on ulrike.log.
marchywka at happy:/home/documents/latex/proj/covidup$ lualatex ulrike.tex 
This is LuaTeX, Version beta-0.80.0 (TeX Live 2015/Debian) (rev 5238) 
 restricted \write18 enabled.
(./ulrike.tex
LaTeX2e <2016/02/01>
Babel <3.9q> and hyphenation patterns for 1 language(s) loaded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo)) (./ulrike.aux)
[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] (./ulrike.aux))
(see the transcript file for additional information)
 264 words of node memory still in use:
   2 hlist, 1 vlist, 1 rule, 2 glue, 40 glue_spec, 1 write nodes
   avail lists: 1:1,2:12,3:2,4:22,6:12,7:1,9:6
<</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on ulrike.pdf (1 page, 10485 bytes).
Transcript written on ulrike.log.
marchywka at happy:/home/documents/latex/proj/covidup$ cat ulrike.
cat: ulrike.: No such file or directory
marchywka at happy:/home/documents/latex/proj/covidup$ cat ulrike.tex
\documentclass{article}
\begin{document}
Grüße
\end{document}


-- 

mike marchywka
306 charles cox
canton GA 30115
USA, Earth 
marchywka at hotmail.com
404-788-1216
ORCID: 0000-0001-9237-455X