[tex-live] Ignored italic correction with lua(la)tex format

[Philipp Stephani]

2011/10/7 Khaled Hosny <khaledhosny at eglug.org>:
> On Thu, Oct 06, 2011 at 11:59:56PM +0200, Philipp Stephani wrote:
>> 2011/10/6 Khaled Hosny <khaledhosny at eglug.org>:
>> > On Thu, Oct 06, 2011 at 11:31:38PM +0200, Philipp Stephani wrote:
>> >> 2011/10/6 Khaled Hosny <khaledhosny at eglug.org>:
>> >> > Hi all,
>> >> >
>> >> > After a long night of debugging and hair pulling, I finally found why
>> >> > italic correction for OpenType math fonts is ignored in luatex and
>> >> > lualatex formats but not in context.
>> >> >
>> >> > For some reason not yet clear to me, the setting of catcode to 11 in
>> >> > luatex-unicode-letters.tex causes luatex to not apply italic correction
>> >> > (might be related to the fact that luatex applies IC selectively).
>> >> >
>> >> > This minimal example shows the issues:
>> >> >
>> >> > \input luaotfload.sty
>> >> > \font\xits={file:xits-math.otf} at 10pt
>> >> > \textfont0=\xits
>> >> > $$\Umathchar"7"0"1D443\Umathchar"4"0`($$
>> >> > \bye
>> >> >
>> >> > after commenting the \catcode part in line 18 of
>> >> > luatex-unicode-letters.tex and rebuilding the format, italic correction
>> >> > is applied.
>> >> >
>> >> > I'm not sure what is the proper fix here, though.
>> >>
>> >> LuaTeX explicitly removes italic correction after letters:
>> >>
>> >>             if (is_new_mathfont(cur_f) && get_char_cat_code(cur_c) == 11) {
>> >>                 *delta = 0;     /* no italic correction in mid-word of
>> >> text font */
>> >>           }
>> >>
>> >> This seems like a bug to me.
>> >
>> > This is on purpose, following to MS implementation/Cambria Math.
>>
>> How can this be, given that Word has no catcodes? Anyway, with Cambria
>> Math it works as expected, maybe even a font bug?
>
> Word does not apply italic correction between letters (i.e. middle of
> the word), catcodes it what luatex is using ti differentiate letters
> from non-letters.

But this is rather contrary to the behavior of traditional TeX which
doesn't use catcodes at all at this stage (see e.g. Dan's posting).

> Cambria has minimal italic correction so the
> difference is not so visible and its sub/superscripts are handled with
> math kerning so the difference is even more invisible. XITS renders fine
> in Word.

For Cambria, the test looks better without italic correction. The
OpenType math document allows math italic correction to be applied
after a run of italic characters, but does not specify how to tell
italic and upright characters apart. But I'm quite sure that catcodes
are the wrong way to make this distinction. Probably LuaTeX needs a
list of all italic characters (there doesn't seem to be a Unicode
property for this).

>
>> > In
>> > either case it makes no sense IMO to give non-letter characters a letter
>> > catcode.
>>
>> The italic P you used in your example is a letter, and the parenthesis
>> has catcode 12.
>
> I see, but then I don't know what the exact detail of what luatex is
> doing, all I know is that it applies italic correction selectively.

It uses the code I posted above: it removes italic corrections after
category-11 characters, but leaves it after category-12 characters.
That's why ConTeXt appears to work. But the mathematical characters
should absolutely have catcode 11 since they have the properties
ID_Start and ID_Continue (by virtue of being letters), so the LuaLaTeX
behavior is correct and should not be altered.