[texhax] Some puzzling TeX

[Stephen Hicks]

On Wed, Feb 16, 2011 at 3:44 PM, Rod Harries <rod at harries.net.nz> wrote:
> I am trying to understand why  one would write
>
> \def\:{\let\@sptoken= } \: %
>
> instead of just \let\@sptoken=   %
> or \def\@sptoken{ }
>
> Explanations (preferably designed for the simple-minded) would be appreciated.

Have you tried either of those two options?  It turns out they don't
behave the same at all -- see the following session:

====
$ latex \\makeatletter
This is pdfTeX, Version 3.1415926-1.40.10 (TeX Live 2009/Debian)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2009/09/24>
Babel <v3.8l> and hyphenation patterns for english, usenglishmax, dumylang, noh
yphenation, loaded.

*\let\@sptoken=  %

*\relax

*\show\@sptoken
> \@sptoken=\relax.
<*> \show\@sptoken

?

*\def\@sptoken{ }

*\show\@sptoken
> \@sptoken=macro:
-> .
<*> \show\@sptoken

?

*\def\:{\let\@sptoken= } \: %

*\show\@sptoken
> \@sptoken=blank space  .
<*> \show\@sptoken

?

*
! Emergency stop.
<*> \end

No pages of output.
Transcript written on texput.log.
====

What's happening here?  It turns out that spaces are particularly
tricky in TeX.  I don't have my TeXBook on me right now, but the way
the \let primitive is defined, it allows an optional "=" in between
the token it's assigning to and the token it's assigning from.  This
optional "=" may be surrounded by any number of spaces.  So when you
simply say "\let\@sptoken= %" it's still waiting for the token to
assign to \@sptoken, and the next non-space it encounters will be it
(hence the \relax on the next line).  In the case of "\def\@sptoken{
}" that's completely different.  This is defining a macro, which has a
completely different category code than a space.  The point here is
that the author presumably wants \@sptoken to really be in every way a
space, probably so he can use \ifcat to check if something else has
the same catcode.  But the \def'ed \@sptoken (and all normal control
sequences) has catcode 16, rather than 9.

So what is this actually doing?  It defines the macro to actually
contain the space.  Moreover, when control characters (i.e. \: rather
than \x) are expanded, they don't eat any following spaces.  So then
"\: " expands to \let\@sptoken=<space><space>, which normally is
impossible to achieve since TeX eats adjacent spaces when it first
tokenizes the file.

Cheers,
steve