Reader Grammar

The reader grammar defines whitespace/comments, identifiers, booleans, characters, strings, numeric tokens, lists, vectors, bytevectors, and quotation prefixes.

See it run

LISPEX
(list #\a #\x41 "\x41;" #true)

Observed result

OUTPUT
(a A A #t)

The base spelling of a character is #\ followed by one scalar, as in #\a. Hexadecimal works too, and there it carries no semicolon. Named characters are only the fourteen listed below. The escapes a string accepts are a closed set as well. There are six, \", \\, \n, \t, \r, and the hexadecimal \xHEX;, which does carry the semicolon. Any other letter after the backslash is E150. There is no \a here, common as it is elsewhere.

#true and #false are the same values as #t and #f.

How to reason about it

  • An integer is written -?(0|[1-9][0-9]*). A rational is an integer numerator over a positive denominator in the one accepted spelling, and a real requires a fraction or an exponent.
  • Dotted-list placement, escape spelling, byte range, and delimiter closure are checked before normalization.
  • Strings spell a hexadecimal scalar as \xHEX;, while characters spell it as #\xHEX without the semicolon.
  • Source spans use the reader input identity and deterministic positions.
  • A leading plus sign makes a name, not a number. Only a minus sign is accepted there, so (define +5 7) simply works.
  • An empty fraction, as in 1., is E100. One of the fraction or the exponent has to actually be written.
  • The only brackets are ( and ). Write [ or { and you get E100 with a message of its own.
  • A dot marks a dotted pair only when a delimiter follows it, so ... and .foo are ordinary names. A vector accepts no dot at all, so #(1 . 2) is E100.
  • Bytevector elements are integers from 0 through 255 written out literally. Even #u8(4/2), which would compute to 2, is rejected.

Pinned token grammar

TokenAccepted shapeRejected boundary
integer-?(0|[1-9][0-9]*)leading zero, radix/exactness prefix
rational<integer>/[1-9][0-9]*, reduced after readzero/negative denominator spelling, leading-zero denominator
realdecimal with fraction and/or exponentbare integer ambiguity, non-finite literal
stringquoted UTF-8 with pinned escapes, including \xHEX;unterminated escape or missing hexadecimal semicolon
character#\<scalar>, #\xHEX, or pinned named forminvalid scalar or a semicolon after the hexadecimal character
bytevector#u8(<byte> ...)element outside 0..255

The named characters are exactly space, newline, linefeed, tab, return, null, nul, delete, rubout, escape, esc, backspace, alarm, and page. Line comments begin with ;. Block comments use nested #| ... |#. #; datum comments and #lang directives are rejected with E120 rather than quietly dropped the way whitespace and comments are.

A common mistake

Reader extensions not listed in the profile fail loudly.

Keep going

The manual walks the same grammar as a reading process, and the catalog lists the code a rejected token produces.

Source Text, Tokens, and Reader · Error and Warning Catalog