Source Text, Tokens, and Reader

The reader accepts a pinned UTF-8 S-expression grammar and reports deterministic reader-phase diagnostics before normalization.

When this matters

Start here when source text is rejected before evaluation, or when you are unsure whether parentheses describe a call or data. The reader owns spelling and structure; evaluation has not begun yet.

See it run

LISPEX
'(alpha 1 2/3 #u8(4 5))

Observed result

OUTPUT
(alpha 1 2/3 #u8(4 5))

Read the example

The leading quote keeps the whole form as data. The reader still recognizes a symbol, integer, rational, and bytevector, but none of them is called or computed. Removing the quote would put alpha in procedure position.

How to reason about it

  • Integer, rational, and real token grammars are disjoint; leading-zero and non-finite numeric spellings are rejected.
  • Lists, dotted pairs, vectors, bytevectors, strings, characters, quote, quasiquote, and comments have pinned forms.
  • Reader failures are E1xx diagnostics with source spans; invalid UTF-8 is rejected at the invocation boundary.

Choose quickly

You wantWriteWhat the reader creates
a namealphaa symbol token that may later be looked up
text"alpha"an immutable string
a proper list as data'(alpha 1)a quoted pair chain ending in ()
a dotted pair as data'(alpha . 1)one pair whose tail is 1
indexed values#(1 2)a vector
bytes#u8(1 2)an immutable bytevector

A common mistake

Radix and exactness prefixes such as #x and #e are outside the current profile.

Current boundaries

  • The language does not silently normalize newlines, paths, or host encodings.

Keep going

Continue with evaluation order to see what happens after reading, or open Reader Grammar for every accepted token shape.

Evaluation order · Reader Grammar