Scanning
--------

.. glossary::

    keyword
        One of the literal words ``bool``, :crowbar:ref:`break`, ``case``,
        ``char``, ``const``, ``continue``, ``default``, ``do``, ``double``,
        ``else``, ``enum``, ``extern``, ``float``, ``for``, ``fragile``,
        ``function``, ``if``, ``include``, ``int``, ``long``, ``return``,
        ``short``, ``signed``, ``sizeof``, ``struct``, ``switch``,
        ``unsigned``, ``void``, or ``while``.
    
    identifier
        A nonempty sequence of characters blah blah blah

        .. todo::

            figure out https://www.unicode.org/reports/tr31/tr31-33.html

    decimal constant
        A sequence of characters matching the regular expression ``[0-9_]+``.
        Denotes the numeric value of the given sequence of decimal digits.
        Underscores are ignored by the compiler, but may be useful separators for other readers.
    
    binary constant
        A sequence of characters matching the regular expression ``0[bB][01_]+``.
        Denotes the numeric value of the given sequence of binary digits (after the ``0[bB]`` prefix has been removed).
        Underscores are ignored by the compiler, but may be useful separators for other readers.
    
    octal constant
        A sequence of characters matching the regular expression ``0o[0-7_]+``.
        Denotes the numeric value of the given sequence of octal digits (after the ``0o`` prefix has been removed).
        Underscores are ignored by the compiler, but may be useful separators for other readers.

    token
        A single atomic unit in a Crowbar source file.
        Has one (and exactly one) of the following types.

.. todo::

    finish transcribing token definitions