Abstract Syntax Trees

Source code is always understood as sequences of tokens. These tokens are extracted from source code by tokenizers. Token sequences define structures which are referred to as productions and which are specified in grammars. Specifications of the all tokens and productions in source code are referred to as abstract syntax trees. Programs that create abstrax syntax trees are referred to as parsers. Extracting abstrax syntax trees from source code with parsers is referred to as parsing.

To see the implemenation of the tokenizer and the token specifications, click here .

To see the grammar, click here .

To see the script that accepts source code and creates all the corresponding layers, including the abstract syntax layer, click here .

Home