parsing - Antlr symbols - Need examples -

i looking @ code base antlr used , many grammars defined. see following being used not clear mean

1. token {one, two}

• i dont see 1 or 2 defined anywhere.

2. symbols -> , ^

• whats purpose of these? see these in both lexer , parser rules

3. symbol []

• this used in following

token {set}

id : ...

token1 : .. set[$id]

what mean? examples of great help.

these items if come antlr3 grammar. first part tokens section (note plural), this:

tokens {   one,   2 } 

which defines number of "virtual" tokens. these called "virtual" because have no input match , used in tree rewriting (e.g. changing tokens type depending on conditions, e.g. outcome of predicate).

the symbols -> , ^ used tree rewriting when generating ast (and no longer supported antlr4 btw, because doesn't produce ast @ all, parse tree). ^ denotes root node, causes parser create tree of current tokens in active rule , use marked token root of tree. -> allows rewrite result more (different order, add virtual tokens, leave out tokens etc.).

and last part of question demonstrates use of token manipulation. syntax causes parser create new token set values (except type, e.g. source position, channel etc.) matched id token. it's part of rewriting machinery.

update: antlr3 homepage still alive , has wiki documentation: https://theantlrguy.atlassian.net/wiki/display/antlr3/antlr+3+wiki+home (or via http://www.antlr3.org).