antlr - Retrive token list from ParserRuleContext -
main question
is there easy way list of tokens (ideally in form of tokenstream
) parser rule class parserrulecontext
?
related answers
in answer question traversal of tokens using parserrulecontext in listener - antlr4 solution appeared:
parserrulecontext pctx = ctx.getparent(); list<terminalnode> nodes = pctx.gettokens(pctx.getstart(), pctx.getstop());
but there no method signature parserrulecontext::gettokens(token, token)
in antlrv4.
my solution
i thought retriving list of tokens tokenstream
using tokenstream:get(index: int)
method, index
value set range of indicies of given parserrulecontext
start/stop tokens.
additional question
is there way subset of tokens tokenstream
in form of tokenstream
?
so, overlooked classes , inferfaces in antlrv4 api.
answer main question
proposed above solution correct. bufferedtokenstream
, commontokenstream
classes have method public list<token> gettokens(int start, int stop)
allows retrive list of tokens given range (especially range between start token , stop token of parserrulecontext
class)
answer additional question
you can utilize listtokensource
class implements tokensource
interface. can create commontokenstream
class passing listtokensource
.
code example of parserrulerewriter
i encapsulate above ideas small code example featuring parserrulerewriter
- tokenstreamrewriter
rewrites given parser rule. in code tokenstream
parameter token stream of full program.
import org.antlr.v4.runtime.*; import java.util.list; public class parserrulerewriter { private tokenstreamrewriter rewriter; public parserrulerewriter(parserrulecontext parserrule, commontokenstream tokenstream) { token start = parserrule.getstart(); token stop = parserrule.getstop(); list<token> ruletokens = tokenstream.gettokens(start.gettokenindex(), stop.gettokenindex()); listtokensource tokensource = new listtokensource(ruletokens); commontokenstream commontokenstream = new commontokenstream(tokensource); commontokenstream.fill(); rewriter = new tokenstreamrewriter(commontokenstream); } public void replace(token token, parserrulecontext rule) { rewriter.replace(token, rule.gettext()); } @override public string tostring() { return rewriter.gettext(); } }
Comments
Post a Comment