Package org.eclipse.xtext.parser.antlr

Class XtextTokenStream

  • All Implemented Interfaces:
    org.antlr.runtime.IntStream, org.antlr.runtime.TokenStream
    public class XtextTokenStream
extends org.antlr.runtime.CommonTokenStream
    A token stream that is aware of the current lookahead.

    • Field Summary

      • Fields inherited from class org.antlr.runtime.CommonTokenStream

        channel, channelOverrideMap, discardOffChannelTokens, discardSet, lastMarker, p, tokens, tokenSource

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected void basicConsume()
      Make super-impl of consume() accessible to subtypes.
      protected int basicMark()
      Make super-impl of mark() accessible to subtypes.
      protected void basicRewind​(int marker)
      Make super-impl of CommonTokenStream.rewind() accessible to subtypes.
      void consume()  
      protected void doSetHiddenTokens​(java.lang.String... lexerRules)  
      int getCurrentLookAhead()  
      protected int getFirstMarker()  
      java.lang.String getLexerErrorMessage​(org.antlr.runtime.Token invalidToken)  
      protected int getTokenIndex​(org.antlr.runtime.Token tok)  
      void initCurrentLookAhead​(int currentLookAhead)  
      int LA​(int i)  
      org.antlr.runtime.Token LT​(int k)
      Same as CommonTokenStream.LT(int) except that we skip hidden tokens even for k == 1.
      int mark()  
      void rewind​(int marker)  
      XtextTokenStream.HiddenTokens setHiddenTokens​(java.lang.String... lexerRules)  
      void setInitialHiddenTokens​(java.lang.String... lexerRules)  
      protected void skipHiddenTokens()  
      java.lang.String toString​(int start, int stop)  

      • Methods inherited from class org.antlr.runtime.CommonTokenStream

        discardOffChannelTokens, discardTokenType, fillBuffer, get, getSourceName, getTokens, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LB, release, reset, rewind, seek, setTokenSource, setTokenTypeChannel, size, skipOffTokenChannels, skipOffTokenChannelsReverse, toString, toString

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • XtextTokenStream

        public XtextTokenStream()

      • XtextTokenStream

        public XtextTokenStream​(org.antlr.runtime.TokenSource tokenSource,
                        int channel)

      • XtextTokenStream

        public XtextTokenStream​(org.antlr.runtime.TokenSource tokenSource,
                        ITokenDefProvider tokenDefProvider)

    • Method Detail

      • toString

        public java.lang.String toString​(int start,
                                 int stop)
        Specified by:
        toString in interface org.antlr.runtime.TokenStream
        Overrides:
        toString in class org.antlr.runtime.CommonTokenStream

      • setInitialHiddenTokens

        public void setInitialHiddenTokens​(java.lang.String... lexerRules)

      • doSetHiddenTokens

        protected void doSetHiddenTokens​(java.lang.String... lexerRules)

      • skipHiddenTokens

        protected void skipHiddenTokens()

      • LA

        public int LA​(int i)
        Specified by:
        LA in interface org.antlr.runtime.IntStream
        Overrides:
        LA in class org.antlr.runtime.CommonTokenStream

      • getTokenIndex

        protected int getTokenIndex​(org.antlr.runtime.Token tok)
        Since:
        2.22

      • consume

        public void consume()
        Specified by:
        consume in interface org.antlr.runtime.IntStream
        Overrides:
        consume in class org.antlr.runtime.CommonTokenStream

      • basicConsume

        protected void basicConsume()
        Make super-impl of consume() accessible to subtypes.
        Since:
        2.22

      • getFirstMarker

        protected int getFirstMarker()
        Since:
        2.1

      • mark

        public int mark()
        Specified by:
        mark in interface org.antlr.runtime.IntStream
        Overrides:
        mark in class org.antlr.runtime.CommonTokenStream

      • basicMark

        protected int basicMark()
        Make super-impl of mark() accessible to subtypes.
        Since:
        2.22

      • rewind

        public void rewind​(int marker)
        Specified by:
        rewind in interface org.antlr.runtime.IntStream
        Overrides:
        rewind in class org.antlr.runtime.CommonTokenStream

      • basicRewind

        protected void basicRewind​(int marker)
        Make super-impl of CommonTokenStream.rewind() accessible to subtypes.
        Since:
        2.22

      • LT

        public org.antlr.runtime.Token LT​(int k)
        Same as CommonTokenStream.LT(int) except that we skip hidden tokens even for k == 1.
        Specified by:
        LT in interface org.antlr.runtime.TokenStream
        Overrides:
        LT in class org.antlr.runtime.CommonTokenStream

      • getCurrentLookAhead

        public int getCurrentLookAhead()

      • initCurrentLookAhead

        public void initCurrentLookAhead​(int currentLookAhead)

      • getLexerErrorMessage

        public java.lang.String getLexerErrorMessage​(org.antlr.runtime.Token invalidToken)