org.jruby.lexer.yacc

Class RubyLexer

java.lang.Object

org.jruby.lexer.LexingCommon

org.jruby.lexer.yacc.RubyLexer

public class RubyLexer
extends LexingCommon

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RubyLexer.Keyword

Field Summary

Fields inherited from class org.jruby.lexer.LexingCommon

__end__seen, AMPERSAND, AMPERSAND_AMPERSAND, AMPERSAND_DOT, ASCII8BIT_ENCODING, BACKSLASH, BACKTICK, BANG, BANG_EQ, BANG_TILDE, BEGIN_DOC_MARKER, braceNest, CALL, CARET, cmdArgumentState, CODING, COLON, COLON_COLON, COMMA, commandStart, conditionState, DOLLAR_BANG, DOLLAR_DOT, DOLLAR_UNDERSCORE, DOT, DOT_DOT, DOT_DOT_DOT, end, END_DOC_MARKER, END_MARKER, EOF, eofp, EQ, EQ_EQ, EQ_EQ_EQ, EQ_GT, EQ_TILDE, EXPR_ARG, EXPR_ARG_ANY, EXPR_BEG, EXPR_BEG_ANY, EXPR_CLASS, EXPR_CMDARG, EXPR_DOT, EXPR_END, EXPR_END_ANY, EXPR_ENDARG, EXPR_ENDFN, EXPR_FITEM, EXPR_FNAME, EXPR_LABEL, EXPR_LABELED, EXPR_MID, EXPR_VALUE, GT, GT_EQ, GT_GT, has_shebang, heredoc_end, heredoc_indent, heredoc_line_indent, KWNOREST, last_cr_line, last_state, LBRACKET, LBRACKET_RBRACKET, LBRACKET_RBRACKET_EQ, LCURLY, lex_lastline, lex_nextline, lex_p, lex_pbeg, lex_pend, lex_state, lexb, line_count, line_offset, LT, LT_EQ, LT_EQ_RT, LT_LT, MINUS, MINUS_AT, MINUS_GT, NIL, OR, OR_OR, parenNest, PERCENT, PLUS, PLUS_AT, Q, QQ, QUESTION, RBRACKET, RCURLY, RPAREN, ruby_sourceline, SEMICOLON, SLASH, src, STAR, STAR_STAR, start, str_dquote, str_dsym, str_dword, STR_FUNC_ESCAPE, STR_FUNC_EXPAND, STR_FUNC_INDENT, STR_FUNC_LABEL, STR_FUNC_LIST, STR_FUNC_QWORDS, STR_FUNC_REGEXP, STR_FUNC_SYMBOL, STR_FUNC_TERM, str_label, str_regexp, str_squote, str_ssym, str_sword, str_xquote, SUFFIX_ALL, SUFFIX_I, SUFFIX_R, TAB_WIDTH, TILDE, token, tokenSeen, tokline, tokp, USASCII_ENCODING, UTF8_ENCODING, yaccValue

Constructor Summary

Constructors

Constructor and Description
RubyLexer(RubyParserBase parser, LexerSource source, IRubyWarnings warnings)

Method Summary

Modifier and Type	Method and Description
protected void	ambiguousOperator(java.lang.String op, java.lang.String syn)
protected int	asTruth(java.lang.String name, ByteList value)
void	compile_error(java.lang.String message)
void	compile_error(java.lang.String message, long start, long end)
StrNode	createStr(ByteList buffer, int flags)
static RubyLexer.Keyword	getKeyword(ByteList str)
static RubyLexer.Keyword	getKeyword(java.lang.String str)
Ruby	getRuntime()
StrTerm	getStrTerm()
void	heredoc_dedent(Node root)
void	heredoc_restore(HeredocTerm here)
protected void	mismatchedRegexpEncodingError(org.jcodings.Encoding optionEncoding, org.jcodings.Encoding encoding)
int	nextc()
int	nextToken()
void	parse_error(java.lang.String message)
protected RegexpOptions	parseRegexpFlags()
void	reset()
protected void	setCompileOptionFlag(java.lang.String name, ByteList value)
protected void	setEncoding(ByteList name)
void	setStrTerm(StrTerm strterm)
protected void	setTokenInfo(java.lang.String name, ByteList value)
void	setWarnings(IRubyWarnings warnings)
int	tokenize_ident(int result)

Methods inherited from class org.jruby.lexer.LexingCommon

checkRegexpFragment, checkRegexpSyntax, column, comment_at_top, compile_error_pos, createAsEncodedString, createTokenByteList, createTokenByteList, createTokenString, createTokenString, dedent_string, flush, getBraceNest, getCmdArgumentState, getConditionState, getCurrentArg, getCurrentLine, getEncoding, getFile, getHeredocIndent, getHeredocLineIndent, getLeftParenBegin, getLexContext, getLineOffset, getParenNest, getRubySourceline, getState, getTokenCR, id, incrementParenNest, isAfterOperator, isARG, isASCII, isASCII, isBEG, isEND, isEndSeen, isGlobalCharPunct, isHexChar, isIdentifierChar, isLabelPossible, isLabelSuffix, isLambdaBeginning, isLexState, isLexStateAll, isLookingAtEOL, isNext_identchar, isOctChar, isSpace, isSpaceArg, lex_goto_eol, lineno, magicCommentEncoding, magicCommentMarker, newtok, numberLiteralSuffix, onMagicComment, optionsEncodingChar, p, parser_magic_comment, parser_prepare, parseRegexpFlags, peek, peek, peekVariableName, precise_mbclen, printState, pushback, readEscape, readUTF8EscapeIntoBuffer, readUTFEscape, readUTFEscapeRegexpLiteral, resetStacks, scanHex, scanHexLiteral, scanOct, set_file_encoding, set_yylval_id, set_yylval_name, setBraceNest, setCurrentArg, setCurrentEncoding, setEncoding, setHeredocIndent, setHeredocLineIndent, setLeftParenBegin, setRegexpEncoding, setRubySourceline, setSource, setState, setValue, strncmp, tokadd_ident, tokadd_mbchar, tokadd_mbchar, tokAdd, tokaddmbc, tokCopy, token, tokHex, update_heredoc_indent, updateStartPosition, updateTokenPosition, validateFormalIdentifier, validateFormalIdentifier, value, warn_balanced, was_bol, whole_match_p

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RubyLexer

public RubyLexer(RubyParserBase parser,
                 LexerSource source,
                 IRubyWarnings warnings)

Method Detail

ambiguousOperator

protected void ambiguousOperator(java.lang.String op,
                                 java.lang.String syn)

Specified by:

ambiguousOperator in class LexingCommon

getKeyword

public static RubyLexer.Keyword getKeyword(ByteList str)

getKeyword

public static RubyLexer.Keyword getKeyword(java.lang.String str)

getRuntime

public Ruby getRuntime()

tokenize_ident

public int tokenize_ident(int result)

Specified by:

tokenize_ident in class LexingCommon

reset

public void reset()

Overrides:

reset in class LexingCommon

nextc

public int nextc()

Specified by:

nextc in class LexingCommon

heredoc_dedent

public void heredoc_dedent(Node root)

compile_error

public void compile_error(java.lang.String message,
                          long start,
                          long end)

compile_error

public void compile_error(java.lang.String message)

Specified by:

compile_error in class LexingCommon

parse_error

public void parse_error(java.lang.String message)

Specified by:

parse_error in class LexingCommon

heredoc_restore

public void heredoc_restore(HeredocTerm here)

nextToken

public int nextToken()
              throws java.io.IOException

Throws:

java.io.IOException

setCompileOptionFlag

protected void setCompileOptionFlag(java.lang.String name,
                                    ByteList value)

Specified by:

setCompileOptionFlag in class LexingCommon

parseRegexpFlags

protected RegexpOptions parseRegexpFlags()
                                  throws java.io.IOException

Specified by:

parseRegexpFlags in class LexingCommon

Throws:

java.io.IOException

mismatchedRegexpEncodingError

protected void mismatchedRegexpEncodingError(org.jcodings.Encoding optionEncoding,
                                             org.jcodings.Encoding encoding)

Specified by:

mismatchedRegexpEncodingError in class LexingCommon

asTruth

protected int asTruth(java.lang.String name,
                      ByteList value)

setTokenInfo

protected void setTokenInfo(java.lang.String name,
                            ByteList value)

Specified by:

setTokenInfo in class LexingCommon

setEncoding

protected void setEncoding(ByteList name)

Specified by:

setEncoding in class LexingCommon

getStrTerm

public StrTerm getStrTerm()

setStrTerm

public void setStrTerm(StrTerm strterm)

setWarnings

public void setWarnings(IRubyWarnings warnings)

createStr

public StrNode createStr(ByteList buffer,
                         int flags)