org.eclipse.xtext.ui.editor.model

Class DocumentTokenSource

java.lang.Object

org.eclipse.xtext.ui.editor.model.DocumentTokenSource

Direct Known Subclasses:

BacktrackingLexerDocumentTokenSource

public class DocumentTokenSource
extends Object

For languages with lexers without backtracking. For these we can start lexing from the first damaged token instead of from the beginning. This is a lot faster especially when editing the trunk of big files. As opposed to the original BacktrackingLexerDocumentTokenSource, the damaged region returned by this class always includes the token touching the region the event refers to, i.e. the first token overlapping wit the region or the token that ends with the beginning of the region.

Author:

Jan Koehnlein - Initial contribution and API, Sebastian Zarnekow - Initial contribution and API, Sven Efftinge

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DocumentTokenSource.IRegionIterable
protected static class	DocumentTokenSource.RepairEntryData
static class	DocumentTokenSource.TokenAdapter
static class	DocumentTokenSource.TokenInfo

Constructor Summary

Constructors

Constructor and Description
DocumentTokenSource()

Method Summary

Methods

Modifier and Type	Method and Description
protected org.eclipse.jface.text.IRegion	computeDamageRegion(org.eclipse.jface.text.DocumentEvent e)
protected Lexer	createLexer(String string) Deprecated. use createTokenSource(String) instead.
protected DocumentTokenSource.TokenInfo	createTokenInfo(org.antlr.runtime.CommonToken token)
protected List<DocumentTokenSource.TokenInfo>	createTokenInfos(String string)
protected org.antlr.runtime.TokenSource	createTokenSource(String string)
protected void	doCheckInvariant(org.eclipse.jface.text.DocumentEvent e)
protected List<DocumentTokenSource.TokenInfo>	getInternalModifyableTokenInfos()
org.eclipse.jface.text.IRegion	getLastDamagedRegion()
protected DocumentTokenSource.RepairEntryData	getRepairEntryData(org.eclipse.jface.text.DocumentEvent e)
Iterable<ILexerTokenRegion>	getTokenInfos()
boolean	isCheckInvariant()
void	setCheckInvariant(boolean checkInvariant)
void	setLexer(com.google.inject.Provider<Lexer> lexer)
protected void	setTokens(List<DocumentTokenSource.TokenInfo> infos)
void	updateStructure(org.eclipse.jface.text.DocumentEvent e)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DocumentTokenSource

public DocumentTokenSource()

Method Detail

getTokenInfos

public Iterable<ILexerTokenRegion> getTokenInfos()

getLastDamagedRegion

public org.eclipse.jface.text.IRegion getLastDamagedRegion()

setLexer

public void setLexer(com.google.inject.Provider<Lexer> lexer)

setTokens

protected void setTokens(List<DocumentTokenSource.TokenInfo> infos)

getInternalModifyableTokenInfos

protected List<DocumentTokenSource.TokenInfo> getInternalModifyableTokenInfos()

Since:

2.4

createTokenInfos

protected List<DocumentTokenSource.TokenInfo> createTokenInfos(String string)

createTokenInfo

protected DocumentTokenSource.TokenInfo createTokenInfo(org.antlr.runtime.CommonToken token)

updateStructure

public void updateStructure(org.eclipse.jface.text.DocumentEvent e)

doCheckInvariant

protected void doCheckInvariant(org.eclipse.jface.text.DocumentEvent e)

computeDamageRegion

protected org.eclipse.jface.text.IRegion computeDamageRegion(org.eclipse.jface.text.DocumentEvent e)

getRepairEntryData

protected DocumentTokenSource.RepairEntryData getRepairEntryData(org.eclipse.jface.text.DocumentEvent e)
                                                          throws Exception

Throws:

Exception

Since:

2.4

createLexer

@Deprecated
protected Lexer createLexer(String string)

Deprecated. use createTokenSource(String) instead.

createTokenSource

protected org.antlr.runtime.TokenSource createTokenSource(String string)

Since:

2.4

setCheckInvariant

public void setCheckInvariant(boolean checkInvariant)

isCheckInvariant

public boolean isCheckInvariant()