com.intellij.lexer
Interface Lexer

All Known Implementing Classes:

CompositeLexer, EmptyLexer, FilterLexer, FlexAdapter, JavaDocLexer, LayeredLexer, LexerBase, MergingLexerAdapter, StringLiteralLexer

public interface Lexer

Interface for breaking a file into a sequence of tokens.

Method Summary

void	advance() Advances the lexer to the next token.
char[]	getBuffer() Returns the buffer over which the lexer is running.
int	getBufferEnd() Returns the offset at which the lexer will stop lexing.
LexerPosition	getCurrentPosition() Returns the current position and state of the lexer.
int	getState() Returns the current state of the lexer.
int	getTokenEnd() Returns the end offset of the current token.
int	getTokenStart() Returns the start offset of the current token.
IElementType	getTokenType() Returns the token at the current position of the lexer.
void	restore(LexerPosition position) Restores the lexer to the specified state and position.
void	start(char[] buffer) Prepare for lexing character data from buffer passed.
void	start(char[] buffer, int startOffset, int endOffset) Prepare for lexing character data from buffer passed.
void	start(char[] buffer, int startOffset, int endOffset, int initialState) Prepare for lexing character data from buffer passed.

Method Detail

start

void start(char[] buffer)

Prepare for lexing character data from buffer passed. Lexing should be performed starting from offset 0 and terminated (EOFed) at offset buffer.length. Internal lexer state is supposed to be initial. YY_INITIAL for JLex and JFlex generated lexer.

Parameters:

buffer - character data for lexing.

start

void start(char[] buffer,
           int startOffset,
           int endOffset)

Prepare for lexing character data from buffer passed. Lexing should be performed starting from offset startOffset and terminated (EOFed) at offset endOffset. Internal lexer state is supposed to be initial. YY_INITIAL for JLex and JFlex generated lexer.

Parameters:

buffer - character data for lexing.

startOffset - offset to start lexing from

endOffset - offset to stop lexing at

start

void start(char[] buffer,
           int startOffset,
           int endOffset,
           int initialState)

Prepare for lexing character data from buffer passed. Lexing should be performed starting from offset startOffset> and terminated (EOFed) at offset endOffset. Internal lexer state is supposed to be initialState. It is guaranteed that the value of initialState has been returned by getState() method of this Lexer at condition startOffset=getTokenStart() This method is used to incrementally relex changed characters using lexing data acquired from this particular lexer sometime in the past.

Parameters:

buffer - character data for lexing.

startOffset - offset to start lexing from

endOffset - offset to stop lexing at

initialState - the initial state of the lexer.

getState

int getState()

Returns the current state of the lexer.

Returns:

the lexer state.

getTokenType

IElementType getTokenType()

Returns the token at the current position of the lexer.

Returns:

the current token.

getTokenStart

int getTokenStart()

Returns the start offset of the current token.

Returns:

the current token start offset.

getTokenEnd

int getTokenEnd()

Returns the end offset of the current token.

Returns:

the current token end offset.

advance

void advance()

Advances the lexer to the next token.

getCurrentPosition

LexerPosition getCurrentPosition()

Returns the current position and state of the lexer.

Returns:

the lexer position and state.

restore

void restore(LexerPosition position)

Restores the lexer to the specified state and position.

Parameters:

position - the state and position to restore to.

getBuffer

char[] getBuffer()

Returns the buffer over which the lexer is running. This method should return the same buffer instance which was passed to the start() method.

Returns:

the lexer buffer.

getBufferEnd

int getBufferEnd()

Returns the offset at which the lexer will stop lexing. This method should return the length of the buffer or the value passed in the endOffset parameter to the start() method.

Returns:

the lexing end offset