Class Tokenizer
- java.lang.Object
-
- org.openstreetmap.josm.tools.template_engine.Tokenizer
-
public class Tokenizer extends Object
This class converts a template string (stream of characters) into a stream of tokens. The result of the tokenization (also called lexical analysis) serves as input for the parserTemplateParser
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Tokenizer.Token
static class
Tokenizer.TokenType
-
Field Summary
Fields Modifier and Type Field Description private int
c
private Tokenizer.Token
currentToken
private int
index
private Set<Character>
specialCharacters
private String
template
private StringBuilder
text
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description private void
getChar()
Tokenizer.Token
lookAhead()
Tokenizer.Token
nextToken()
Tokenizer.Token
skip(char lastChar)
-
-
-
Field Detail
-
specialCharacters
private final Set<Character> specialCharacters
-
c
private int c
-
index
private int index
-
currentToken
private Tokenizer.Token currentToken
-
text
private final StringBuilder text
-
-
Method Detail
-
getChar
private void getChar()
-
nextToken
public Tokenizer.Token nextToken() throws ParseError
- Throws:
ParseError
-
lookAhead
public Tokenizer.Token lookAhead() throws ParseError
- Throws:
ParseError
-
skip
public Tokenizer.Token skip(char lastChar)
-
-