lexer.h File Reference

Classes

class   LexerInterface

Namespaces

  maxon

Enumerations

enum   LEXERSYMBOL : UChar {
   NONE ,
   ILLEGAL ,
   ENDOFFILE ,
   IDENTIFIER ,
   STRING ,
   STRINGCHR ,
   NUMBER ,
   NEWLINE ,
   CURLYBRACKET_OPEN ,
   CURLYBRACKET_CLOSE ,
   PARENTHESE_OPEN ,
   PARENTHESE_CLOSE ,
   SQUAREBRACKET_OPEN ,
   SQUAREBRACKET_CLOSE ,
   SLASH ,
   EQUAL ,
   COMMA ,
   SEMICOLON ,
   COLON ,
   PLUS ,
   TILDE ,
   PREPROCESSOR ,
   MULTIPLY ,
   PIPE ,
   AMPERSAND ,
   OPERATOR
}
enum   LEXERSYMBOLFLAGS {
   NONE ,
   NO_NUMBER ,
   ALLOWSTRINGS ,
   MINUS_OPERATOR
}
enum   LEXERINITFLAGS {
   NONE ,
   NEED_EOL ,
   REWIND_WHEN_DONE ,
   ALLOW_DOLLAR_IDENT ,
   ALLOW_AT_IDENT ,
   NON_ASCII_IS_IDENT ,
   HASH_COMMENTS ,
   ESCAPE_IN_SINGLE_QUOTED ,
   C_ESCAPE_SEQUENCES ,
   SPACESCOMPATIBILITY ,
   C4DSTRCOMPATIBILITY
}

函数

enum maxon::LEXERSYMBOL UChar   MAXON_ENUM_LIST (LEXERSYMBOL)
enum maxon::LEXERSYMBOLFLAGS   MAXON_ENUM_FLAGS (LEXERSYMBOLFLAGS)
enum maxon::LEXERINITFLAGS   MAXON_ENUM_FLAGS (LEXERINITFLAGS)

变量

  NONE
  ILLEGAL
  ENDOFFILE
  IDENTIFIER
  STRING
  STRINGCHR
  NUMBER
  NEWLINE
  CURLYBRACKET_OPEN
  CURLYBRACKET_CLOSE
  PARENTHESE_OPEN
  PARENTHESE_CLOSE
  SQUAREBRACKET_OPEN
  SQUAREBRACKET_CLOSE
  SLASH
  EQUAL
  COMMA
  SEMICOLON
  COLON
  PLUS
  TILDE
  PREPROCESSOR
  MULTIPLY
  PIPE
  AMPERSAND
  OPERATOR
  NO_NUMBER
  ALLOWSTRINGS
  MINUS_OPERATOR
  NEED_EOL
  REWIND_WHEN_DONE
  ALLOW_DOLLAR_IDENT
  ALLOW_AT_IDENT
  NON_ASCII_IS_IDENT
  HASH_COMMENTS
  ESCAPE_IN_SINGLE_QUOTED
  C_ESCAPE_SEQUENCES
  SPACESCOMPATIBILITY
  C4DSTRCOMPATIBILITY

Variable Documentation

◆  NONE

NONE

◆  ILLEGAL

ILLEGAL

◆  ENDOFFILE

ENDOFFILE

The end of file or stream was reached.

◆  IDENTIFIER

IDENTIFIER

An arbitrary symbol that is neither a string, nor a value. It may contain unicode characters.

◆  STRING

STRING

A double-quoted string.

◆  STRINGCHR

STRINGCHR

A single-quoted string.

◆  NUMBER

NUMBER

A numerical value.

◆  NEWLINE

NEWLINE

'\r', '\n' or a combination of both.

◆  CURLYBRACKET_OPEN

CURLYBRACKET_OPEN

'{' symbol.

◆  CURLYBRACKET_CLOSE

CURLYBRACKET_CLOSE

'}' symbol.

◆  PARENTHESE_OPEN

PARENTHESE_OPEN

'(' symbol.

◆  PARENTHESE_CLOSE

PARENTHESE_CLOSE

')' symbol.

◆  SQUAREBRACKET_OPEN

SQUAREBRACKET_OPEN

'[' symbol.

◆  SQUAREBRACKET_CLOSE

SQUAREBRACKET_CLOSE

']' symbol.

◆  SLASH

SLASH

'/' symbol.

◆  EQUAL

EQUAL

'=' symbol.

◆  COMMA

COMMA

',' symbol.

◆  SEMICOLON

SEMICOLON

';' symbol.

◆  COLON

COLON

':' symbol.

◆  PLUS

PLUS

'+' symbol.

◆  TILDE

TILDE

'~' symbol.

◆  PREPROCESSOR

PREPROCESSOR

'#' symbol.

◆  MULTIPLY

MULTIPLY

'*' symbol.

◆  PIPE

PIPE

'|' symbol.

◆  AMPERSAND

AMPERSAND

'&' symbol.

◆  OPERATOR

OPERATOR

< One of the operator symbols added by LexerInterface::AddOperator.

◆  NO_NUMBER

NO_NUMBER

If set, numbers (and characters as '.' or '-') are not recognized as numbers. Instead either an LEXERSYMBOL::IDENTIFIER or LEXERSYMBOL::ILLEGAL will be generated (and GetIdent() returns the symbol).

◆  ALLOWSTRINGS

ALLOWSTRINGS

If set, strings starting with " or ' will be detected and LEXERSYMBOL::STRING is returned. GetIdent() returns the content of the string.

◆  MINUS_OPERATOR

MINUS_OPERATOR

If set, the minus symbol in front of a number will be reported as own token.

◆  NEED_EOL

NEED_EOL

The lexer stops at each end of line and returns LEXERSYMBOL::OPERATOR_NEWLINE whenever a newline appears.

◆  REWIND_WHEN_DONE

REWIND_WHEN_DONE

The lexer will rewind the stream to the last interpreted character when it is done. Only set this if you want to continue scanning the open stream. It will only work if the stream supports seeking. Also a string decoding that has an exact counterpart (e.g. StringEncodings::Utf8) needs to be passed to the lexer.

◆  ALLOW_DOLLAR_IDENT

ALLOW_DOLLAR_IDENT

Allow $ as part of identifiers.

◆  ALLOW_AT_IDENT

ALLOW_AT_IDENT

Allow @ as part of identifiers (but not as start).

◆  NON_ASCII_IS_IDENT

NON_ASCII_IS_IDENT

If set, non-ASCII characters (code > 127) are considered to be part of identifiers.

◆  HASH_COMMENTS

HASH_COMMENTS

If set, comments are introduced by the hash character (instead of C++-like comments).

◆  ESCAPE_IN_SINGLE_QUOTED

ESCAPE_IN_SINGLE_QUOTED

If set, escape characters are also considered in single-quoted-strings.

◆  C_ESCAPE_SEQUENCES

C_ESCAPE_SEQUENCES

If set, a subset of C escape sequences is recognized in strings (\n, \t and \r).

◆  SPACESCOMPATIBILITY

SPACESCOMPATIBILITY

If set, a newline character will be included in the result of LexerInterface::GetLeadingSpaces.

◆  C4DSTRCOMPATIBILITY

C4DSTRCOMPATIBILITY

If set, the lexer will handle the escape character in a special manner. Then the escape character works only for the following character if it matches ".