LexerInterface Class Reference IO

#include <lexer.h>

详细描述

Lexer class. This class provides functionality to scan text files. the stream is tokenized for easy reading.

公共成员函数

MAXON_METHOD Result < void >  Init ( UrlOrInputStream &&fileName, LEXERINITFLAGS flags, const String &stringEscapeCharacters, const StringDecodingRef &stringDecoding)
MAXON_METHOD Result < void >  AddOperator (const String &chars)
MAXON_METHOD Result < void >  关闭 ()
MAXON_METHOD Result < Utf32Char ReadNextChar ()
MAXON_METHOD Result < Bool SkipSpaces ()
MAXON_METHOD Result < Bool SkipLineToEnd ( Bool collectSkipped=false)
MAXON_METHOD Result < LEXERSYMBOL ReadNextSymbol ( LEXERSYMBOLFLAGS flags= LEXERSYMBOLFLAGS::NONE )
MAXON_METHOD Result < void >  ReadNextSymbolExpected ( LEXERSYMBOL expectedSymbols, LEXERSYMBOLFLAGS flags= LEXERSYMBOLFLAGS::NONE )
MAXON_METHOD void  RewindSymbol ()
MAXON_METHOD LEXERSYMBOL   GetSymbol () const
MAXON_METHOD Int   GetLeadingSpace () const
MAXON_METHOD String   GetIdent () const
MAXON_METHOD Result < void >  GetFloat ( Float32 &val) const
MAXON_METHOD Result < void >  GetFloat ( Float64 &val) const
MAXON_METHOD Result < void >  GetInt ( Int32 &val) const
MAXON_METHOD Result < void >  GetInt ( Int64 &val) const
MAXON_METHOD Result < void >  GetUInt ( UInt32 &val) const
MAXON_METHOD Result < void >  GetUInt ( UInt64 &val) const
MAXON_METHOD Result < Int64 GetCurrentReadPosition () const
MAXON_METHOD Result < Int64 GetSymbolPosition () const
MAXON_METHOD Bool   CheckIdent (const Char *cmp) const
MAXON_METHOD Bool   CheckIdent (const String &cmp) const
MAXON_METHOD Int   GetLine () const

静态公共成员函数

static MAXON_METHOD LexerInterface Alloc ( MAXON_SOURCE_LOCATION_DECLARATION )

私有成员函数

  MAXON_INTERFACE_NONVIRTUAL ( LexerInterface , MAXON_REFERENCE_NORMAL , "net.maxon.interface.lexer")

成员函数文档编制

◆  MAXON_INTERFACE_NONVIRTUAL()

MAXON_INTERFACE_NONVIRTUAL ( LexerInterface   ,
MAXON_REFERENCE_NORMAL   ,
"net.maxon.interface.lexer"   
)
private

◆  Alloc()

static MAXON_METHOD LexerInterface * Alloc ( MAXON_SOURCE_LOCATION_DECLARATION   )
static
参数
[in] allocLocation Source location.

◆  Init()

MAXON_METHOD Result <void> Init ( UrlOrInputStream &&  fileName ,
LEXERINITFLAGS   flags ,
const String stringEscapeCharacters ,
const StringDecodingRef &  stringDecoding  
)

Initializes the lexer class. This functions opens the stream.

参数
[in] fileName Url or Input Stream.
[in] flags Control flags for the lexer. See LEXERINITFLAGS for description.
[in] stringEscapeCharacters Defines characters that allow to escape " ' and the escape character itself within strings. By Default this is '\'.
[in] stringDecoding String encoding for this file. If a nullptr is passed (recommended) then StreamConversions::UtfTextDecoder will be used to auto-detect the file encoding.
返回
OK on success.

◆  AddOperator()

MAXON_METHOD Result <void> AddOperator ( const String chars )

Adds an operator symbol to the lexer. This is a symbol which consists of several non-letter characters such as && or +=. The lexer will then return such a character sequence as a single symbol of type LEXERSYMBOL::OPERATOR .

参数
[in] chars The symbol to add.

◆  Close()

MAXON_METHOD Result <void> Close ( )

Closes the stream.

◆  ReadNextChar()

MAXON_METHOD Result < Utf32Char > ReadNextChar ( )

Reads the next character out of the stream.

返回
Returns the character that has been read. This function returns 0 if the end of the file has been reached.

◆  SkipSpaces()

MAXON_METHOD Result < Bool > SkipSpaces ( )

Skips the spaces from the current position.

返回
True if the function was successful. False if the end of the file has been reached.

◆  SkipLineToEnd()

MAXON_METHOD Result < Bool > SkipLineToEnd ( Bool   collectSkipped = false )

Skips line to the end. If the lexer is in NEED_EOL mode the method stops at line end. Otherwise it will advance to the next line and stops when non-whitespace is found.

参数
[in] collectSkipped If true then the skipped bytes are returned with the next GetIdent() call.
返回
True if the function was successful. False if the end of the file has been reached. Otherwise an error.

◆  ReadNextSymbol()

MAXON_METHOD Result < LEXERSYMBOL > ReadNextSymbol ( LEXERSYMBOLFLAGS   flags = LEXERSYMBOLFLAGS::NONE )

Reads the next symbol in the stream. GetIdent() returns the name of the symbol/ident/number/operator.

参数
[in] flags See LEXERSYMBOLFLAGS for details.
返回
Next detected symbol. LEXERSYMBOL::ENDOFFILE if the end of file has been reached.

◆  ReadNextSymbolExpected()

MAXON_METHOD Result <void> ReadNextSymbolExpected ( LEXERSYMBOL   expectedSymbols ,
LEXERSYMBOLFLAGS   flags = LEXERSYMBOLFLAGS::NONE  
)

Reads the next symbol in the stream and checks if it's identical to requiredSymbols. GetIdent() returns the name of the symbol/ident/number/operator.

参数
[in] expectedSymbols Symbol that is required.
[in] flags See LEXERSYMBOLFLAGS for details.
返回
Next detected symbol. LEXERSYMBOL::ENDOFFILE if the end of file has been reached.

◆  RewindSymbol()

MAXON_METHOD void RewindSymbol ( )

Rewinds the current symbol so that the next ReadNextSymbol returns exactly the same as before.

◆  GetSymbol()

MAXON_METHOD LEXERSYMBOL GetSymbol ( ) const

Returns the current symbol.

◆  GetLeadingSpace()

MAXON_METHOD Int GetLeadingSpace ( ) const

Returns the number of leading spaces of the symbol. Tabs count as up to 8 spaces.

◆  GetIdent()

MAXON_METHOD String GetIdent ( ) const

Returns the current String ident.

◆  GetFloat() [1/2]

MAXON_METHOD Result <void> GetFloat ( Float32 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the floating point value of this number.

参数
[out] val Reference to the value.

◆  GetFloat() [2/2]

MAXON_METHOD Result <void> GetFloat ( Float64 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the floating point value of this number.

参数
[in] val Reference to the value.

◆  GetInt() [1/2]

MAXON_METHOD Result <void> GetInt ( Int32 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.

参数
[in] val Reference to the value.

◆  GetInt() [2/2]

MAXON_METHOD Result <void> GetInt ( Int64 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.

参数
[in] val Reference to the value.

◆  GetUInt() [1/2]

MAXON_METHOD Result <void> GetUInt ( UInt32 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.

参数
[in] val Reference to the value.

◆  GetUInt() [2/2]

MAXON_METHOD Result <void> GetUInt ( UInt64 val ) const

In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.

参数
[in] val Reference to the value.

◆  GetCurrentReadPosition()

MAXON_METHOD Result < Int64 > GetCurrentReadPosition ( ) const

Returns the current reading position.

◆  GetSymbolPosition()

MAXON_METHOD Result < Int64 > GetSymbolPosition ( ) const

Returns the position of the current symbol.

返回
The position of the current symbol.

◆  CheckIdent() [1/2]

MAXON_METHOD Bool CheckIdent ( const Char cmp ) const

Checks against the current identifier.

参数
[in] cmp C-style string to compare with.
返回
Returns true if identical.

◆  CheckIdent() [2/2]

MAXON_METHOD Bool CheckIdent ( const String cmp ) const

Checks against the current identifier.

参数
[in] cmp String to compare with.
返回
Returns true if identical.

◆  GetLine()

MAXON_METHOD Int GetLine ( ) const

Returns the current line number within the file.