Tokenizer class that breaks text into tokens. More...

#include <style_tokenizer.h>

Public Member Functions

 StyleTokenizer (const std::string &text)
 Construct tokenizer for a text string.
 
void read (StyleToken &out_token, bool eat_whitespace, bool eat_comments=true)
 Reads the next token.
 
std::vector< StyleTokenread_property_value (StyleToken &token, bool &out_important_flag)
 Reads all tokens belonging to a property value.
 

Static Public Member Functions

static std::vector< StyleTokentokenize (const std::string &text)
 Returns all tokens except start/end whitespace and comments.
 

Detailed Description

Tokenizer class that breaks text into tokens.

Constructor & Destructor Documentation

◆ StyleTokenizer()

clan::StyleTokenizer::StyleTokenizer ( const std::string & text)

Construct tokenizer for a text string.

Member Function Documentation

◆ read()

void clan::StyleTokenizer::read ( StyleToken & out_token,
bool eat_whitespace,
bool eat_comments = true )

Reads the next token.

◆ read_property_value()

std::vector< StyleToken > clan::StyleTokenizer::read_property_value ( StyleToken & token,
bool & out_important_flag )

Reads all tokens belonging to a property value.

◆ tokenize()

static std::vector< StyleToken > clan::StyleTokenizer::tokenize ( const std::string & text)
static

Returns all tokens except start/end whitespace and comments.


The documentation for this class was generated from the following file: