Welcome, guest | Sign In | My Account | Store | Cart

Notice! PyPM is being replaced with the ActiveState Platform, which enhances PyPM’s build and deploy capabilities. Create your free Platform account to download ActivePython or customize Python with the packages you require and get automatic updates.

Download
ActivePython
INSTALL>
pypm install reflex

How to install reflex

  1. Download and install ActivePython
  2. Open Command Prompt
  3. Type pypm install reflex
 Python 2.7Python 3.2Python 3.3
Windows (32-bit)
0.1 Available View build log
Windows (64-bit)
0.1 Available View build log
Mac OS X (10.5+)
0.1 Available View build log
Linux (32-bit)
0.1 Available View build log
Linux (64-bit)
0.1 Available View build log
 
Author
License
Choice of GPL or Python license
Imports
Lastest release
version 0.1 on Jan 5th, 2011

Reflex: A lightweight lexical scanner library.

Reflex supports regular expressions, rule actions, multiple scanner states, tracking of line/column numbers, and customizable token classes.

Reflex is not a "scanner generator" in the sense of generating source code. Instead, it generates a scanner object dynamically based on the set of input rules sepecified. The rules themselves are ordinary python regular expressions, combined with rule actions which are simply python functions.

Example use:

# Create a scanner. The "start" parameter specifies the name of the # starting state. Note: The state argument can be any hashable python # type. scanner = reflex.scanner( "start" )

# Add some rules. # The whitespace rule has no actions, so whitespace will be skipped scanner.rule( "s+" )

# Rules for identifiers and numbers. TOKEN_IDENT = 1 TOKEN_NUMBER = 2 scanner.rule( "[a-zA-Z_][w_]*", token=TOKEN_IDENT ) scanner.rule( "0x[da-fA-F]+|d+", token=TOKEN_NUMBER )

# The "string" rule kicks us into the string state TOKEN_STRING = 3 scanner.rule( """, tostate="string" )

# Define the string state. "string_escape" and "string_chars" are # action functions which handle the parsed charaxcters and escape # sequences and append them to a buffer. Once a quotation mark # is encountered, we set the token type to be TOKEN_STRING # and return to the start state. scanner.state( "string" ) scanner.rule( """, tostate="start", token=TOKEN_STRING ) scanner.rule( "\.", string_escape ) scanner.rule( "[^"\]+", string_text )

Invoking the scanner: The scanner can be called as a function which takes a reference to a stream (such as a file object) which iterates over input lines. The "context" argument is for application use, The result is an iterator which produces a series of tokens. The same scanner can be used to parse multiple input files, by creating a new stream for each file.

# Return an instance of the scanner. token_iter = scanner( istream, context )

Getting the tokens. Here is a simple example of looping through the input tokens. A real-world use would most likely involve comparing vs. the type of the current token.

# token.id is the token type (the same as the token= argument in the rule) # token.value is the actual characters that make up the token. # token.line is the line number on which the token was encountered. # token.pos is the column number of the first character of the token. for token in token_iter: print token.id, token.value, token.line, token.pos

Action functions are python functions which take a single argument, which is the token stream instance.

# Action function to handle striing text. # Appends the value of the current token to the string data def string_text( token_stream ): string_data += scanner.token.value

The token_stream object has a number of interesting and usable attributes:

states: dictionary of scanner states state: the current state stream: the input line stream context: the context pointer that was passed to the scanner token: the current token line: the line number of the current parse position pos: the column number of the current parse position

Note - reflex currently has a limit of 99 rules for each state. (That is the maximum number of capturing groups allowed in a python regular expression.)

Docutils System Messages

System Message: ERROR/3 (<string>, line 22); backlink

Unknown target name: "a-za-z".

Subscribe to package updates

Last updated Jan 5th, 2011

Download Stats

Last month:3

What does the lock icon mean?

Builds marked with a lock icon are only available via PyPM to users with a current ActivePython Business Edition subscription.

Need custom builds or support?

ActivePython Enterprise Edition guarantees priority access to technical support, indemnification, expert consulting and quality-assured language builds.

Plan on re-distributing ActivePython?

Get re-distribution rights and eliminate legal risks with ActivePython OEM Edition.