Welcome, guest | Sign In | My Account | Store | Cart

Evaluate constant expressions, including list, dict and tuple using the abstract syntax tree created by compiler.parse. Since compiler does the work, handling arbitratily nested structures is transparent, and the implemenation is very straightforward.

Python, 61 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import compiler

class Unsafe_Source_Error(Exception):
    def __init__(self,error,descr = None,node = None):
        self.error = error
        self.descr = descr
        self.node = node
        self.lineno = getattr(node,"lineno",None)
        
    def __repr__(self):
        return "Line %d.  %s: %s" % (self.lineno, self.error, self.descr)
    __str__ = __repr__    
           
class SafeEval(object):
    
    def visit(self, node,**kw):
        cls = node.__class__
        meth = getattr(self,'visit'+cls.__name__,self.default)
        return meth(node, **kw)
            
    def default(self, node, **kw):
        for child in node.getChildNodes():
            return self.visit(child, **kw)
            
    visitExpression = default
    
    def visitConst(self, node, **kw):
        return node.value

    def visitDict(self,node,**kw):
        return dict([(self.visit(k),self.visit(v)) for k,v in node.items])
        
    def visitTuple(self,node, **kw):
        return tuple(self.visit(i) for i in node.nodes)
        
    def visitList(self,node, **kw):
        return [self.visit(i) for i in node.nodes]

class SafeEvalWithErrors(SafeEval):

    def default(self, node, **kw):
        raise Unsafe_Source_Error("Unsupported source construct",
                                node.__class__,node)
            
    def visitName(self,node, **kw):
        raise Unsafe_Source_Error("Strings must be quoted", 
                                 node.name, node)
                                 
    # Add more specific errors if desired
            

def safe_eval(source, fail_on_error = True):
    walker = fail_on_error and SafeEvalWithErrors() or SafeEval()
    try:
        ast = compiler.parse(source,"eval")
    except SyntaxError, err:
        raise
    try:
        return walker.visit(ast)
    except Unsafe_Source_Error, err:
        raise

The topic of how to parse a constant list or dictionary safely crops up fairly regularly, e.g.

Examples (from recent c.l.py thread):

>>> goodsource =  """[1, 2, 'Joe Smith', 8237972883334L,   # comment
...       {'Favorite fruits': ['apple', 'banana', 'pear']},  # another comment
...       'xyzzy', [3, 5, [3.14159, 2.71828, []]]]"""
...

Providing the source contains only constants and arbitrarily nested list/dict/tuple, the result is the same as built-in eval:

safe_eval(good_source) [1, 2, 'Joe Smith', 8237972883334L, {'Favorite fruits': ['apple', 'banana', 'pear']}, 'xyzzy', [3, 5, [3.1415899999999999, 2.71828, []]]]

>>> assert _ == eval(good_source)

But this should fail, due to unquoted string literal

>>> badsource = """[1, 2, JoeSmith, 8237972883334L,   # comment
...       {'Favorite fruits': ['apple', 'banana', 'pear']},  # another comment
...       'xyzzy', [3, 5, [3.14159, 2.71828, []]]]"""
...

safe_eval(bad_source) Traceback (most recent call last): [...] Unsafe_Source_Error: Line 1. Strings must be quoted: JoeSmith

Alternatively, ignore the unsafe content and parse the rest:

safe_eval(bad_source, fail_on_error = False) [1, 2, None, 8237972883334L, {'Favorite fruits': ['apple', 'banana', 'pear']}, 'xyzzy', [3, 5, [3.1415899999999999, 2.71828, []]]]

This should not overload the processor:

effbot = "''100000022222222*2" safe_eval(effbot) Traceback (most recent call last): [...] Unsafe_Source_Error: Line 1. Unsupported source construct: compiler.ast.Mul

This implementation is a slight refinement to one I originally posted to c.l.py: http://groups-beta.google.com/group/comp.lang.python/msg/c18b2f99a14cfc20

Alternative approaches: Inspecting bytecodes: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286134

Manually parsing lists: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/281056

7 comments

Niki Spahiev 16 years, 5 months ago  # | flag

None. in order to handle None change visitName to start with

if node.name == 'None':
  return None
Niki Spahiev 16 years, 5 months ago  # | flag

cache. cache is not effective because getattr is called every time.

Michael Spencer (author) 15 years, 8 months ago  # | flag

cache. Good point. Cache now removed.

John Marshall 15 years, 2 months ago  # | flag

Minor fix to visitTuple. Instead of:

return tuple(self.visit(i) for i in node.nodes)

I think you want:

return tuple([self.visit(i) for i in node.nodes])
Gabriel Genellina 15 years, 2 months ago  # | flag

Not really. The tuple constructor will iterate along the returned generator; no need to construct an intermediate list. (Python>=2.4)

Ned Batchelder 14 years, 2 months ago  # | flag

Doesn't work for negative numbers. Turns out -123 is not a constant in Python, it's 123 with unary minus applied to it. So to handle negative numbers, you need to add a method to SaveEval:

def visitUnarySub(self, node, **kw):
    return -self.visit(node.getChildNodes()[0])
APP 9 years, 5 months ago  # | flag

Doesn't work with booleans either; simple fix:

Replace 'visitName' with:

def visitName(self,node, **kw):
    if node.name == 'True':
        return True
    elif node.name == 'False':
        return False
    raise Unsafe_Source_Error("Strings must be quoted", 
                             node.name, node)
Created by Michael Spencer on Tue, 25 Jan 2005 (PSF)
Python recipes (4591)
Michael Spencer's recipes (1)

Required Modules

Other Information and Tasks