ActiveState Code

Recipe 364469: "Safe" Eval

Evaluate constant expressions, including list, dict and tuple using the abstract syntax tree created by compiler.parse. Since compiler does the work, handling arbitratily nested structures is transparent, and the implemenation is very straightforward.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import compiler

class Unsafe_Source_Error(Exception):
    def __init__(self,error,descr = None,node = None):
        self.error = error
        self.descr = descr
        self.node = node
        self.lineno = getattr(node,"lineno",None)
        
    def __repr__(self):
        return "Line %d.  %s: %s" % (self.lineno, self.error, self.descr)
    __str__ = __repr__    
           
class SafeEval(object):
    
    def visit(self, node,**kw):
        cls = node.__class__
        meth = getattr(self,'visit'+cls.__name__,self.default)
        return meth(node, **kw)
            
    def default(self, node, **kw):
        for child in node.getChildNodes():
            return self.visit(child, **kw)
            
    visitExpression = default
    
    def visitConst(self, node, **kw):
        return node.value

    def visitDict(self,node,**kw):
        return dict([(self.visit(k),self.visit(v)) for k,v in node.items])
        
    def visitTuple(self,node, **kw):
        return tuple(self.visit(i) for i in node.nodes)
        
    def visitList(self,node, **kw):
        return [self.visit(i) for i in node.nodes]

class SafeEvalWithErrors(SafeEval):

    def default(self, node, **kw):
        raise Unsafe_Source_Error("Unsupported source construct",
                                node.__class__,node)
            
    def visitName(self,node, **kw):
        raise Unsafe_Source_Error("Strings must be quoted", 
                                 node.name, node)
                                 
    # Add more specific errors if desired
            

def safe_eval(source, fail_on_error = True):
    walker = fail_on_error and SafeEvalWithErrors() or SafeEval()
    try:
        ast = compiler.parse(source,"eval")
    except SyntaxError, err:
        raise
    try:
        return walker.visit(ast)
    except Unsafe_Source_Error, err:
        raise

Discussion

The topic of how to parse a constant list or dictionary safely crops up fairly regularly, e.g.

Examples (from recent c.l.py thread):

>>> goodsource =  """[1, 2, 'Joe Smith', 8237972883334L,   # comment
...       {'Favorite fruits': ['apple', 'banana', 'pear']},  # another comment
...       'xyzzy', [3, 5, [3.14159, 2.71828, []]]]"""
...

Providing the source contains only constants and arbitrarily nested list/dict/tuple, the result is the same as built-in eval:

safe_eval(good_source) [1, 2, 'Joe Smith', 8237972883334L, {'Favorite fruits': ['apple', 'banana', 'pear']}, 'xyzzy', [3, 5, [3.1415899999999999, 2.71828, []]]]

>>> assert _ == eval(good_source)

But this should fail, due to unquoted string literal

>>> badsource = """[1, 2, JoeSmith, 8237972883334L,   # comment
...       {'Favorite fruits': ['apple', 'banana', 'pear']},  # another comment
...       'xyzzy', [3, 5, [3.14159, 2.71828, []]]]"""
...

safe_eval(bad_source) Traceback (most recent call last): [...] Unsafe_Source_Error: Line 1. Strings must be quoted: JoeSmith

Alternatively, ignore the unsafe content and parse the rest:

safe_eval(bad_source, fail_on_error = False) [1, 2, None, 8237972883334L, {'Favorite fruits': ['apple', 'banana', 'pear']}, 'xyzzy', [3, 5, [3.1415899999999999, 2.71828, []]]]

This should not overload the processor:

effbot = "''100000022222222*2" safe_eval(effbot) Traceback (most recent call last): [...] Unsafe_Source_Error: Line 1. Unsupported source construct: compiler.ast.Mul

This implementation is a slight refinement to one I originally posted to c.l.py: http://groups-beta.google.com/group/comp.lang.python/msg/c18b2f99a14cfc20

Alternative approaches: Inspecting bytecodes: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286134

Manually parsing lists: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/281056

Comments

  1. 1. At 5:22 a.m. on 28 jun 2005, Niki Spahiev said:

    None. in order to handle None change visitName to start with

    if node.name == 'None':
  return None
  2. 2. At 5:26 a.m. on 28 jun 2005, Niki Spahiev said:

    cache. cache is not effective because getattr is called every time.

  3. 3. At 10:15 a.m. on 26 mar 2006, Michael Spencer (the author) said:

    cache. Good point. Cache now removed.

  4. 4. At 8:14 a.m. on 12 sep 2006, John Marshall said:

    Minor fix to visitTuple. Instead of:

    return tuple(self.visit(i) for i in node.nodes)

    I think you want:

    return tuple([self.visit(i) for i in node.nodes])
  5. 5. At 8:18 p.m. on 28 sep 2006, Gabriel Genellina said:

    Not really. The tuple constructor will iterate along the returned generator; no need to construct an intermediate list. (Python>=2.4)

  6. 6. At 5:13 a.m. on 8 oct 2007, Ned Batchelder said:

    Doesn't work for negative numbers. Turns out -123 is not a constant in Python, it's 123 with unary minus applied to it. So to handle negative numbers, you need to add a method to SaveEval:

    def visitUnarySub(self, node, **kw):
    return -self.visit(node.getChildNodes()[0])

Sign in to comment