Welcome, guest | Sign In | My Account | Store | Cart

A small, powerful templating language using template strings embedded in standard Python syntax. Templates are valid Python source, compiled directly to bytecode. Variable substitution is performed using 'string.Template'.

Python, 158 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
'''
A simple, embeddable templating language.  

Templates are executable Python code with strings embedded in them.  Template 
strings embedded in the code are emitted when the template is executed.
Variable substitution is performed using the common $var/${expr} notation.  The
${expr} form may include any expression, while the $var form may only contain 
a simple name.

A template string is a string used by itself on a line, not as part of an 
expression.  E.g.,

import sys
for count,segment in enumerate(sys.path):
  """
  $count: $segment
  """

Docstrings for modules or functions do not count as template strings and 
are not emitted.

Copyright (C) 2005 Kevin Schluff
License: Python license
'''

import string, re
import compiler
from compiler import ast, misc, visitor, pycodegen

class Template:
  """
  A Template is created from a Python source file with embedded 
  template strings.  Once created, a Template can  be reused by 
  calling emit() with different substitution variables.
  """

  def __init__(self, source):
    """
    Create a new template and compile it to bytecode.
    
    source - an open file or a string containing the template text.
    """
    if hasattr(source,'read'):
      filename = source.name
      source = source.read()
    else:
      filename = "<string>"
    self.code = TemplateCompiler().compile(source, filename)
      
  def emit(self, fd, **context):
    """
    Emit the text of the template, substituting the variables provided. 
    
    fd - A file-like object to write the output to.  E.g.,sys.stdout or StringIO.
    kwargs - Variables to substitute are passed as keyword arguments.
    """
    context['__StringTemplate'] = StringTemplate
    context['__template_fd'] = fd
    context['__EvalMapper'] = EvalMapper
    exec self.code in context
    
class TemplateCompiler(visitor.ASTVisitor):
  """
  The TemplateCompiler is an ASTVisitor that walks over the parsed AST, 
  transforming template strings into string.Templates.
  """
  # Strip a single leading newline and any tabs or spaces after the 
  # last newline.
  TEXT_RE = re.compile(r"\n?(.*\n)[ \t]*", re.DOTALL )
  
  def __init__(self):
    """
    Create a TemplateCompiler instance.
    """
    visitor.ASTVisitor.__init__(self)
    
  def compile(self, source, filename="<string>"):
    """
    Compile the template source into a code object suitable for execution
    with exec or eval.
    
    source - template (Python) source to compile.
    filename - the filename used in the compiled code.  This name will be
               used in tracebacks.
    """
    mod = compiler.parse(source)
    misc.set_filename(filename, mod)
    self.preorder(mod, self)
    generator = pycodegen.ModuleCodeGenerator(mod)
    code = generator.getCode()
    return code
    
  def visitStmt(self, node):
    """
    Visit a Stmt node to replace all of the template strings with 
    code that emits the string.
    """
    nodes = []
    for child in node.nodes:
      if isinstance(child, ast.Discard):
        children = self.replaceDiscard(child)
        for newNode in children:
          nodes.append(newNode)
      else:
        nodes.append(child)
            
    node.nodes = nodes
    
    for n in node.nodes:
      self.dispatch(n)
      
  def replaceDiscard(self, node):
    """
    Replace a single discard statement with a series of statements
    that write out the string. 
    """
    # Only operate on constant expressions
    if not isinstance(node.expr, ast.Const):
      return [node]

    value = self.TEXT_RE.sub(r"\1",node.expr.value)
    
    # This code replaces each template string
    subst = """
__mapper = __EvalMapper(globals(), locals())
__template_fd.write(__StringTemplate(%s).safe_substitute(__mapper))
""" % value.__repr__()

    module = compiler.parse(subst)
    nodes = module.node.nodes
        
    return nodes

class StringTemplate(string.Template):
  
    pattern = re.compile(r"""
    %(delim)s(?:
      (?P<escaped>%(delim)s) |   # Escape sequence of two delimiters
      (?P<named>%(id)s)      |   # delimiter and a Python identifier
      {(?P<braced>%(expr)s)} |   # delimiter and a braced identifier
      (?P<invalid>)              # Other ill-formed delimiter exprs
    )
   """ % {"delim":r"\$", "id":r"[_a-z][_a-z0-9]*", "expr":r".*?"},
     re.VERBOSE | re.IGNORECASE )
    
class EvalMapper:

    def __init__(self, globals_, locals_):
        self.globals = globals_
        self.locals = locals_

    def __getitem__(self, name):
        return eval(name, self.globals, self.locals)
              
if __name__ == "__main__":
  import sys
  template = Template(open(sys.argv[1]))
  template.emit(sys.stdout)

There's no shortage of Python templating tools available, ranging from full-blown templating engines to simpler regular expression based tools. The goal of this one is to provide something small enough to distribute as a single file, without losing the full expressiveness of Python.

In this recipe, a template is a standard Python module with template strings embedded into it. A template string is just a string used by itself on a line, not as part of an expression. Conceptually, a template module is executed and the strings are emitted as they are encountered. Variables are substituted in the template string using the names that are in scope when the string is encountered.

Use of standard Python syntax makes the complete power of the language available to templates. This also makes debugging of template logic easy, since tracebacks point to any errors.

This recipe works by leveraging the standard 'compiler' package and the 'string.Template' class introduced in 2.4. A template is compiled into bytecode by parsing it using 'compiler.parse()' to generate an AST. The AST is modified to replace any template strings (or other constant expressions) with code that generates a 'string.Template' and calls safe_substitute() on it. The AST is then compiled into bytecode as a code object, similar to a code object created by calling the builtin 'compile()'. Text is emitted from the template by 'exec'ing the code object, with any variables to be substitued as the globals.

An example template: <pre> """ A simple template example. This docstring won't be emitted. """

def writeName(name):

if name == "Mary": """ Mary had a little ${pet.kind}. Its ${pet.coat} was ${pet.colour} as $comparison. Everywhere that Mary went, her ${pet.kind} was sure to go.

"""

else: """ Hello, $name.

"""

"""

"""

names = ["Bo Peep", "Humpty", "Mary"] """ Names: ${", ".join([n.upper() for n in names])}

"""

for name in names: writeName(name)

"""

"""

class Pet: pass

if __name__ == "__main__": import sys from simple_template import Template

example = open(__file__)

pet = Pet() pet.kind = "lamb" pet.coat = "fleece" pet.colour = "white"

template = Template(example) template.emit(sys.stdout, pet=pet, comparison="snow")

</pre>

1 comment

Kevin Schluff (author) 18 years, 6 months ago  # | flag

Due credit. The template syntax used in this recipe is quite similar to Quixote's PTL, which I had seen. It turns out that the implementation is also fairly similar, which I was not aware of (this is a clean-room implementation, I promise!). Both operate using the compiler module and modifying ASTs, although PTL is much more sophisticated in what it can do.

PTL is somewhat different in that it returns any template strings found in a function, while this recipe evaluates the strings wherever they are found. As a result, the usage patterns are fairly different. Nonetheless, kudos to the team at mems-exchange.org for an original templating approach.