Welcome, guest | Sign In | My Account | Store | Cart

This module provides an easy way to parse simple formatted strings. It works similar to the version C programmers are used to.

Python, 136 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
"""
Small scanf-implementation.

Python has powerful regular expressions but sometimes they are totally overkill
when you just want to parse a simple-formatted string.
C programmers use the scanf-function for these tasks (see link below).

This implementation of scanf translates the simple scanf-format into
regular expressions. Unlike C you can be sure that there are no buffer overflows
possible.


For more information see
  * http://www.python.org/doc/current/lib/node49.html
  * http://en.wikipedia.org/wiki/Scanf
"""
import re
import sys


__all__ = ["scanf"]


DEBUG = False



# As you can probably see it is relatively easy to add more format types
scanf_translate = [
    (re.compile(_token), _pattern, _cast) for _token, _pattern, _cast in [
    ("%c", "(.)", lambda x:x),
    ("%(\d)c", "(.{%s})", lambda x:x),
    ("%(\d)[di]", "([+-]?\d{%s})", int),
    ("%[di]", "([+-]?\d+)", int),
    ("%u", "(\d+)", int),
    ("%[fgeE]", "(\d+\.\d+)", float),
    ("%s", "(\S+)", lambda x:x),
    ("%([xX])", "(0%s[\dA-Za-f]+)", lambda x:int(x, 16)),
    ("%o", "(0[0-7]*)", lambda x:int(x, 7)),
    ]]



# Cache formats
SCANF_CACHE_SIZE = 1000
scanf_cache = {}


def _scanf_compile(format):
    """
    This is an internal function which translates the format into regular expressions
    
    For example:
    >>> format_re, casts = _scanf_compile('%s - %d errors, %d warnings')
    >>> print format_re.pattern
    (\S+) \- ([+-]?\d+) errors, ([+-]?\d+) warnings
    
    Translated formats are cached for faster use
    """
    compiled = scanf_cache.get(format)
    if compiled:
        return compiled

    format_pat = ""
    cast_list = []
    i = 0
    length = len(format)
    while i < length:
        found = None
        for token, pattern, cast in scanf_translate:
            found = token.match(format, i)
            if found:
                cast_list.append(cast)
                groups = found.groupdict() or found.groups()
                if groups:
                    pattern = pattern % groups
                format_pat += pattern
                i = found.end()
                break
        if not found:
            char = format[i]
            # escape special characters
            if char in "()[]-.+*?{}<>\\":
                format_pat += "\\"
            format_pat += char
            i += 1
    if DEBUG:
        print "DEBUG: %r -> %s" % (format, format_pat)
    format_re = re.compile(format_pat)
    if len(scanf_cache) > SCANF_CACHE_SIZE:
        scanf_cache.clear()
    scanf_cache[format] = (format_re, cast_list)
    return format_re, cast_list



def scanf(format, s=None):
    """
    scanf supports the following formats:
      %c       One character
      %5c      5 characters
      %d       int value
      %7d      int value with length 7
      %f       float value
      %o       octal value
      %X, %x   hex value
      %s       string terminated by whitespace

    Examples:
    >>> scanf("%s - %d errors, %d warnings", "/usr/sbin/sendmail - 0 errors, 4 warnings")
    ('/usr/sbin/sendmail', 0, 4)
    >>> scanf("%o %x %d", "0123 0x123 123")
    (66, 291, 123)


    If the parameter s is a file-like object, s.readline is called.
    If s is not specified, stdin is assumed.

    The function returns a tuple of found values
    or None if the format does not match.
    """

    if s == None: s = sys.stdin
    if hasattr(s, "readline"): s = s.readline()

    format_re, casts = _scanf_compile(format)
    found = format_re.match(s)
    if found:
        groups = found.groups()
        return tuple([casts[i](groups[i]) for i in range(len(groups))])



if __name__ == "__main__":
    import doctest
    doctest.testmod(verbose=True, report=True)

4 comments

Vinay Sajip 17 years, 2 months ago  # | flag

Radix error? The octal version should be int(x, 8) rather than int(x, 7). Octal "0123" is 83, not 66, AFAICT.

Bruce Deam 16 years ago  # | flag

Floating point numbers. Thank you for the useful module Henning. The re for parsing floats can use the more elaborate representation from the python.org manual page, except the inner matches are not required (and cause an error). Therefore change line 36 to use the "(?:" grouping rather than "(" grouping, giving:

("%[fgeE]", "([-+]?(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][-+]?\d+)?)", float),
alsuren 14 years, 11 months ago  # | flag

To enable support for named substitutions (note: they still come out as positional arguments) remove the list comprehension part from the initial definition, and add these two lines below it:

# Add on the extra formats with named arguments
_scanf_translate = _scanf_translate + [
    (_token.replace('%', "%\([a-zA-Z_]+\)"), _pattern, _cast) for (_token, _pattern, _cast) in _scanf_translate]
# And now compile.
_scanf_translate = [
    (re.compile(_token), _pattern, _cast) for (_token, _pattern, _cast) in _scanf_translate]

Note: I am aware that you could do it without duplicating all of the regular expressions, but regexps are hard enough to debug as it is, so to help future format writers, we might as well be nice to them, and make the scanf compiler try their original regexp first, before trying the mangled one.

southof40 14 years, 8 months ago  # | flag

Just what I needed ! - thank you for your work on this.