Welcome, guest | Sign In | My Account | Store | Cart

The module optparse was a great addition to Python 2.3, since it is much more powerful and easier to use than getopt. Using optparse, writing command-line tools is a breeze. However, the power of optparse comes together with a certain verbosity. This recipe allows to use optparse with a minimum of boilerplate, trading flexibility for easy of use. Still, it covers 95% of my common needs, so I think it may be useful to others.

Python, 69 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
"""\
:Author: M. Simionato
:Date: April 2004
:Title: A much simplified interface to optparse.

You should use optionparse in your scripts as follows.
First, write a module level docstring containing something like this
(this is just an example):

'''usage: %prog files [options]
   -d, --delete: delete all files
   -e, --erase = ERASE: erase the given file'''
   
Then write a main program of this kind:

# sketch of a script to delete files
if __name__=='__main__':
    import optionparse
    option,args=optionparse.parse(__doc__)
    if not args and not option: optionparse.exit()
    elif option.delete: print "Delete all files"
    elif option.erase: print "Delete the given file"

Notice that ``optionparse`` parses the docstring by looking at the
characters ",", ":", "=", "\\n", so be careful in using them. If
the docstring is not correctly formatted you will get a SyntaxError
or worse, the script will not work as expected.
"""

import optparse, re, sys

USAGE = re.compile(r'(?s)\s*usage: (.*?)(\n[ \t]*\n|$)')

def nonzero(self): # will become the nonzero method of optparse.Values       
    "True if options were given"
    for v in self.__dict__.itervalues():
        if v is not None: return True
    return False

optparse.Values.__nonzero__ = nonzero # dynamically fix optparse.Values

class ParsingError(Exception): pass

optionstring=""

def exit(msg=""):
    raise SystemExit(msg or optionstring.replace("%prog",sys.argv[0]))

def parse(docstring, arglist=None):
    global optionstring
    optionstring = docstring
    match = USAGE.search(optionstring)
    if not match: raise ParsingError("Cannot find the option string")
    optlines = match.group(1).splitlines()
    try:
        p = optparse.OptionParser(optlines[0])
        for line in optlines[1:]:
            opt, help=line.split(':')[:2]
            short,long=opt.split(',')[:2]
            if '=' in opt:
                action='store'
                long=long.split('=')[0]
            else:
                action='store_true'
            p.add_option(short.strip(),long.strip(),
                         action = action, help = help.strip())
    except (IndexError,ValueError):
        raise ParsingError("Cannot parse the option string correctly")
    return p.parse_args(arglist)

The following script is an example of how to use the recipe.

<pre>

"""An example script invoking optionparse, my wrapper around optparse.

usage: %prog [options] args -p, --positional: print positional arguments -1, --option1=OPTION1: print option1 -2, --option2=OPTION2: print option2 """

import optionparse opt, args = optionparse.parse(__doc__) if not opt and not args: optionparse.exit() if opt.positional: print args if opt.option1: print opt.option1 if opt.option2: print opt.option2

</pre>

The optionparse.parse() function parses the docstring and internally builds an option parser object using optparse; then it uses that parser to parse the command line arguments (please do not confuse parsing the docstring with parsing the command line!) It returns an object containing the given options and a list of positional arguments.

If no options and no positional arguments are given, the script exits and returns an helpful message:

<pre> $ python example.py An example script invoking optionparse.

usage: example.py [options] args -p, --positional: print positional arguments -1, --option1=OPTION1: print option1 -2, --option2=OPTION2: print option2 </pre>

A similar message is also obtained if the -h or --help option is passed.

If the -p flag is passed, the list of positional arguments is displayed:

<pre> $ python example.py -p *.txt [list-of-text-files-in-the-current-directory] </pre>

If the option argument 1 or 2 are passed, they are displayed:

<pre> $ python example.py -1hello -2world hello world </pre>

I think you get the idea. Within the current implementation there are restrictions with the format of the usage block in the docstring: for instance it cannot contain blank lines and one must be careful with characters such as ":" "," "=". It is up to you to build up a more sophisticated parser, if you care enough. The purpose of this recipe is just to give the idea.

5 comments

Simon Brunning 19 years, 11 months ago  # | flag

Beautiful. Parsing the module's docstring for the command line options is a brilliant idea. It feels really pythonic; most other languages, for example, treat indentation as purely documentary, and require you to repeat your code block delimitation using delimitation characters and using indentation, whereas Python treats the indentation as definitive, and doesn't require you to repeat yourself. Similarly, this recipe treats the docstring's definition of the supported command line options as definitive, and doesn't require you to repeat yourself.

David Boddie 19 years, 11 months ago  # | flag

See also. You could probably perform more complicated parsing, and introduce some additional syntax checking at the same time, by adapting parts of the CMDSyntax module's Syntax class:

http://www.boddie.org.uk/david/Projects/Python/CMDSyntax/

Jean Brouwers 19 years, 10 months ago  # | flag

__doc__ disappears with -OO. Just a reminder that the __doc__ string will be empty when Python is invoked with -OO. Otherwise, an excellent recipe.

Henry Crutcher 17 years, 6 months ago  # | flag

Can easily be enhanced to take default values. With the code below, I'm very much enjoying this recipe. I've tweaked it slightly to allow default values, and thought that might be a useful change...

"""\
:Author: M. Simionato
:Date: April 2004
:Title: A much simplified interface to optparse.
:modified: Modified by Henry Crutcher to support default values

You should use optionparse in your scripts as follows.
First, write a module level docstring containing something like this
(this is just an example):

'''usage: %prog files [options]
   -d, --delete: delete all files
   -e, --erase = ERASE: erase the given file
   -F, --fill_pattern = 0xFF: the fill pattern used to erase the given file'''

Then write a main program of this kind:

# sketch of a script to delete files
if __name__=='__main__':
    import optionparse
    option,args=optionparse.parse(__doc__)
    if not args and not option: optionparse.exit()
    elif option.delete: print "Delete all files"
    elif option.erase:
      print "Delete the given file with %s" % option.fill_pattern


Notice that ``optionparse`` parses the docstring by looking at the
characters ",", ":", "=", "\\n", so be careful in using them. If
the docstring is not correctly formatted you will get a SyntaxError
or worse, the script will not work as expected.
"""

import optparse, re, sys

USAGE = re.compile(r'(?s)\s*usage: (.*?)(\n[ \t]*\n|$)')

def nonzero(self): # will become the nonzero method of optparse.Values
    "True if options were given"
    for v in self.__dict__.itervalues():
        if v is not None: return True
    return False

optparse.Values.__nonzero__ = nonzero # dynamically fix optparse.Values

class ParsingError(Exception): pass

optionstring=""

def exit(msg=""):
    raise SystemExit(msg or optionstring.replace("%prog",sys.argv[0]))

def parse(docstring, arglist=None):
    global optionstring
    optionstring = docstring
    match = USAGE.search(optionstring)
    if not match: raise ParsingError("Cannot find the option string")
    optlines = match.group(1).splitlines()
    try:
        p = optparse.OptionParser(optlines[0])
        for line in optlines[1:]:
            opt, help=line.split(':')[:2]
            short,long=opt.split(',')[:2]
            if '=' in opt:
                # unless the value on the other side of = is the same
                # (modulo case) it is used as the default

(comment continued...)

Henry Crutcher 17 years, 6 months ago  # | flag

(...continued from previous comment)

                action='store'
                long, default=long.split('=')[:2]
                if default.lower()==long:
                  default=None
            else:
                action='store_true'
            p.add_option(short.strip(),long.strip(),
                         action = action, help = help.strip(), default=default)
    except (IndexError,ValueError):
        raise ParsingError("Cannot parse the option string correctly")
    return p.parse_args(arglist)