ActiveState Code

Recipe 438119: shell - Easily call executables from Python

One common aspect of programming is calling executables and processing results. It is not as easy or elegant to do this in Python as it is in a shell scripting language, such as bash. This script implements a shell-like module in Python which allows you to easily call executables and work with the results.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
shell.py:
import sys

class Shell:
    def __init__(self):
        self.prefix = '/bin'
        self.env = {}
        self.stdout = None
        self.stderr = None
        self.wait = False

    def __getattr__(self, command):
        def __call(*args, **keywords):
            if command == 'prefix':
                return self.prefix
            if command == 'stdout':
                return self.stdout
            if command == 'stderr':
                return self.stderr
            if command.startswith('__'):
                return None
            if self.prefix:
                exe = '%s/%s' % (self.prefix, command)
            else:
                exe = command
            import os, subprocess
            if os.path.exists(exe):
                exeargs = [exe]
                if keywords:
                    for i in args.iteritems(): exeargs.extend(i)
                if args:
                    exeargs.extend(args)
                exeargs = [str(i) for i in exeargs]
                cwd = os.path.abspath(os.curdir)
                p = subprocess.Popen(exeargs, bufsize=1, cwd=cwd, env=self.env, stdout=subprocess.PIPE, close_fds=False, universal_newlines=True)
                if self.wait:
                    ret = p.wait()
                else:
                    ret = p.returncode
                result = None
                if p.stdout:
                    self.stdout = p.stdout.readlines()
                if p.stderr:
                    self.stderr = p.stderr.readlines()
                return ret
            else:
                raise Exception('No executable found at %s' % exe)
        return __call

sys.modules[__name__] = Shell()

Discussion

One of the advantages of shell scripting is tight integration between the language and the executables it calls. For example, to get a list of files in the current directory into a variable using the bash shell, you can use:

files=ls if [ $? ]; then echo $files; fi

One of the major disadvantages of most shell scripting languages is that they are difficult to work with in other ways. Control flow, modularisation and object-oriented features have a clunky syntax or are completely missing from most shell scripting languages. It would be really nice to be able to approximate shell scripting with Python.

Example usage:

import shell shell.ls('/usr') print shell.stdout

ret = shell.ps('ax') if not ret: print shell.stdout

This code could probably do with some tidying up, for example it may be nice to return a result object from shell calls: processes = shell.ps('ax').stdout

Any comments or suggestions are welcome.

Comments

  1. 1. At 7:53 a.m. on 22 jul 2005, Bill Mill said:

    Just get ipython. Get ipython from http://ipython.scipy.org/ . It a mature tool which does all that and much more.

  2. 2. At 4:33 p.m. on 22 jul 2005, Ian Bicking said:

    subprocess. I don't think ipython is really a replacement for something like this. ipython is interactive, but a recipe like this is very usable in a "script" environment (and maybe in any environment). I'd feel weird creating a script specifically intended to be piped to ipython.

    If it were to be a fuller module, it would be good to use the stdlib subprocess module, and it would be better if it returned a real object. And perhaps if shell.attr was equivalent to shell('attr'), if you want to give a full path to an executable.

  3. 3. At 10:58 a.m. on 11 feb 2006, Anonymous said:

    calling with multiple parameters ... I assumed (shame on me) that I could make this call:

    >>> shell.cat('/etc/fstab /etc/crontab')

    as I would on the command-line and get a stdout with both files. Instead I need:

    >>> shell.cat('/etc/fstab', '/etc/crontab')

    This makes sense once I look at the code, especially in the context of this example. However, it was a bit less obvious when attempting:

    >>>shell.prefix = '/usr/bin'
>>>shell.svnlook('changed -r%s %s') % (rev, repos)

    But thanks for the introduction to __getattr__ I'm continually impressed with Python's flexibility.

Sign in to comment