ActiveState Code

Recipe 535136: Dependencies graph of a script or module

This is a recipe to generate a diagram of dependencies of a script. It uses Python's modulefinder to get the dependencies, the two scripts available from http://www.tarind.com/depgraph.html to generate a dot file, and graphviz to convert the dot file to PNG. It also filters out a lot of noise and facilitates configurability.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
import sys, os
from tempfile import mkstemp
from getopt import getopt, GetoptError
from subprocess import call as oscall
from depgraph2dot import pydepgraphdot
from py2depgraph import mymf


def genDepGraph(inScript, options):
    path = sys.path[:]
    debug = 0
    mf = mymf(path, debug, options.exclude)
    mf.run_script(inScript)
    
    # find all modules in standard lib and set them aside for later;
    # assume that modules that don't have a filename are from stdlib
    ignore = set()
    for moduleName, module in mf.modules.iteritems():
        ign = False
        if options.ignoreNoFile and (not module.__file__):
            ign = True
        if options.ignoreStdlib and module.__file__:
            path1 = os.path.abspath(os.path.dirname(module.__file__)).lower()
            path2 = os.path.abspath('c:\python24\lib').lower()
            if path1 == path2:
                ign = True
        if ign:
            ignore.add(moduleName)
            
    return dict(depgraph=mf._depgraph, types=mf._types), ignore


class MyDepGraphDot(pydepgraphdot):
    def __init__(self, buffer, ignore=None):
        self.__depgraph = buffer['depgraph']
        self.__types = buffer['types']
        tmpfd, tmpname = mkstemp('.dot', 'depgraph_')
        os.close(tmpfd)
        self.__output = file(tmpname, 'w')
        self.__output_name = tmpname
        self.__ignore = ignore or set()
        #print 'Will ignore modules:', self.__ignore
        
    def toocommon(self, s, type):
        if s in self.__ignore:
            return 1
        return pydepgraphdot.toocommon(self, s, type)
    
    def get_data(self):
        return self.__depgraph, self.__types
    def get_output_file(self):
        return self.__output
    def get_output_name(self):
        self.__output.close()
        return self.__output_name


class Options:
    def __init__(self):
        self.exclude = []        # list of module names to exclude from analysis
        self.ignoreStdlib = True # modules from Python's stdlib will not be in graph
        self.ignoreNoFile = True # modules that don't have an associated file will not be in graph
        self.dotPath = r'C:\Program Files\Graphviz2.16\bin\dot'
        self.args = sys.argv[1]

        print 'Will %sinclude stdlib modules' % (self.ignoreStdlib and 'NOT ' or '')
        print 'Will %sinclude "file-less" modules' % (self.ignoreNoFile and 'NOT ' or '')
        print 'Will exclude the following modules and anything imported by them:', self.exclude
        print 'Will use "%s" as dot' % self.dotPath
    

# Process command line args
options = Options()

# start processing script for dependencies
inScript = options.args[0]
try:
    buffer, ignore = genDepGraph(inScript, options)
except IOError, exc: 
    print 'ERROR:', exc
    sys.exit(-1)
    
if not buffer['depgraph']:
    print 'NO dependencies of interest! Nothing to generate, exiting.'
    sys.exit()
    
dotdep = MyDepGraphDot(buffer, ignore)
dotdep.main( sys.argv )

# convert to png
basename = os.path.splitext(os.path.basename(inScript))[0]
pngOutput = basename + '_depgraph.png'
print 'Generating %s from %s' % (pngOutput, dotdep.get_output_name())
oscall([
    options.dotPath, '-Tpng', '-o', pngOutput, dotdep.get_output_name()]
    )

# cleanup
os.remove(dotdep.get_output_name())

Discussion

Showing dependencies is useful when documenting the design of a package/module. Due to the dynamic nature of Python, just grepping for import statements is not a reliable method of determining dependencies. Some handy scripts exist at http://www.tarind.com/depgraph.html but they are not easy to use on Windows, and not easy to configure. The above recipe uses the two scripts as modules to make it trivial to generate the dependencies graph, and also adds configurability.

To use it: - download the two Python scripts from http://www.tarind.com/depgraph.html and install them in your path (e.g. Python's Scripts folder) - install graphviz - save the above recipe as a Python script, put in your path - verify that the path to dot is correct (Options.dotPath) - open a command shell, cd to the folder containing the script you want to analyse - run the recipe on that file, e.g. c:\Python24\Scripts\depgraph.py main.py

Note that I removed the command line parsing part of the code, in the Options.__init__() just before the print statements.

Comments

  1. 1. At 2:57 p.m. on 28 nov 2007, Chad Cooper said:

    error. I get this error, as if its not parsing sys.argv[1] properly??

    F:\GIS\Projects\Land>depgraph.py ArkomaLand.py

    Will NOT include stdlib modules

    Will NOT include "file-less" modules

    Will exclude the following modules and anything imported by them: []

    Will use "C:\Program Files\Graphviz2.16\bin\dot" as dot

    ERROR: [Errno 2] No such file or directory: 'A'

  2. 2. At 8:06 a.m. on 29 nov 2007, Chad Cooper said:

    fixed it. changed:

    inScript = options.args[0]

    to:

    inScript = options.args

Sign in to comment