Welcome, guest | Sign In | My Account | Store | Cart

This is a quick snippet that I use occasionally to profile some pure-Python code, using hotshot. Basically it is this:

  1. Put this @hotshotit decorator on the function you want to profile.
  2. Run your code through some representative paces. The result will be a <functionname>.prof in the current directory.
  3. Process the .prof file and print the top 20 hotspots with the given "show_stats.py" script.

Props to Todd for slapping this code together.

Hotshot is a little out of favour now, so I should -- or Todd :) -- should really come up with an equivalent that uses cProfile.

Python, 12 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
hotshotProfilers = {}
def hotshotit(func):
    def wrapper(*args, **kw):
        import hotshot
        global hotshotProfilers
        prof_name = func.func_name+".prof"
        profiler = hotshotProfilers.get(prof_name)
        if profiler is None:
            profiler = hotshot.Profile(prof_name)
            hotshotProfilers[prof_name] = profiler
        return profiler.runcall(func, *args, **kw)
    return wrapper

Example of usage:

@hotshotit
def my_slow_function():
    #...

Here is a "show_stats.py" script to project the .prof that the above will generate.

# Usage: python show_stats.py foo.prof
import sys
import hotshot, hotshot.stats
stats = hotshot.stats.load(sys.argv[1])
stats.strip_dirs()
stats.sort_stats('time', 'calls')
stats.print_stats(20)

Here is a example of the output "show_stats.py" will give:

        2089198 function calls (2016559 primitive calls) in 11.447 CPU seconds

  Ordered by: internal time, call count
  List reduced from 88 to 20 due to restriction <20>

  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   57716    1.908    0.000    5.593    0.000 parser.py:682(parsemakesyntax)
     893    1.558    0.002   11.447    0.013 parser.py:467(parsestream)
227122/155505    1.472    0.000    1.679    0.000 parser.py:192(itermakefilechars)
   71661    1.406    0.000    1.406    0.000 parser.py:369(iterlines)
   76507    0.687    0.000    0.981    0.000 parserdata.py:34(__add__)
   76507    0.451    0.000    1.653    0.000 parser.py:73(getloc)
   71661    0.430    0.000    2.153    0.000 parser.py:127(readline)
   64578    0.409    0.000    0.409    0.000 parser.py:97(findtoken)
   64098    0.355    0.000    0.355    0.000 parser.py:672(__init__)
   71411    0.281    0.000    0.312    0.000 data.py:87(append)
  298705    0.266    0.000    0.266    0.000 parserdata.py:9(_charlocation)
   70768    0.234    0.000    0.234    0.000 parser.py:69(append)
   76507    0.184    0.000    0.221    0.000 parser.py:32(findlast)
   70179    0.179    0.000    0.179    0.000 parser.py:86(skipwhitespace)
   56295    0.167    0.000    0.167    0.000 parserdata.py:428(append)
   64409    0.163    0.000    0.213    0.000 parser.py:155(get)
   58978    0.148    0.000    0.231    0.000 parser.py:122(__init__)
   10035    0.121    0.000    0.629    0.000 parser.py:357(_iterflatten)
   15874    0.117    0.000    0.123    0.000 parser.py:246(itercommandchars)
   93986    0.112    0.000    0.112    0.000 parserdata.py:29(__init__)

1 comment

Robert Kern 15 years, 2 months ago  # | flag

I've done something similar for cProfile.

http://packages.python.org/line_profiler/#kernprof

Created by Trent Mick on Fri, 20 Feb 2009 (MIT)
Python recipes (4591)
Trent Mick's recipes (28)

Required Modules

Other Information and Tasks