Welcome, guest | Sign In | My Account | Store | Cart

This recipe provides a number of functions to get the memory usage of a Python application on Linux.

Python, 43 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import os

_proc_status = '/proc/%d/status' % os.getpid()

_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
          'KB': 1024.0, 'MB': 1024.0*1024.0}

def _VmB(VmKey):
    '''Private.
    '''
    global _proc_status, _scale
     # get pseudo file  /proc/<pid>/status
    try:
        t = open(_proc_status)
        v = t.read()
        t.close()
    except:
        return 0.0  # non-Linux?
     # get VmKey line e.g. 'VmRSS:  9999  kB\n ...'
    i = v.index(VmKey)
    v = v[i:].split(None, 3)  # whitespace
    if len(v) < 3:
        return 0.0  # invalid format?
     # convert Vm value to bytes
    return float(v[1]) * _scale[v[2]]


def memory(since=0.0):
    '''Return memory usage in bytes.
    '''
    return _VmB('VmSize:') - since


def resident(since=0.0):
    '''Return resident memory usage in bytes.
    '''
    return _VmB('VmRSS:') - since


def stacksize(since=0.0):
    '''Return stack size in bytes.
    '''
    return _VmB('VmStk:') - since

To find the memory usage in a particular section of code these functions are typically used as follows

<pre> m0 = memory() ... m1 = memory(m0) </pre>

Getting and parsing the /proc//status file is not the most efficient way to determine the memory usage.

This recipe has been tested on RedHat Linux 8.0 with Python 2.3.2.

3 comments

sa li 19 years, 5 months ago  # | flag

RE: memory usage. it's good idea! thank you.

but it can only get the information for current process. in my Application, i want only to get the memory usage for children process(os.popen("children_process") or os.exec("children_process")).

how can i do?

Jean Brouwers (author) 19 years, 4 months ago  # | flag

memory usage of child processes. You can get the memory usage of any process by using the proper _process_status string: simply replace os.getpid() with the pid of the process. To get the pid of a spawned process, use the Popen3 class from the standard popen2 module.

For more details, see recipe 6.6 "Capturing the Output and Error Streams from a Unix Shell Command" by Brent Burley on page 227 of the Python Cookbook by O'Reilly, check the documentation of the popen2 module and source code in file .../lib/python.../popen2.py of your Python installation.

sa li 19 years, 4 months ago  # | flag

i think that the "proper_process_status" can be used only in Linux, it can't be runned in Unix (i.e.Solaris). because the file /proc/pid/sttus is a binary-file.

perhaps the function "resource.getrusage(resource.RUSAGE_CHILDREN)" is a choice too. but I don't konw how use it. :(

import resoure os.popen("command")

>>> resource.getrusage(resource.RUSAGE_CHILDREN)
(0.004999, 0.0089979999999999991, 0, 0, 0, 0, 444, 22, 0, 0, 0, 0, 0, 0, 40, 3)
[2:4] is the memory usage, but "0"!, why!?
can you help me?
Created by Jean Brouwers on Sun, 11 Jul 2004 (PSF)
Python recipes (4591)
Jean Brouwers's recipes (1)

Required Modules

Other Information and Tasks