Welcome, guest | Sign In | My Account | Store | Cart

C-coded Python extension types have an aura of mystery and difficulty they don't really deserve. Sure, it's a lot of work to implement every possible nicety, but a fundamental useful type doesn't take much.

Python, 105 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
# copy these lines to a file named: setup.py
# To build and install the elemlist module you need the following
# setup.py file that uses the distutils:
from distutils.core import setup, Extension

setup (name = "elemlist",
       version = "1.0",
       maintainer = "Alex Martelli",
       maintainer_email = "aleax@aleax.it",
       description = "Sample Python module",

       ext_modules = [Extension('elemlist',sources=['elemlist.c'])]
)
# end of file: setup.py

/* copy these lines to a file named: elemlist.c */
/* and here comes elemlist.c itself with really minimal comments: */
#include "Python.h"

/* type-definition & utility-macros */
typedef struct {
    PyObject_HEAD
    PyObject *car, *cdr;
} cons_cell;
staticforward PyTypeObject cons_type;
/* a typetesting macro (we don't use it here) */
#define is_cons(v) ((v)->ob_type == &cons_type)
/* macros to access car & cdr, both as lvalues & rvalues */
#define carof(v) (((cons_cell*)(v))->car)
#define cdrof(v) (((cons_cell*)(v))->cdr)

/* ctor (factory-function) and dtor */
static cons_cell*
cons_new(PyObject *car, PyObject *cdr)
{
    cons_cell *cons = PyObject_NEW(cons_cell, &cons_type);
    if(cons) {
        cons->car = car; Py_INCREF(car);  /* INCREF when holding a PyObject* */
        cons->cdr = cdr; Py_INCREF(cdr);  /* ditto */
    }
    return cons;
}
static void
cons_dealloc(cons_cell* cons)
{
    /* DECREF when releasing previously-held PyObject*'s */
    Py_DECREF(carof(cons)); Py_DECREF(cdrof(cons));
    PyObject_DEL(cons);
}

/* Python type-object */
statichere PyTypeObject cons_type = {
    PyObject_HEAD_INIT(0)    /* initialize to 0 to ensure Win32 portability */
    0,                 /*ob_size*/
    "cons",            /*tp_name*/
    sizeof(cons_cell), /*tp_basicsize*/
    0,                 /*tp_itemsize*/
    /* methods */
    (destructor)cons_dealloc, /*tp_dealloc*/
    /* implied by ISO C: all zeros thereafter */
};

/* module-functions */
static PyObject*
cons(PyObject *self, PyObject *args)    /* the exposed factory-function */
{
    PyObject *car, *cdr;
    if(!PyArg_ParseTuple(args, "OO", &car, &cdr))
        return 0;
    return (PyObject*)cons_new(car, cdr);
}
static PyObject*
car(PyObject *self, PyObject *args)     /* car-accessor */
{
    PyObject *cons;
    if(!PyArg_ParseTuple(args, "O!", &cons_type, &cons))  /* type-checked */
        return 0;
    return Py_BuildValue("O", carof(cons));
}
static PyObject*
cdr(PyObject *self, PyObject *args)     /* cdr-accessor */
{
    PyObject *cons;
    if(!PyArg_ParseTuple(args, "O!", &cons_type, &cons))  /* type-checked */
        return 0;
    return Py_BuildValue("O", cdrof(cons));
}
static PyMethodDef elemlist_methods[] = {
    {"cons",   cons,   METH_VARARGS},
    {"car",    car,    METH_VARARGS},
    {"cdr",    cdr,    METH_VARARGS},
    {0, 0}
};

/* module entry-point (module-initialization) function */
void
initelemlist(void)
{
    /* Create the module and add the functions */
    PyObject *m = Py_InitModule("elemlist", elemlist_methods);
    /* Finish initializing the type-objects */
    cons_type.ob_type = &PyType_Type;
}

/* end of file: elemlist.c */

This module is roughly equivalent to the Python-coded module: def cons(car,cdr): return car, cdr def car(conscell): return conscell[0] def cdr(conscell): return conscell[1] except the C version is about 25 times more lines of code, even net of comments and empty lines (AND not much faster than the Python-coded version, either:-).

However, the point is demonstrating a minimal C-coded extension type -- I'm not even supporting object-methods (except the necessary destructor), but, rather, module-level functions for car and cdr access. This also shows the utter simplicity of building a C-coded extension module on any platform, thanks to the distutils -- they do all of the hard work. Most interestingly, the C-coded version is a good platform for experimentation -- adding all sorts of interesting doodads to the cons-cell objects makes for an interesting self-tutorial in writing Python C-coded extensions. Most useful, for performance, would be a "free-list", keeping up to the last N freed cons-cells for immediate reuse (having DECREF'd their car and cdr, of course) -- N should be exposed as a read-write module attribute, another interesting exercise. Most interesting is to have a good unit-test for this, and timing/profiling to show the usefulness (if any) of any performance tweaks.

By the way, Lisp-savvy readers will have recognized from the names involved that this (in either the Python or C implementations) is the core of a Lisp-ish list -- using some NIL marker (e.g. the empty Python tuple ()) as the cdr of the last cons-cell of a list, and otherwise consing up a list by the trick of having every cdr always be another cons-cell. Helper functions to let cons-cells be treated as full-fledged Python sequences when they do represent a canonical list is another fun project. If you want mutable cons-cells, as in real Lisp, you need to add setcar and setcdr functions, too. One might in fact (easily) _constrain_ the cdr to be either () or another cons-cell (giving up on generality for a bit of extra checking...).

As this is meant as an introduction to writing extension modules in C for Python, here are the step-by-step instructions, assuming you have a Windows machine with Python 2.0 or later, and Microsoft Visual C++ 6 (or the free command-line equivalent that you can download from Microsoft's site as a part of their ".NET Framework SDK") -- you can presumably translate mentally to other platforms such as Linux with gcc, etc, while using non-Microsoft compilers on Windows takes more work.

  1. make a new directory, say C:\Temp\EL
  2. open a command-prompt (MSDOS Box) and CD to the new directory
  3. in the new directory make files setup.py and elemlist.c with the contents of the above text split as per the comments therein
  4. run at the DOS prompt: C:\Temp\EL>python setup.py install

This will give lots of output, but presumably all goes well and the new elemlist extension has been built and installed. Now test it: 4. run at the DOS prompt: C:\Temp\EL>python (snipped -- various greeting messages from Python)

>>> from elemlist import cons
>>> a=cons(1,cons(2,cons(3,())))
>>> from elemlist import car, cdr
>>> car(cdr(a))
2
>>>
  1. Done -- your new extension module is ready and installed!

4 comments

Kathy Gerber 19 years, 3 months ago  # | flag

Thanks! This worked instantly in linux as well.

steve steiner 17 years, 7 months ago  # | flag

Didn't work on OS X. Didn't work on OS X (10.3.3), Python complains that there is no module elemlist.

Will post solution when I figure it out unless someone beats me to it.

Marcus Goldfish 16 years, 9 months ago  # | flag

Didn't work (Python 2.4). Returned this error:

"error: The .NET Framework SDK needs to be installed before building extensions for Python."

Python install seems to be working fine. Curiously, I have Visual Studio .NET professional installed.

Marcus Goldfish 16 years, 9 months ago  # | flag

Found the fix for Python 2.4 on Windows XP. I spent a couple of hours researching the problem this afternoon. It seems that Python 2.4 on Windows XP requires some modifications to distutils to use Microsoft compilers (and the development environment needs to be setup correctly). I followed the instructions at

http://www.vrplumber.com/programming/mstoolkit/index.html

and was able to successfully reproduce this recipe.

Marcus

Created by Alex Martelli on Fri, 10 Aug 2001 (PSF)
Python recipes (4591)
Alex Martelli's recipes (27)

Required Modules

  • (none specified)

Other Information and Tasks