ActiveState Code

Recipe 54352: Defining Python class methods in C

This recipe shows how to define a new Python class from a C extension module. The class methods are implemented in C, but the class can still be instantiated, extended, subclassed, etc. from Python. The same technique can also be used to extend an existing Python class with methods written in C.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
#include <Python.h>

static PyObject* Foo_init(PyObject *self, PyObject *args)
{
    printf("Foo.__init__ called\n");
    Py_INCREF(Py_None);
    return Py_None;
}

static PyObject* Foo_doSomething(PyObject *self, PyObject *args)
{
    printf("Foo.doSomething called\n");
    Py_INCREF(Py_None);
    return Py_None;
}

static PyMethodDef FooMethods[] = 
{
    {"__init__", Foo_init, METH_VARARGS, 
	 "doc string"},
    {"doSomething", Foo_doSomething, METH_VARARGS,
	 "doc string"},
    {NULL},
};

static PyMethodDef ModuleMethods[] = { {NULL} };

#ifdef __cplusplus
extern "C"
#endif
void initFoo()
{
    PyMethodDef *def;

    /* create a new module and class */
    PyObject *module = Py_InitModule("Foo", ModuleMethods);
    PyObject *moduleDict = PyModule_GetDict(module);
    PyObject *classDict = PyDict_New();
    PyObject *className = PyString_FromString("Foo");
    PyObject *fooClass = PyClass_New(NULL, classDict, className);
    PyDict_SetItemString(moduleDict, "Foo", fooClass);
    Py_DECREF(classDict);
    Py_DECREF(className);
    Py_DECREF(fooClass);
    
    /* add methods to class */
    for (def = FooMethods; def->ml_name != NULL; def++) {
	PyObject *func = PyCFunction_New(def, NULL);
	PyObject *method = PyMethod_New(func, NULL, fooClass);
	PyDict_SetItemString(classDict, def->ml_name, method);
	Py_DECREF(func);
	Py_DECREF(method);
    }
}

Discussion

The usual method of creating new types in an extension module is to define a new instance of PyTypeObject and provide callbacks to the various C functions that implement the type. However, it may be better to define the new type as a Python class so that the type can be instantiated, subclassed, etc. from Python. In some cases, defining a custom exception type for instance, it is even required that the new type be a Python class.

The class methods are defined in C functions and described by a table of PyMethodDef's in the same way that module methods are defined. The key that allows these methods to become class methods is that they are each wrapped first in a PyCFunction object and then in a PyMethod object. The PyCFunction turns the C function into a python object, and the PyMethod associates the function with a particular class. Finally, the methods are added to the class' dictionary which makes them callable by instances of the class.

Notes: 1) Base classes can be specified for the new class by passing a tuple of class objects as the first arg to PyClass_New. These can be existing python classes. 2) The second argument passed to PyCFunction_New becomes the "self" arg passed to the C function. This can be any Python object, but it's not very useful in this context since you could just as easily keep a static C variable. 3) The class instance is passed to the C functions as the first argument in the "args" tuple.

Comments

  1. 1. At 9:33 a.m. on 29 nov 2001, Vladimir Sironja said:

    getattr trouble.

    Hi,
great article. Here a short follow-up.
Problems arise when trying to implement __getattr__ in a way described above.
The C-implementation is not called if added in a same way as "doSomething".
According to Python Language Reference, __getattr__ must be available at the
time of class definition and will not be called if added to __dict__ later.
So, __getattr__, __setattr__ and co must be added before calling PyClass_New.
Obviously, they are added as functions, and not as methods.

Code:

static PyMethodDef AttrMethods[] =
{
  { "__setattr__", test_setattr, METH_VARARGS },
  { "__getattr__", test_getattr, METH_VARARGS },
  { NULL, NULL },
};

for(PyMethodDef* _def = AttrMethods; _def->ml_name; _def++) {
      PyObject *func = PyCFunction_New(_def, NULL);
      PyDict_SetItemString(classDict, _def->ml_name, func);
      Py_DECREF(func);
  }
  // finally add class and all the methods as before
  testClass = PyClass_New(NULL, classDict, className);
  // ...

  // skipped

static PyObject* test_getattr(PyObject *self, PyObject *args) {
    PyObject *selfobj, *arg0;
    if(!PyArg_ParseTuple(args, "OO:test_foo", &selfobj, &arg0)) {
        // handle error
    }
    if (!PyString_Check(arg0)) {
        return NULL;
    }
    char* name = PyString_AsString(arg0);
    if (!name) return NULL;
    if (strcmp(name, "id")==0) {
        return Py_BuildValue("i", 42);
    }
    // throw attribute not found and and ...
}

Bye
Vlado
  2. 2. At 10:44 a.m. on 24 jul 2003, Anonymous said:

    With python 2.2 or greater you can do a lot better. With python 2.2 or better you can do a lot better than this. Python 2.2 and better have the new Class/Type unification. This was not documented until version 2.3. See the Python 2.3 "Extending and Embedding" for an example of the new interface.

    http://python.org/doc/2.3c1/ext/ext.html

Sign in to comment