Welcome, guest | Sign In | My Account | Store | Cart

To 'return None' from your Python-callable C function (Python recipe) by Alex Martelli
ActiveState Code (http://code.activestate.com/recipes/52309/)

Often a function written in C for Python needs to return nothing in particular -- a "return None" in Python terms; but _don't_ just "return Py_None" from C, as that will mess up reference counts!

Python, 30 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/* suppose we need an empty C-coded function, equivalent to Python:
def empty1(*args):
    pass
or identically:
def empty2(*args):
    return None
There is still a right and a wrong way to do it...!
*/

/* WRONG! will mess up reference counts...: */
static PyObject*
empty3(PyObject* self, PyObject* args)
{
    return Py_None;
}

/* Fine! the simplest way to do it...: */
static PyObject*
empty4(PyObject* self, PyObject* args)
{
    return Py_BuildValue("");
}

/* Fine! the canonical approach...: */
static PyObject*
empty5(PyObject* self, PyObject* args)
{
    Py_INCREF(Py_None);
    return Py_None;
}

None, the Python object we must explicitly return from a C-coded function that is Python-callable, IS a Python object, still subject to all normal reference count rules. The returned object must be incref'd by the returning function.

So, a bare "return Py_None;" is a nasty lurking bug; either explicitly incref the None object you're returning, or (simpler, but costs a few machine cycles) delegate the work to handy function Py_BuildValue (the latter can be used for just about all return-value-from-C-to-Python needs, which offers potential advantages of uniformity) by just calling it with an empty formatstring.

Tags: extending

3 comments

Edward Ream 22 years, 7 months ago  # | flag

why I only use Py_BuildValue("") to return None.

I am careful always to return Py_BuildValue("") rather than
Py_None when I extend or embed Python with C code.

Returning Py_None will not work if you build your C code with a
compiler other than the compiler used to build Python itself!
IMO, this point should be emphasized in the extending and embedding docs.

Returning Py_None anywhere outside the actual Python build is
dangerous because Py_None is a struct whose layout depends on
1) the compiler used and
2) various C/C++ compiler settings.

In contrast, return Py_BuildValue("") can never fail, because
Py_BuildValue("") returns a pointer to a struct that was built
when Python was built.

If your C code does return Py_None, then you must guarantee that
your compiler and build settings match the compiler and build
settings used to build Python.

Fixing the problem at (and in) the C source seems much safer to me.

Edward K. Ream
Jason Orendorff 22 years, 3 months ago  # | flag

Eh? This comment about Py_BuildValue() doesn't make sense to me.

Many things ordinarily done in extension modules depend on the compiler's alignment rules matching those of the Python executable. Just off the top of my head:

  • using Py_INCREF and Py_DECREF at all

  • using any other macros defined in Python.h, such as PyString_AS_STRING()

  • declaring a static PyTypeObject

If your compiler settings aren't fully link-compatible with the ones used to build Python, then it seems to me that Py_None is the least of your problems.

Edward Ream 22 years, 1 month ago  # | flag

Ooops. Thanks for this correction. I really made a hash of this posting. For me, replacing Py_None by Py_BuildValue("") got everything working. As you point out, this was really just by accident.

My original thought was that it would be easier to write extensions if macros like Py_INCREF, Py_DECREF, PyTypeObject, etc. were functions. Replacing macros with functions in the C API would reduce (but not eliminate!) the compiler settings that could cause problems at link time. In particular, compiler settings affect function calls as well. I am acutely embarrassed not to have realized this. Thanks again for setting me straight.

Edward