Welcome, guest | Sign In | My Account | Store | Cart

Python distutils + pkg-config (Python recipe) by Micah Dowty
ActiveState Code (http://code.activestate.com/recipes/502261/)

This is a short and sweet recipe for using pkg-config with your distutils extension modules.

Python, 17 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
#!/usr/bin/env python
from distutils.core import setup
from distutils.extension import Extension
import commands

def pkgconfig(*packages, **kw):
    flag_map = {'-I': 'include_dirs', '-L': 'library_dirs', '-l': 'libraries'}
    for token in commands.getoutput("pkg-config --libs --cflags %s" % ' '.join(packages)).split():
        kw.setdefault(flag_map.get(token[:2]), []).append(token[2:])
    return kw

setup(
    name = "myPackage",
    ext_modules=[
        Extension("extension", ["extension_main.c"], **pkgconfig('glib-2.0')),
    ],
)

Pkg-config is a very common system for obtaining path information about system libraries in a portable way. Makefiles typically invoke "pkg-config --cflags" to generate compiler flags, and "pkg-config --libs" to generate library flags for the linker.

This recipe makes it easy to use pkg-config's output with distutils. Now you can use system libraries without hard-coding paths, as long as those libraries are distributed with pkg-config data files.

The example builds a module "extension.so" that requires the GLib library.

Tags: extending

3 comments

Thomas Pönitz 16 years, 11 months ago  # | flag

Problems with additional arguments. So I made some changes:

def pkgconfig(packages, *kw):

flag_map = {'-I': 'include_dirs', '-L': 'library_dirs', '-l': 'libraries'}

for token in commands.getoutput("pkg-config --libs --cflags %s" % ' '.join(packages)).split():

    if flag_map.has_key(token[:2]):

        kw.setdefault(flag_map.get(token[:2]), []).append(token[2:])

    else: # throw others to extra_link_args

        kw.setdefault('extra_link_args', []).append(token)


for k, v in kw.iteritems(): # remove duplicated

    kw[k] = list(set(v))

return kw
George Snyder 9 years, 1 month ago  # | flag

Here is a version that handles flags which aren't -I, -L, or -l:

def pkgconfig(*packages, **kw):
    flag_map = {'-I': 'include_dirs', '-L': 'library_dirs', '-l': 'libraries'}
    for token in commands.getoutput("pkg-config --libs --cflags %s" % ' '.join(packages)).split():
        if token[:2] in flag_map:
            kw.setdefault(flag_map.get(token[:2]), []).append(token[2:])
        else:
            kw.setdefault('extra_compile_args', []).append(token)
    return kw
Matej Smid 7 years, 10 months ago  # | flag

Hi, I separated handling of compile and link flags, made the code a bit more general and Python 3 compatible. I made a gist also https://gist.github.com/smidm/ff4a2c079fed97a92e9518bd3fa4797c .

import subprocess


def pkgconfig(*packages, **kw):
    """
    Query pkg-config for library compile and linking options. Return configuration in distutils
    Extension format.

    Usage: 

    pkgconfig('opencv')

    pkgconfig('opencv', 'libavformat')

    pkgconfig('opencv', optional='--static')

    pkgconfig('opencv', config=c)

    returns e.g.  

    {'extra_compile_args': [],
     'extra_link_args': [],
     'include_dirs': ['/usr/include/ffmpeg'],
     'libraries': ['avformat'],
     'library_dirs': []}

     Intended use:

     distutils.core.Extension('pyextension', sources=['source.cpp'], **c)

     Set PKG_CONFIG_PATH environment variable for nonstandard library locations.

    based on work of Micah Dowty (http://code.activestate.com/recipes/502261-python-distutils-pkg-config/)
    """
    config = kw.setdefault('config', {})
    optional_args = kw.setdefault('optional', '')

    # { <distutils Extension arg>: [<pkg config option>, <prefix length to strip>], ...}
    flag_map = {'include_dirs': ['--cflags-only-I', 2],
                'library_dirs': ['--libs-only-L', 2],
                'libraries': ['--libs-only-l', 2],
                'extra_compile_args': ['--cflags-only-other', 0],
                'extra_link_args': ['--libs-only-other', 0],
                }
    for package in packages:
        for distutils_key, (pkg_option, n) in flag_map.items():
            items = subprocess.check_output(['pkg-config', optional_args, pkg_option, package]).decode('utf8').split()
            config.setdefault(distutils_key, []).extend([i[n:] for i in items])
    return config