Welcome, guest | Sign In | My Account | Store | Cart

Extending the 'logging' module (Python recipe) by Ori Peleg
ActiveState Code (http://code.activestate.com/recipes/474089/)

Adding new format specifiers to the logging module. In this example, it's for the user name and the name of the function that logged the message.

Python, 54 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# xlog.py

import logging

# Adding the 'username' and 'funcname' specifiers
# They must be attributes of the log record

# Custom log record
class OurLogRecord(logging.LogRecord):
    def __init__(self, *args, **kwargs):
        logging.LogRecord.__init__(self, *args, **kwargs)
        self.username = current_user()
        self.funcname = calling_func_name()

# Custom logger that uses our log record
class OurLogger(logging.getLoggerClass()):
    def makeRecord(self, *args, **kwargs):
        return OurLogRecord(*args, **kwargs)

# Register our logger
logging.setLoggerClass(OurLogger)


# Current user
def current_user():
    import pwd, os
    try:
        return pwd.getpwuid(os.getuid()).pw_name
    except KeyError:
        return "(unknown)"

# Calling Function Name
def calling_func_name():
    return calling_frame().f_code.co_name

import os, sys
def calling_frame():
    f = sys._getframe()

    while True:
        if is_user_source_file(f.f_code.co_filename):
            return f
        f = f.f_back

def is_user_source_file(filename):
    return os.path.normcase(filename) not in (_srcfile, logging._srcfile)

def _current_source_file():
    if __file__[-4:].lower() in ['.pyc', '.pyo']:
        return __file__[:-4] + '.py'
    else:
        return __file__

_srcfile = os.path.normcase(_current_source_file())

So you want to add new format specifiers to the logging module? Piece of cake, thanks to Vinay Sajip's clean code (Vinay wrote the logging module).

If you save the code above in xlog.py, the following code will work:

<pre> import logging import xlog # register our custom logger

use the new format specifiers

logging.basicConfig( format="%(filename)s: %(username)s says '%(message)s' in %(funcname)s" )

def foo(): logging.getLogger("trace").warn("Hi mom!")

foo() </pre>

This will print: <pre> prompt> python usercode.py usercode.py: charlie says 'Hi mom!' in foo </pre>

Presented as a lightning talk in OSDC::Israel::2006, http://www.osdc.org.il

Tags: debugging

3 comments

Ryan Mills 17 years, 9 months ago  # | flag

Python 2.4 Error. Trying to do this with Python 2.4 has the following traceback:

import logging
import xlog

logging.basicConfig(level=logging.DEBUG,
                    format="%(name)s %(levelname)s %(message)s in %(funcname)s",
                    filename='UtilTest' + '.log',
                    filemode='w')

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/logging/__init__.py", line 729, in emit
    msg = self.format(record)
  File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/logging/__init__.py", line 615, in format
    return fmt.format(record)
  File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/logging/__init__.py", line 406, in format
    s = self._fmt % record.__dict__
KeyError: 'funcname'
Ori Peleg (author) 17 years, 8 months ago  # | flag

That happens when you use the "root logger" Solution: use a named logger instead of the root logger:

logging.getLogger("somename").error(...)

Explanation:

The extended logger class is only used for loggers that haven't been created yet. The root logger (what you get with getLogger() or getLogger('')) is instantiated before the default logger class can be changed.

Hope this helps...

Tim Black 16 years, 1 month ago  # | flag

update to this recipe for Python 2.5.

Python 2.5 has added the "extra" parameter to Logger.makeRecord. The LogRecord and OurLogRecord constructors do not expect this new arg, so it is an error to pass all args from OurLogger.makeRecord to OurLogRecord.__init__ (and similarly to logging.LogRecord.__init__) using *args. Instead, you should pass the expected arg list as follows:

class OurLogRecord(logging.LogRecord):
    def __init__(self, name, level, fn, lno, msg, args, exc_info, func):
        # Don't pass all args to LogRecord constructor bc it doesn't expect "extra"
        logging.LogRecord.__init__(self, name, level, fn, lno, msg, args, exc_info, func)
        # Adding format specifiers is as simple as adding attributes with
        # same name to the log record object:
        self.funcname = calling_func_name()

class OurLogger(logging.getLoggerClass()):
    def makeRecord(self, name, level, fn, lno, msg, args, exc_info, func=None, extra=None):
        # Don't pass all makeRecord args to OurLogRecord bc it doesn't expect "extra"
        rv = OurLogRecord(name, level, fn, lno, msg, args, exc_info, func)
        # Handle the new extra parameter.
        # This if block was copied from Logger.makeRecord
        if extra:
            for key in extra:
                if (key in ["message", "asctime"]) or (key in rv.__dict__):
                    raise KeyError("Attempt to overwrite %r in LogRecord" % key)
                rv.__dict__[key] = extra[key]
        return rv

Also, it looks to me like the new "extra" parameter is sort of a kludgey way to do the same thing we're trying to accomplish with the subclassing approach.

Thoughts?

Tim