Welcome, guest | Sign In | My Account | Store | Cart

The recipe shows how to subclass python-daemon's [1] DaemonContext to add logging support. In particular, it is possible to ask to keep the files related to a list of loggers (loggers_preserve) open and to redirect stdout and stderr to a logger (e.g., one using a RotatingFileHandler).

[1] See http://pypi.python.org/pypi/python-daemon/

Python, 186 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
#!/usr/bin/env python

import os
import sys
from logging import Logger
import daemon

class FileLikeLogger:
    "wraps a logging.Logger into a file like object"

    def __init__(self, logger):
        self.logger = logger

    def write(self, str):
        str = str.rstrip() #get rid of all tailing newlines and white space
        if str: #don't log emtpy lines
            for line in str.split('\n'):
                self.logger.critical(line) #critical to log at any logLevel

    def flush(self):
        for handler in self.logger.handlers:
            handler.flush()

    def close(self):
        for handler in self.logger.handlers:
            handler.close()

def openFilesFromLoggers(loggers):
    "returns the open files used by file-based handlers of the specified loggers"
    openFiles = []
    for logger in loggers:
        for handler in logger.handlers:
            if hasattr(handler, 'stream') and \
               hasattr(handler.stream, 'fileno'):
                openFiles.append(handler.stream)
    return openFiles
      
class LoggingDaemonContext(daemon.DaemonContext):

    def _addLoggerFiles(self):
        "adds all files related to loggers_preserve to files_preserve"
        for logger in [self.stdout_logger, self.stderr_logger]:
            if logger:
                self.loggers_preserve.append(logger)
        loggerFiles = openFilesFromLoggers(self.loggers_preserve)
        self.files_preserve.extend(loggerFiles)

    def __init__(
        self,
        chroot_directory=None,
        working_directory='/',
        umask=0,
        uid=None,
        gid=None,
        prevent_core=True,
        detach_process=None,
        files_preserve=[],   # changed default
        loggers_preserve=[], # new
        pidfile=None,
        stdout_logger = None,  # new
        stderr_logger = None,  # new
        #stdin,   omitted!
        #stdout,  omitted!
        #sterr,   omitted!
        signal_map=None,
        ):

        self.stdout_logger = stdout_logger
        self.stderr_logger = stderr_logger
        self.loggers_preserve = loggers_preserve

        devnull_in = open(os.devnull, 'r+')
        devnull_out = open(os.devnull, 'w+')
        files_preserve.extend([devnull_in, devnull_out])

        daemon.DaemonContext.__init__(self,
            chroot_directory = chroot_directory,
            working_directory = working_directory,
            umask = umask,
            uid = uid,
            gid = gid,
            prevent_core = prevent_core,
            detach_process = detach_process,
            files_preserve = files_preserve, 
            pidfile = pidfile,
            stdin = devnull_in,
            stdout = devnull_out,
            stderr = devnull_out,
            signal_map = signal_map) 

    def open(self): 
        self._addLoggerFiles() 
        daemon.DaemonContext.open(self)
        if self.stdout_logger:
            fileLikeObj = FileLikeLogger(self.stdout_logger)
            sys.stdout = fileLikeObj
        if self.stderr_logger:
            fileLikeObj = FileLikeLogger(self.stderr_logger)
            sys.stderr = fileLikeObj


#---------------------------------------------------------------
if __name__ == '__main__':

    # since this test uses chroot, it should be called as superuser (sudo..)

    import logging
    import logging.handlers
    import random
    import time
    import urllib2

    #-- setting up a rotating file logger -------
    def getRotFileLogger(name, filePath, logLevel=logging.DEBUG, format=None):
        format = format or '%(message)s'
        my_logger = logging.getLogger(name)
        my_logger.setLevel(logLevel)
        handler = logging.handlers.RotatingFileHandler(
                      filePath, maxBytes=2000, backupCount=2)
        formatter = logging.Formatter(format)
        handler.setFormatter(formatter)
        my_logger.addHandler(handler)
        return my_logger

    #-- some utilities ---
    def rmTestFiles(*fList):
        for f in fList:
            rmTestFile(f)

    def rmTestFile(fileName):
        if os.path.isfile(fileName):
            os.remove(fileName)

    def rmTestDir(dirName):
        if os.path.isdir(dirName):
            os.rmdir(dirName)

    #-- clean up the test directory and file structure ----
    rmTestFile('jail/beacon.txt')
    rmTestFile('jail/jailTestFile.txt')
    rmTestDir('jail')
    rmTestFiles('test.log', 'stdout.log', 'stderr.log', 'test.file')
    os.mkdir(os.path.join(os.getcwd(), 'jail'))
    open('jail/beacon.txt', 'w').write('I should be found')

    #-- set up loggers and files for the daemon
    testLogger = getRotFileLogger('test', 'test.log')
    stdoutLogger = getRotFileLogger('stdout', 'stdout.log')
    stderrLogger = getRotFileLogger('stderr', 'stderr.log')
    testFile = open('test.file', 'w')

    #-- test that all work before opening the DaemonContext
    testLogger.info('testLogger: before opening context')
    stdoutLogger.info('stdoutLogger: before opening context')
    stderrLogger.info('stderrLogger: before opening context')
    testFile.write('testFile: hello to a file before context\n')

    #-- get and configure the DaemonContext
    context = LoggingDaemonContext()
    context.files_preserve=[testFile]
    context.loggers_preserve=[testLogger]
    context.stdout_logger = stdoutLogger
    context.stderr_logger = stderrLogger
    context.chroot_directory = os.path.join(os.getcwd(), 'jail')

    #-- test whether it all works
    with context:
        # should appear in stdout.log
        print "stdout: hello!"
        # should appear in test.log
        testLogger.info('testLogger: hello, just testing')
        # should appear in test.file
        testFile.write('testFile: hello to a file\n')

        #testing chroot
        print "If chroot works, I should see beacon.txt: %s" % os.listdir('.')
        open('jailTestFile.txt', 'w').write('this was written in the jail\n')
        # do these things need access to devices???:
        print "time is: %s" % time.time() 
        print "this is a random number: %s" % random.randint(1, 100)
        # the above seems to work without errors
        # but the following needs access to devices that are not available in the jail
        print urllib2.urlopen('http://www.python.org/').read(20)

        # should appear in stderr.file
        raise (Exception("stderrLogger: bummer!"))

2 comments

Pedro Algarvio 10 years, 11 months ago  # | flag

I've tweaked your recipe a bit, http://paste.pocoo.org/show/311209/

My issue is that I get an "Aborted" message on the console while running my code with the above paste(with sudo). However if sudo is not used although I'm unable to chroot, the daemon works properly.

Look at the output: http://paste.pocoo.org/show/311210/

What could it be?

Regards, Pedro Algarvio.

Bud P. Bruegger (author) 10 years, 11 months ago  # | flag

Hi Pedro,

I had a quick look and don't understand where the "Aborted" could come from. It seems to me that it is not in my recipe, not in your tweaking code, and not in daemon.py; I assume it must come from one of your inports.

What I can tell from your output is that chroot doesn't work. Setting up chroot correctly is rather difficult (I have postponed that on my project and also haven't tested my recipe in a chroot environment) and I would recommend to test your code without chrooting in a first step.

In my experience, if something goes wrong with python daemon, often there is no feedback on what went wrong. You may want to try to comment out "close_all_open_files" in DaemonContext.open (daemon.py) during debugging.

HTH

-bud

PS. Maybe when you reuse other people's code in your work, you could add a link to the source next to your copyright?

Created by Bud P. Bruegger on Mon, 25 Oct 2010 (BSD)
Python recipes (4591)
Bud P. Bruegger's recipes (2)

Required Modules

  • (none specified)

Other Information and Tasks