Fork a daemon process on Unix (Python recipe) by Jürgen Hermann

ActiveState Code (http://code.activestate.com/recipes/66012/)

os.chdir('/') is not needed.

Sure a "cd /" is needed. If you ever wanted to delete the directory you started a demon in that does NOT have that code, you'd think otherwise.

It's also worth redirecting the standard file descriptors. Check out the UNIX Programming FAQ for the full skinny on forking a daemon:

http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16

This cookbook entry's a pretty good implementation of the above, except that one of the things the FAQ entry advises is closing the standard file descriptors and redirecting them elsewhere (eg /dev/null, or log files, etc.). I've been bitten in the ass by forgetting to do this, so it's worth remembering. :-)

In the simplest form:

# Redirect standard file descriptors
sys.stdin = open('/dev/null', 'r')
sys.stdout = open('/dev/null', 'w')
sys.stderr = open('/dev/null', 'w')

The FAQ entry also does things in a slightly different order - in particular it doesn't set the umask, or chdir('/') until after the second fork - but I don't know if that's important.

It also gives further justification for calling os.chdir('/'), namely "what if the sysadmin wants to umount the filesystem the daemon was run from?". Good stuff...

More reliable i/o stream redirection. Just reassigning to the sys streams is not 100% effective if you are importing modules that write to stdin and stdout from C code. Perhaps the modules shouldn't do that, but this code will make sure that all stdin and stdout will go where you expect it to.

import os, sys

out_log = file('/out/log/file/name', 'a+')
err_log = file('/err/log/file/name', 'a+', 0)
dev_null = file('/dev/null', 'r')

os.dup2(out_log.fileno(), sys.stdout.fileno())
os.dup2(err_log.fileno(), sys.stderr.fileno())
os.dup2(dev_null.fileno(), sys.stin.fileno())

oops... Of course, that should be:

... os.stdin ...

not

... os.stin ...

Wrapping it all together... So taking everyone's suggestions I got this. Just save this into a file called "daemonize.py". If you run it as a script it will daemonize an example main() function. It should be self-documenting...

Did I miss anything?

Noah

#!/usr/bin/env python
import sys, os

'''This module is used to fork the current process into a daemon.
    Almost none of this is necessary (or advisable) if your daemon
    is being started by inetd. In that case, stdin, stdout and stderr are
    all set up for you to refer to the network connection, and the fork()s
    and session manipulation should not be done (to avoid confusing inetd).
    Only the chdir() and umask() steps remain as useful.
    References:
        UNIX Programming FAQ
            1.7 How do I get my program to act like a daemon?
                http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16

        Advanced Programming in the Unix Environment
            W. Richard Stevens, 1992, Addison-Wesley, ISBN 0-201-56317-7.
    '''

def daemonize (stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
    '''This forks the current process into a daemon.
    The stdin, stdout, and stderr arguments are file names that
    will be opened and be used to replace the standard file descriptors
    in sys.stdin, sys.stdout, and sys.stderr.
    These arguments are optional and default to /dev/null.
    Note that stderr is opened unbuffered, so
    if it shares a file with stdout then interleaved output
    may not appear in the order that you expect.
    '''
    # Do first fork.
    try:
        pid = os.fork()
        if pid > 0:
            sys.exit(0) # Exit first parent.
    except OSError, e:
        sys.stderr.write ("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror)    )
        sys.exit(1)

    # Decouple from parent environment.
    os.chdir("/")
    os.umask(0)
    os.setsid()

    # Do second fork.
    try:
        pid = os.fork()
        if pid > 0:
            sys.exit(0) # Exit second parent.
    except OSError, e:
        sys.stderr.write ("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror)    )
        sys.exit(1)

    # Now I am a daemon!

    # Redirect standard file descriptors.
    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())

(comment continued...)

(...continued from previous comment)

def main ():
    '''This is an example main function run by the daemon.
    This prints a count and timestamp once per second.
    '''
    import time
    sys.stdout.write ('Daemon started with pid %d\n' % os.getpid() )
    sys.stdout.write ('Daemon stdout output\n')
    sys.stderr.write ('Daemon stderr output\n')
    c = 0
    while 1:
        sys.stdout.write ('%d: %s\n' % (c, time.ctime(time.time())) )
        sys.stdout.flush()
        c = c + 1
        time.sleep(1)

if __name__ == "__main__":
    daemonize('/dev/null','/tmp/daemon.log','/tmp/daemon.log')
    main()

Things can be done much easier. I took a look at the code, and I remembered that I've written various Perl daemons some time ago. So, I took their source and translated it into python code. Here's what works for me

!/usr/bin/python

import sys import os try: pid = os.fork() except: print "Could not fork" sys.exit(1) if pid: # Let parent die ! sys.exit(0) else: try: # Create new session os.setsid() except: print "Could not create new session" sys.exit(1) while 1: # Your daemon code here

That's all, so I'm asking myself why you try these double-fork stuff ?

Adding start/stop/restart behavior.

'''
    This module is used to fork the current process into a daemon.
    Almost none of this is necessary (or advisable) if your daemon
    is being started by inetd. In that case, stdin, stdout and stderr are
    all set up for you to refer to the network connection, and the fork()s
    and session manipulation should not be done (to avoid confusing inetd).
    Only the chdir() and umask() steps remain as useful.
    References:
        UNIX Programming FAQ
            1.7 How do I get my program to act like a daemon?
                http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
        Advanced Programming in the Unix Environment
            W. Richard Stevens, 1992, Addison-Wesley, ISBN 0-201-56317-7.

    History:
      2001/07/10 by Jürgen Hermann
      2002/08/28 by Noah Spurrier
      2003/02/24 by Clark Evans

      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012
'''
import sys, os, time
from signal import SIGTERM

def deamonize(stdout='/dev/null', stderr=None, stdin='/dev/null',
              pidfile=None, startmsg = 'started with pid %s' ):
    '''
        This forks the current process into a daemon.
        The stdin, stdout, and stderr arguments are file names that
        will be opened and be used to replace the standard file descriptors
        in sys.stdin, sys.stdout, and sys.stderr.
        These arguments are optional and default to /dev/null.
        Note that stderr is opened unbuffered, so
        if it shares a file with stdout then interleaved output
        may not appear in the order that you expect.
    '''
    # Do first fork.
    try:
        pid = os.fork()
        if pid > 0: sys.exit(0) # Exit first parent.
    except OSError, e:
        sys.stderr.write("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)

    # Decouple from parent environment.
    os.chdir("/")
    os.umask(0)
    os.setsid()

    # Do second fork.
    try:
        pid = os.fork()
        if pid > 0: sys.exit(0) # Exit second parent.
    except OSError, e:
        sys.stderr.write("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)

    # Open file descriptors and print start message
    if not stderr: stderr = stdout
    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    pid = str(os.getpid())
    sys.stderr.write("\n%s\n" % startmsg % pid)
    sys.stderr.flush()
    if pidfile: file(pidfile,'w+').write("%s\n" % pid)

(comment continued...)

(...continued from previous comment)

    # Redirect standard file descriptors.
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())

def startstop(stdout='/dev/null', stderr=None, stdin='/dev/null',
              pidfile='pid.txt', startmsg = 'started with pid %s' ):
    if len(sys.argv) > 1:
        action = sys.argv[1]
        try:
            pf  = file(pidfile,'r')
            pid = int(pf.read().strip())
            pf.close()
        except IOError:
            pid = None
        if 'stop' == action or 'restart' == action:
            if not pid:
                mess = "Could not stop, pid file '%s' missing.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)
            try:
               while 1:
                   os.kill(pid,SIGTERM)
                   time.sleep(1)
            except OSError, err:
               err = str(err)
               if err.find("No such process") > 0:
                   os.remove(pidfile)
                   if 'stop' == action:
                       sys.exit(0)
                   action = 'start'
                   pid = None
               else:
                   print str(err)
                   sys.exit(1)
        if 'start' == action:
            if pid:
                mess = "Start aborded since pid file '%s' exists.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)
            deamonize(stdout,stderr,stdin,pidfile,startmsg)
            return
    print "usage: %s start|stop|restart" % sys.argv[0]
    sys.exit(2)

def test():
    '''
        This is an example main function run by the daemon.
        This prints a count and timestamp once per second.
    '''
    sys.stdout.write ('Message to stdout...')
    sys.stderr.write ('Message to stderr...')
    c = 0
    while 1:
        sys.stdout.write ('%d: %s\n' % (c, time.ctime(time.time())) )
        sys.stdout.flush()
        c = c + 1
        time.sleep(1)

if __name__ == "__main__":
    startstop(stdout='/tmp/deamonize.log',
              pidfile='/tmp/deamonize.pid')
    test()

Why double-fork? I'm sure the references (Stevens, FAQ) explain this. The reason is we don't like zombies.

The second fork _is_ necessary. The first fork accomplishes two things - allow the shell to return, and allow you to do a setsid().

The setsid() removes yourself from your controlling terminal. You see, before, you were still listed as a job of your previous process, and therefore the user might accidentally send you a signal. setsid() gives you a new session, and removes the existing controlling terminal.

The problem is, you are now a session leader. As a session leader, if you open a file descriptor that is a terminal, it will become your controlling terminal (oops!). Therefore, the second fork makes you NOT be a session leader. Only session leaders can acquire a controlling terminal, so you can open up any file you wish without worrying that it will make you a controlling terminal.

So - first fork - allow shell to return, and permit you to call setsid()

Second fork - prevent you from accidentally reacquiring a controlling terminal.

And here is the translation in German.

#!/usr/bin/python


#       Dieses Modul wird verwendet, um sein aktuelles Skript in eine Daemon-App. zu versetzen.
#       Sollte ihr Daemon ueber inetd laufen (s. inet daemon Linux) so sind die meisten, hier
#   angefuehrten Routinen nicht notwendig. Ist dies der Fall, dass std -in,-out u. -err ueber
#   das "Netzwerk" arbeiten, so sollten forks-commands u. Session Manipulation vermieden werden.
#   Nur chdir() und unmask() erweisen sich in diesem Fall als brauchbar.
#
#       Quellennachweise:
#            UNIX Programming FAQ
#                   1.7 How do I get my program to act like a daemon?
#                   http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
#           Advanced Programming in the Unix Environment
#                   W. Richard Stevens, 1992, Addison-Wesley, ISBN 0-201-56317-7.
#           (http://www.yendor.com/programming/unix/apue/ch13.html)
#
#   Geschichte:
#           2001/07/10 by J|rgen Hermann
#           2002/08/28 by Noah Spurrier
#           2003/02/24 by Clark Evans
#       2004/06/16 by Peter Landl / IFO.net
#
#       http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012

import sys, os, time
from signal import SIGTERM

def deamonize(stdout = '/dev/null', stderr = None, stdin = '/dev/null', pidfile = None, startmsg = 'started with pid %s' ):
    '''
    Diese Routine gabelt ("forks", kloned) den aktuellen Prozess in einen Daemon.
    Die Parameter stdin, stdout und stderr sind Dateinamen, welche die
    Standard- Err-/Ein-/Aus- gabe "ersetzen". Diese Argumente sind optional
    und zeigen standardmaessig ins Nirvana (/dev/null).
    Zu beachten ist, dass stderr ungepuffert geoeffnet ist und so, wenn es
    doppelt offen scheint, nicht die Daten enthaellt, die sie erwarten.
        [Letzter Satz im Original: Note that stderr is opened unbuffered, so
         if it shares a file with stdout then interleaved output
         may not appear in the order that you expect.]
    '''
    # Erstes fork (erster Klon) => fork erstellt aus dem gesamten Prozess einen child-Prozess
    try:
        pid = os.fork()
        if (pid > 0):
        sys.exit(0) # Parent-Prozess schliessen
    except OSError, e:
        sys.stderr.write("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)

    # Parent "Table/Umgebung" verlassen
    os.chdir("/")
    os.umask(0)
    os.setsid()

    # Zweites fork
    try:
        pid = os.fork()
        if (pid > 0):
        sys.exit(0) # Zweiten Parent-Prozess schliessen
    except OSError, e:
        sys.stderr.write("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)

(comment continued...)

(...continued from previous comment)

    # Standard Ein-/Ausgaben oeffnen und Standard-message ausgeben
    if (not stderr):    # Wurde stderr nicht uebergeben => stdout-Pfad nehmen
    stderr = stdout

    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    pid = str(os.getpid())
    sys.stderr.write("\n%s\n" % startmsg % pid)
    sys.stderr.flush()
    if pidfile: file(pidfile,'w+').write("%s\n" % pid)

    # Standard Ein-/Ausgaben auf die Dateien umleiten
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())

# start/stop/restart Routine fuer den Daemon
def startstop(stdout='/dev/null', stderr=None, stdin='/dev/null', pidfile='pid.txt', startmsg = 'gestartet mit pid %s' ):

    if len(sys.argv) > 1:
        action = sys.argv[1]
        try:
            pf  = file(pidfile,'r')
            pid = int(pf.read().strip())    # Von geoeffnetem pid-File "pid" auslesen um Prozess zu kontrollieren (ueber kill..)
            pf.close()
        except IOError:
            pid = None

        if ((action == 'stop') or (action == 'restart')):
            if (not pid):
                mess = "Konnte Prozess nicht stoppen, pid-Datei '%s' fehlt.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)
            try:
               while 1:
                   os.kill(pid,SIGTERM)
                   time.sleep(1)
            except OSError, err:
               err = str(err)
               if err.find("No such process") > 0:
                   os.remove(pidfile)
                   if 'stop' == action:
                       sys.exit(0)
                   action = 'start'
                   pid = None
               else:
                   print str(err)
                   sys.exit(1)
        if ('start' == action):
            if (pid):
                mess = "Start abgebrochen, da Daemon (bzw. Daemon-pid-file '%s') noch existiert.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)
            deamonize(stdout,stderr,stdin,pidfile,startmsg)
            return
    print "Daemon-Syntax: %s start|stop|restart" % sys.argv[0]
    sys.exit(2)

(comment continued...)

(...continued from previous comment)

def test():
    '''
    Das ist eine Test-Funktion, welche ueber den Daemon laeuft.
    Diese Routine schreibt jede Sekunde den Zaehlerstand u. einen Zeitstempel in die Standard-Ausgabe
    '''
    sys.stdout.write ('Text zur Standard-Ausgabe senden...')
    sys.stderr.write ('Text zur Standard-Fehler-Ausgabe senden...')
    c = 0
    while 1:
        sys.stdout.write ('%d: %s\n' % (c, time.ctime(time.time())) )
        sys.stdout.flush()
        c = c + 1
        time.sleep(1)

if (__name__ == "__main__"):
    startstop(stdout = '/tmp/deamonize.log', pidfile = '/tmp/deamonize.pid')

    test()

Restart should handle not finding the pid. This is very useful code, but there is one detail that is wrong. If startstop() gets action='restart' and pid has not been set, the daemon is not started. Instead, the code reports "Could not stop, pid file '%s' missing." and calls sys.exit(1).

The code in the next few lines handles this correctly: if the os.kill() fails then the exception handler sets action = 'start' and falls through to allow the daemon to be started.

So, the code should read:

if 'stop' == action or 'restart' == action:
   if not pid:
       mess = "Could not stop, pid file '%s' missing.\n"
       sys.stderr.write(mess % pidfile)
       if 'stop' == action:
           sys.exit(1)
       action = 'start'
       pid = None
   else:
      try:
         while 1:
             os.kill(pid,SIGTERM)
             time.sleep(1)
      except OSError, err:
         err = str(err)
         if err.find("No such process") > 0:
             os.remove(pidfile)
             if 'stop' == action:
                 sys.exit(0)
             action = 'start'
             pid = None
         else:
             print str(err)
             sys.exit(1)

even safer to flush first. Before dup'ing a new file into the underlying stdout/stderr file descriptors, you should flush the stdio buffers. Otherwise, it is entirely possible that pending output could get sent to the wrong file.

Thus:

sys.stdout.flush()
sys.stderr.flush()
os.dup2(out_log.fileno(), sys.stdout.fileno())
os.dup2(err_log.fileno(), sys.stderr.fileno())
os.dup2(dev_null.fileno(), sys.stdin.fileno())

More general startstop method. Made startstop more general by including the action parameter. That enables you to call it from everywhere. Also added status parameter, that performs a very simple check if process is running.

def startstop(stdout='/dev/null', stderr=None, stdin='/dev/null',
              pidfile='pid.txt', startmsg = 'started with pid %s', action='start' ):
    if action:
        try:
            pf  = file(pidfile,'r')
            pid = int(pf.read().strip())
            pf.close()
        except IOError:
            pid = None

        if 'stop' == action or 'restart' == action:
            if not pid:
                mess = "Could not stop, pid file '%s' missing.\n"
                sys.stderr.write(mess % pidfile)
                if 'stop' == action:
                    sys.exit(1)
                action = 'start'
                pid = None
            else:
               try:
                  while 1:
                      os.kill(pid,SIGTERM)
                      time.sleep(1)
               except OSError, err:
                  err = str(err)
                  if err.find("No such process") > 0:
                      os.remove(pidfile)
                      if 'stop' == action:
                          sys.exit(0)
                      action = 'start'
                      pid = None
                  else:
                      print str(err)
                      sys.exit(1)

        if 'start' == action:
            if pid:
                mess = "Start aborded since pid file '%s' exists.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)

            deamonize(stdout,stderr,stdin,pidfile,startmsg)
            return

        if 'status' == action:
            if not pid:
                sys.stderr.write('Status: Stopped\n')

            else: sys.stderr.write('Status: Running\n')
            sys.exit(0)

Advanced Programming in the UNIX(R) Environment (2nd Edition).

BTW, There is now a second edition of this great boot.
ISBN: 0-201-43307-9
http://www.awprofessional.com/title/0201433079

close newfd before dup2.

The Linux man page of dup2 says:

"""
int dup2(int oldfd, int newfd)
...
If newfd was open, any errors that would have been reported at  close()time,  are lost.
A careful programmer will not use dup2 without closing newfd first.
"""

You can add this before dup2:
    os.close(sys.stdin.fileno())
    os.close(sys.stdout.fileno())
    os.close(sys.stderr.fileno())

I don't know if os.close() flushes python interal buffers.
Doing sys.stdout.flush() and sys.stderr.flush() before the close
does not hurt.

PS: sys.stdin.close() does not work

file(stdin) dos not work ? My python didn't accept the three lines:

si = file(stdin, 'r')
so = file(stdout, 'a+')
se = file(stderr, 'a+', 0)

.. it gave me a name error:

Python 2.3.4 (#1, Mar 10 2006, 06:12:09)
[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os,sys
>>> si = file(stdin, 'r')
Traceback (most recent call last):
  File "", line 1, in ?
NameError: name 'stdin' is not defined

.. should it be '/dev/stdin':

>>> si = file('/dev/stdin', 'r')
<open file '/dev/stdin', mode 'r' at 0xb7f9f2a0>

.. or perhaps sys.stdin (which does not work) ?

>>> si = file(sys.stdin, 'r')
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, file found

Implemenation based on this recipe and discussion. Some time ago we made an implementation of this daemonizing feature.

Its a class you can use to daemonize any class providing a run method and the stdin, stdout and stderr as class attributes.

For interested people the code is available here:

http://svn.plone.org/svn/collective/bda.daemon

here's another implementation for 2008 ;)

this focuses on the startup part, not the daemonize itself, in a way i feel makes the flow a bit more clear.

class Actions():

    startmsg = "Started with pid %s\n"
    abortmsg = "Start aborted; pid file '%s' exists.\n"
    stopfailmsg = "Could not stop, pid file '%s' missing.\n"
    stdout='/dev/null'
    stderr=None
    stdin='/dev/null'
    pidfile='pid.txt'
    pidstomp=False      # allows start() to stomp the pid file if desired

    def __init__(self, **keys ):
        # assign any overrides to self
        for n,v in keys.iteritems():
            setattr( self, n, v )

        # setup pid number
        try:
            pf = file(self.pidfile,'r')
            self.pid = int(pf.read().strip())
            pf.close()
        except IOError:
            self.pid = None

    def start( self ):
        if self.pid and not self.pidstomp:
            sys.stderr.write( self.abortmsg % self.pidfile )
            sys.exit(1)
        daemonize(self.stdout,self.stderr,self.stdin,self.pidfile,self.startmsg)
        return True

    def restart( self ):
        if self.pid:
            self.stop()
        self.start()

    def stop( self ):
        if not self.pid:
            sys.stderr.write( self.stopfailmsg % self.pidfile )
            sys.exit(1)
        try:
            while 1:
                os.kill(self.pid,SIGTERM)
                time.sleep(1)
        except OSError, err:
            err = str(err)
            if err.find("No such process") > 0:
                os.remove(self.pidfile)
            else:
                sys.stderr.write( err )
                sys.exit(1)

if __name__ == '__main__':

    try:
        action= sys.argv[1]
        method= getattr( Actions, action )
    except IndexError, KeyError:
        print "usage: %s start|stop|restart" % sys.argv[0]
        sys.exit(2)

    print "Running %s for daemon..." % action

    # could replace ... with fixed options, or parse args from sys.argv 
    if method( Actions( ...  ) ):
        run()               

    sys.exit(0)