This context manager provides a convenient, Pythonic way to temporarily replace the file descriptors of
stderr, redirecting to either
os.devnull or files of your choosing. Swapping the C-level file descriptors is required when suppressing output from compiled extension modules, such as those built using F2PY. It functions equally well for pure-Python code. UPDATE: (see below).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108
import os class Silence: """Context manager which uses low-level file descriptors to suppress output to stdout/stderr, optionally redirecting to the named file(s). >>> import sys, numpy.f2py >>> # build a test fortran extension module with F2PY ... >>> with open('hellofortran.f', 'w') as f: ... f.write('''\ ... integer function foo (n) ... integer n ... print *, "Hello from Fortran!" ... print *, "n = ", n ... foo = n ... end ... ''') ... >>> sys.argv = ['f2py', '-c', '-m', 'hellofortran', 'hellofortran.f'] >>> with Silence(): ... # assuming this succeeds, since output is suppressed ... numpy.f2py.main() ... >>> import hellofortran >>> foo = hellofortran.foo(1) Hello from Fortran! n = 1 >>> print "Before silence" Before silence >>> with Silence(stdout='output.txt', mode='w'): ... print "Hello from Python!" ... bar = hellofortran.foo(2) ... with Silence(): ... print "This will fall on deaf ears" ... baz = hellofortran.foo(3) ... print "Goodbye from Python!" ... ... >>> print "After silence" After silence >>> # ... do some other stuff ... ... >>> with Silence(stderr='output.txt', mode='a'): ... # appending to existing file ... print >> sys.stderr, "Hello from stderr" ... print "Stdout redirected to os.devnull" ... ... >>> # check the redirected output ... >>> with open('output.txt', 'r') as f: ... print "=== contents of 'output.txt' ===" ... print f.read() ... print "================================" ... === contents of 'output.txt' === Hello from Python! Hello from Fortran! n = 2 Goodbye from Python! Hello from stderr ================================ >>> foo, bar, baz (1, 2, 3) >>> """ def __init__(self, stdout=os.devnull, stderr=os.devnull, mode='w'): self.outfiles = stdout, stderr self.combine = (stdout == stderr) self.mode = mode def __enter__(self): import sys self.sys = sys # save previous stdout/stderr self.saved_streams = saved_streams = sys.__stdout__, sys.__stderr__ self.fds = fds = [s.fileno() for s in saved_streams] self.saved_fds = map(os.dup, fds) # flush any pending output for s in saved_streams: s.flush() # open surrogate files if self.combine: null_streams = [open(self.outfiles, self.mode, 0)] * 2 if self.outfiles != os.devnull: # disable buffering so output is merged immediately sys.stdout, sys.stderr = map(os.fdopen, fds, ['w']*2, *2) else: null_streams = [open(f, self.mode, 0) for f in self.outfiles] self.null_fds = null_fds = [s.fileno() for s in null_streams] self.null_streams = null_streams # overwrite file objects and low-level file descriptors map(os.dup2, null_fds, fds) def __exit__(self, *args): sys = self.sys # flush any pending output for s in self.saved_streams: s.flush() # restore original streams and file descriptors map(os.dup2, self.saved_fds, self.fds) sys.stdout, sys.stderr = self.saved_streams # clean up for s in self.null_streams: s.close() for fd in self.saved_fds: os.close(fd) return False
Tools like F2PY are a wonderful way to wrap legacy code in beautiful Python bindings. However, it can be cumbersome to deal with suppressing/capturing an extension module's standard output, since the usual tricks like replacing
sys.stdout are not effective. This context manager handles the process of swapping in and out file descriptors so the interfacing code can remain simple and readable. It sure beats digging into digging through the Fortran to rewrite every
The same effect can be accomplished with a
try...finally block, but a context manager is much more reusable. I've avoided hard-coding the file numbers (1 and 2 for
stderr) to allow for nested
Silence contexts (for instance, to capture all output except in a specific case, where output should be redirected differently).
Silence will work on pure-Python code just as well, it's important to note that it is not capable of sending output to
StringIO objects, since they do not implement a
fileno() method. In fact, in this implementation it cannot use open file handles at all--a potential point of improvement.
I found the trick of using
os.dup2 in this answer on Stack Overflow. A writeup on my blog includes a walkthrough with few additional examples.
UPDATE: I've revised the code slightly:
- The class now accounts for some cases where the standard streams weren't being properly flushed on
- For more robust nesting capability,
Silencenow saves and restores the magic
- When the stdout and stderr are to be combined, new unbuffered instances of
sys.stderrare opened so output is written in the proper sequence, instead of writing all stdout first (since it is flushed first). This will probably only work when capturing pure-Python output but it's nice nonetheless.
- Some redundant lines have been removed from the code, and more detailed examples and documentation have been added to the docstring.
UPDATE 2: Added
for fd in self.saved_fds: os.close(fd) to
__exit__ to properly release file descriptors. (Thanks, Andrew!)
The current implementation leaks file descriptors. I discovered this when I ran some code that used this context manager in a long loop. It seems that because the saved file descriptors are never properly released the system limit can quite easily be reached. I patched my version of the code to include an extra line in the __exit__ method before the return line:
This seems to fix the problem.
While this code is over 4 years old, I feel it is incredibly useful. However, it does not seem to work with Python 3. It throws the error:
This seems to be an issue with Python (see for example 'this link').
I don't know that much about python to solve it myself so, is there any suggestion?