This context manager provides a convenient, Pythonic way to temporarily replace the file descriptors of stdout and stderr, redirecting to either os.devnull or files of your choosing. Swapping the C-level file descriptors is required when suppressing output from compiled extension modules, such as those built using F2PY. It functions equally well for pure-Python code. UPDATE: (see below).
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | import os
class Silence:
"""Context manager which uses low-level file descriptors to suppress
output to stdout/stderr, optionally redirecting to the named file(s).
>>> import sys, numpy.f2py
>>> # build a test fortran extension module with F2PY
...
>>> with open('hellofortran.f', 'w') as f:
... f.write('''\
... integer function foo (n)
... integer n
... print *, "Hello from Fortran!"
... print *, "n = ", n
... foo = n
... end
... ''')
...
>>> sys.argv = ['f2py', '-c', '-m', 'hellofortran', 'hellofortran.f']
>>> with Silence():
... # assuming this succeeds, since output is suppressed
... numpy.f2py.main()
...
>>> import hellofortran
>>> foo = hellofortran.foo(1)
Hello from Fortran!
n = 1
>>> print "Before silence"
Before silence
>>> with Silence(stdout='output.txt', mode='w'):
... print "Hello from Python!"
... bar = hellofortran.foo(2)
... with Silence():
... print "This will fall on deaf ears"
... baz = hellofortran.foo(3)
... print "Goodbye from Python!"
...
...
>>> print "After silence"
After silence
>>> # ... do some other stuff ...
...
>>> with Silence(stderr='output.txt', mode='a'):
... # appending to existing file
... print >> sys.stderr, "Hello from stderr"
... print "Stdout redirected to os.devnull"
...
...
>>> # check the redirected output
...
>>> with open('output.txt', 'r') as f:
... print "=== contents of 'output.txt' ==="
... print f.read()
... print "================================"
...
=== contents of 'output.txt' ===
Hello from Python!
Hello from Fortran!
n = 2
Goodbye from Python!
Hello from stderr
================================
>>> foo, bar, baz
(1, 2, 3)
>>>
"""
def __init__(self, stdout=os.devnull, stderr=os.devnull, mode='w'):
self.outfiles = stdout, stderr
self.combine = (stdout == stderr)
self.mode = mode
def __enter__(self):
import sys
self.sys = sys
# save previous stdout/stderr
self.saved_streams = saved_streams = sys.__stdout__, sys.__stderr__
self.fds = fds = [s.fileno() for s in saved_streams]
self.saved_fds = map(os.dup, fds)
# flush any pending output
for s in saved_streams: s.flush()
# open surrogate files
if self.combine:
null_streams = [open(self.outfiles[0], self.mode, 0)] * 2
if self.outfiles[0] != os.devnull:
# disable buffering so output is merged immediately
sys.stdout, sys.stderr = map(os.fdopen, fds, ['w']*2, [0]*2)
else: null_streams = [open(f, self.mode, 0) for f in self.outfiles]
self.null_fds = null_fds = [s.fileno() for s in null_streams]
self.null_streams = null_streams
# overwrite file objects and low-level file descriptors
map(os.dup2, null_fds, fds)
def __exit__(self, *args):
sys = self.sys
# flush any pending output
for s in self.saved_streams: s.flush()
# restore original streams and file descriptors
map(os.dup2, self.saved_fds, self.fds)
sys.stdout, sys.stderr = self.saved_streams
# clean up
for s in self.null_streams: s.close()
for fd in self.saved_fds: os.close(fd)
return False
|
Tools like F2PY are a wonderful way to wrap legacy code in beautiful Python bindings. However, it can be cumbersome to deal with suppressing/capturing an extension module's standard output, since the usual tricks like replacing sys.stdout are not effective. This context manager handles the process of swapping in and out file descriptors so the interfacing code can remain simple and readable. It sure beats digging into digging through the Fortran to rewrite every PRINT statement!
The same effect can be accomplished with a try...finally block, but a context manager is much more reusable. I've avoided hard-coding the file numbers (1 and 2 for stdout and stderr) to allow for nested Silence contexts (for instance, to capture all output except in a specific case, where output should be redirected differently).
While Silence will work on pure-Python code just as well, it's important to note that it is not capable of sending output to StringIO objects, since they do not implement a fileno() method. In fact, in this implementation it cannot use open file handles at all--a potential point of improvement.
I found the trick of using os.dup2 in this answer on Stack Overflow. A writeup on my blog includes a walkthrough with few additional examples.
UPDATE: I've revised the code slightly:
- The class now accounts for some cases where the standard streams weren't being properly flushed on
__enter__and__exit__. - For more robust nesting capability,
Silencenow saves and restores the magicsys.__stdout__andsys.__stderr__streams. - When the stdout and stderr are to be combined, new unbuffered instances of
sys.stdoutandsys.stderrare opened so output is written in the proper sequence, instead of writing all stdout first (since it is flushed first). This will probably only work when capturing pure-Python output but it's nice nonetheless. - Some redundant lines have been removed from the code, and more detailed examples and documentation have been added to the docstring.
UPDATE 2: Added for fd in self.saved_fds: os.close(fd) to __exit__ to properly release file descriptors. (Thanks, Andrew!)
The current implementation leaks file descriptors. I discovered this when I ran some code that used this context manager in a long loop. It seems that because the saved file descriptors are never properly released the system limit can quite easily be reached. I patched my version of the code to include an extra line in the __exit__ method before the return line:
This seems to fix the problem.
While this code is over 4 years old, I feel it is incredibly useful. However, it does not seem to work with Python 3. It throws the error:
This seems to be an issue with Python (see for example 'this link').
I don't know that much about python to solve it myself so, is there any suggestion?
Best regards