Popular Python recipes tagged "meta:requires=logging"

JSON Formatted Logging (Python)

2016-05-27T01:07:42-07:00

Python recipe 580667 by Michael Blan Palmer (json, logging, python).

I have created a package that outputs JSON formatted lines to a log file. It can make use of the standard logging parameters and/or take custom input. The use of JSON in the log file allows for easy filtering and processing.

How to use super() effectively -- Python 3.2 version (Python)

2015-12-01T16:23:09-08:00

Python recipe 579130 by Lance E Sloan (cooperative, inheritance, multiple, next_in_mro, super).

Python's super() provides a unique and amazing capability. It allows subclasses to be written to reorder a chain method calls. The recipe demonstrates all of the tradecraft needed to get super() to do your bidding.

How to use super() effectively -- Python 2.7 version (Python)

2015-12-01T16:20:26-08:00

Python recipe 579129 by Lance E Sloan (cooperative, inheritance, multiple, next_in_mro, super).

Set the logging level to every logger (Python)

2015-10-28T23:16:26-07:00

Python recipe 579119 by Tim McNamara (configuration, logging).

Python's logging allocates a name to every logger. That makes it hard to do something like, setting everything to logging.ERROR. Here's one way you might go about that:

How to use super() effectively -- Python 2.7 version (Python)

2011-06-01T19:19:37-07:00

Python recipe 577721 by Raymond Hettinger (cooperative, inheritance, multiple, next_in_mro, super). Revision 8.

NondurableLogger class for use with concurrent.futures.ProcessPoolExecutor's submit and map methods (Python)

2014-02-10T17:50:59-08:00

Python recipe 578827 by Peter Santoro (logging, multiprocessing). Revision 2.

I needed a simple logging solution that I could use with with concurrent.futures.ProcessPoolExecutor and this is my initial recipe.

Recursive Multimedia (audio, video) M3U Playlist Generator (Python)

2014-02-08T01:03:36-08:00

Python recipe 578771 by Mano Bastardo (audio, ffmpeg, generate, m3u, mulitmedia, os, playlist, python, system, video). Revision 9.

Generate an m3u playlist searching recursively for multimedia files (video or audio) in the given directory. Information from ID3 tags will be extracted for audio files with FFmpeg available.

How to use super() effectively (Python)

2011-06-01T19:17:58-07:00

Python recipe 577720 by Raymond Hettinger (cooperative, inheritance, multiple, next_in_mro, super). Revision 8.

Print Logger Internals (Python)

2012-12-24T04:11:35-08:00

Python recipe 578389 by Dave Bailey (logging).

This helper function simplifies the difficult task of setting up and maintaining a logging system. Changes to a logging system can cause unanticipated consequences such as lost messages or duplicates. Debugging a logging hierarchy can be a tedious task. This function overrides the internal __repr__ functions of the internal classes and allows a print statement to generate the complete logger hierarchy with its associated internals. It allows easy debugging a logger and allows changes to be easily detected.

Distributed lock manager for Python (Python)

2012-07-04T21:03:32-07:00

Python recipe 578194 by pavel (distributed, lock, locking, locks, manager, socket, thread, threading, wsgi). Revision 2.

Distributed lock manager provides mutex(es) over network. It is used to synchronize processes running on different machines, e.g. WSGI processes in case of web applications. Lock object is compatible with threading.Lock and can be used as a context manager ("with statement"). It can be easily modified to use UNIX sockets instead of TCP/IP. Communication protocol is text based.

First start server process:

$ chmod +x dlm.py
$ ./dlm.py

Usage:

from dlm import LockClient

client = LockClient('localhost', 27272, 'client_name')

lock = client.mkLock('lock_name')

lock.acquire()
# critical section here...
lock.release()

# using context manager
with lock:
    # critical section here...

Directory Pruner 4 (Python)

2012-06-06T22:00:00-07:00

Python recipe 578154 by Stephen Chappell (delete, directory, file, gui, utility). Revision 2.

Module providing GUI capability to prune any directory.

The code presented in this module is for the purposes of: (1) ascertaining the space taken up by a directory, its files, its sub-directories, and its sub-files; (2) allowing for the removal of the sub-files, sub-directories, files, and directory found in the first purpose; (3) giving the user a GUI to accomplish said purposes in a convenient way that is easily accessible.

Securely processing Twilio requests from Tornado (Python)

2011-10-05T15:29:44-07:00

Python recipe 577893 by Jesse Davis (authentication, http, https, nginx, telephony, tornado, twilio).

Twilio is a telephony service that POSTs to a callback URL on your server and asks you what to do when it receives phone calls or SMSes to the numbers you rent from Twilio. But securing your communications with Twilio can be complex if you're using Tornado behind Nginx. This shows you how to protect your Twilio callback URL with HTTP Authentication, request-signing, and (optionally) SSL.

I'm using HTTP Authentication code from Kevin Kelley, and I wrote the rest myself.

Directory Pruner 2 (Python)

2011-04-05T01:37:14-07:00

Python recipe 577633 by Stephen Chappell (delete, directory, file, gui, utility).

This program builds on work done in recipe 577632, adds new options to the context menu, and experiments with threading the GUI. Directory Pruner 2 appear to work better on Windows 7 while Directory Pruner 1 (the non-threaded version) performs better on Windows XP. Those are the primary platforms on which the two programs have been tested. Please use these applications at your own risk.

If you do not like something about this recipe and want to vote it down, please let everyone what you find fault with, how you would improve it, and an example of the code you would improve.

Simple creation, configuration and installation of logging handlers (Python)

2011-06-01T19:05:40-07:00

Python recipe 577731 by Hank (logging).

A helper class and function to make it easy to configure logging, and an example of using it to send INFO to sys.stdout and WARNING or above to sys.stderr.

This is a simple port to Python 2 (I tested on 2.7) of Nick Coghlan's Python 3 recipe for doing the same thing.

Trace decorator for debugging (Python)

2011-01-24T18:40:51-08:00

Python recipe 577551 by Kevin L. Sitze (classes, debug, debugging, decorators, functions, inspection, modules, trace). Revision 2.

This package provides a decorator for tracing function and method calls in your applications. The tracing capabilities are managed through the logging package, and several mechanisms are provided for controlling the destination of the trace output.

It also provides functionality for adding decorators to existing classes or modules.

Directory Pruner 3 (Python)

2011-04-05T23:15:58-07:00

Python recipe 577635 by Stephen Chappell (delete, directory, file, gui, utility).

After considering the monolithic design of recipe 577633 and deciding that it should be divided into several smaller modules, Directory Pruner 3 was written to take advantage of better abstraction methodologies. There are a total of twelve files in the project, and all of them except the first are contained within a directory called "Directory Pruner 3" such that they make up a package in Python. Like the other Directory Pruners, please use and configure this program at your own computer's risk.

If you have any comments or wish to vote this recipe down, please provide an explanation as to what you find fault with in this program and also a solution that you would implement to fix the problem.

Simple creation, configuration and installation of logging handlers (Python)

2011-06-01T12:13:21-07:00

Python recipe 577496 by Nick Coghlan (logging).

Creation of log message handler for the logging module is often a multi-step process, involving creation of the handler object, configuration of the message levels and formats, installation of any filters and then actual connection of the handler to the relevant logger object.

This helper function allows all of these things to be specified up front in a single function call, which then takes care of configuring the handler object appropriately.

Logging asserts (Python)

2010-03-01T03:22:43-08:00

Python recipe 577074 by d.schlabing (logging). Revision 2.

This tries to marry assert statements with logging. If you use asserts together with logging and the script stops because an assertion is not fulfilled, you will not find any information about this in the logs. Unless you do something like this:

meal = ["eggs", "bacon", "spam"]
try:
    assert "spam" not in meal, "But I don't like spam!"
except AssertionError:
    log.exception("But I don't like spam!")
    raise

With the proposed recipe a somewhat similar behaviour can be achieved by:

log_assert("spam" not in meal, "But I don't like spam!")

LoggingWebMonitor - a central logging server and monitor. (Python)

2010-02-02T01:56:42-08:00

Python recipe 577025 by Gabriel Genellina (client_server, debugging, distributed, logging, remote, sysadmin, web). Revision 3.

LoggingWebMonitor listens for log records sent from other processes running in the same box or network. Collects and saves them concurrently in a log file. Shows a summary web page with the latest N records received.

Multiprocess-safe logging file-handler + interprocess RLock (Python)

2010-09-22T17:30:10-07:00

Python recipe 577395 by Jan Kaliszewski (concurrency, file, file_lock, flock, handler, logging, process, rlock, threading, threadsafe). Revision 11.

A Python 2.x/3.x-compatibile multiprocess-safe logging file-handler (logging.FileHandler replacement, designed for logging to a single file from multiple independent processes) together with a simple interprocess recursive lock -- universal abstract classes + Unix/Linux implementation.

Update: It's is a deeply revised version. Especially, now it --

is Python 2.4, 2.5, 2.6, 3.1 -compatibile (previously Py>=2.6 was needed); probably works also with 2.7, 3.0 and 3.2 (but not tested if it does);
is multiprocess-safe as well as thread-safe (proviously thread safety within a process was missed);
is based on public interfaces only (previously FileHandler._open() was called and overriden);
implement full RLock instance interface, as documented for threading.RLock (previously non-blocking mode and context-manager interface were missing).

The module contains:

Unix/Linux-only example implementation (with flock-based locking): FLockRLock and FLockFileHandler classes.
universal abstract classes -- which may be useful at developing implementation for non-Unix platforms: MultiprocessRLock, MultiprocessFileHandler, LockedFileHandler,

Also a quick-and-dirty test was added.

It is still an alpha version -- I'll be very grateful for any feedback.

Further updates:

2010-09-20: Some corrections, especially: non-blocking mode bug in MultiprocessRLock.acquire() fixed; _test() function improved; plus fixes in the description below.
2010-09-22: _test() improved and moved to description section. Mistaken copyright-notice removed.