LRU and LFU cache decorators (Python)

2010-08-01T01:19:23-07:00

Python recipe 498245 by Raymond Hettinger (shortcuts). Revision 6.

One-line decorator call adds caching to functions with hashable arguments and no keyword arguments. When the maximum size is reached, the least recently used entry or least frequently used entry is discarded -- appropriate for long-running processes which cannot allow caches to grow without bound. Includes built-in performance instrumentation.

Guard against an exception in the wrong place (Python)

2017-06-25T17:17:43-07:00

Python recipe 580808 by Steven D'Aprano (context, exception, guard, manager). Revision 2.

Sometimes exception handling can obscure bugs unless you guard against a particular exception occurring in a certain place. One example is that accidentally raising StopIteration inside a generator will halt the generator instead of displaying a traceback. That was solved by PEP 479, which automatically has such StopIteration exceptions change to RuntimeError. See the discussion below for further examples.

Here is a class which can be used as either a decorator or context manager for guarding against the given exceptions. It takes an exception (or a tuple of exceptions) as argument, and if the wrapped code raises that exception, it is re-raised as another exception type (by default RuntimeError).

For example:

try:
    with exception_guard(ZeroDivisionError):
        1/0  # raises ZeroDivisionError
except RuntimeError:
    print ('ZeroDivisionError replaced by RuntimeError')


@exception_guard(KeyError)
def demo():
    return {}['key']  # raises KeyError

try:
    demo()
except RuntimeError:
    print ('KeyError replaced by RuntimeError')

A memoize decorator for instance methods (Python)

2010-11-04T20:23:35-07:00

Python recipe 577452 by Daniel Miller (cache, functools, memoize, partial).

A simple result-caching decorator for instance methods. NOTE: does not work with plain old non-instance-method functions. The cache is stored on the instance to prevent memory leaks caused by long-term caching beyond the life of the instance (almost all other recipes I found suffer from this problem when used with instance methods).

Py2.6+ and Py3.0+ backport of Python 3.3's LRU Cache (Python)

2013-03-06T05:38:15-08:00

Python recipe 578078 by Raymond Hettinger (cache, lru). Revision 6.

Full-featured O(1) LRU cache backported from Python3.3. The full Py3.3 API is supported (thread safety, maxsize, keyword args, type checking, __wrapped__, and cache_info). Includes Py3.3 optimizations for better memory utilization, fewer dependencies, and fewer dict lookups.

Finding the percentile of the values (Python)

2011-04-25T03:41:08-07:00

Python recipe 511478 by Wai Yip Tung (algorithms). Revision 2.

This function find the percentile of a list of values. Note that the list must be sorted already.

Python reader + writer for spss sav files (Linux, Mac & Windows) (Python)

2013-02-20T22:07:27-08:00

Python recipe 577811 by Albert-Jan Roskam (reader, sav, spss, writer). Revision 12.

Python Program to READ & WRITE Spss system files (.sav) (Linux,Mac or Windows)

Check https://pypi.python.org/pypi/savReaderWriter/ for the latest version (including the libraries!)
Requires libspssdio.so.1 (LINUX) or libspssdio.dylib (MAC) or spssio32.dll (WINDOWS) plus associated libaries, which can be freely downloaded from: https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/We70df3195ec8_4f95_9773_42e448fa9029/page/Downloads%20for%20IBM%C2%AE%20SPSS%C2%AE%20Statistics?lang=en
It is recommended to download the v21 I/O files (required for writing zlib (.zsav) compressed files)
December 2012 (complete rewrite):
Added support for slicing, indexing, array slicing + other special methods
Added support for writing spss date fields
Added support for almost all meta data (missing values, sets, roles, etc.)
Added support for 64 bit Windows (tested with Win7) and other OSs (z/Linux, Solaris, HP Linux, IBM AIX (untested though)
Added support for reading and writing zlib compressed (.zsav) files
Removed pesky segfault error when freeing memory
Removed errors related to encoding
Changed some Reader defaults (verbose=False, returnHeader=False)
Renamed SavDataDictionaryReader into SavHeaderReader

LINUX:

Installation (tested on Linux Ubuntu 10):

additional packages/files needed are: intel-icc8-libs_8.0-1_i386.deb,libicu32_3.2-3_i386.deb, libstdc++5_3.3.6-20_i386.deb, libirc.so.
Run the following commands in your terminal: sudo apt-get install intel-icc8-libs; sudo apt-get install libicu32; sudo apt-get install libstdc++5.
Then convert libirc.a (static) to libirc.so (dynamic), save in same location as libspssdio.so.1: ar vx intel-icc8-libs_8.0-1_i386.deb; tar -xzvf data.tar.gz ./usr/lib/libirc.a; ar -x libirc.a.

Calling the program:

In the TERMINAL type: export LD_LIBRARY_PATH=/path/of/additional/sofiles; python /some/path/wrapperToProgram.py. You may also add ld_library_path to .bashrc
The wrapper starts with "from SavReaderWriter import *", followed by e.g. stuff from the if __name__ == '__main__' section

MAC OS:

you must put all the dylib files that come with the IBM SPSS_Statistics_InputOutput_Modules_* package in the macos directory somewhere that OS X can find them
one simple way to accomplish this is to copy them to /usr/lib

WINDOWS:

You can also find this dll in the installation directory of SPSS (although SPSS is _not_ needed!)
The .dll should be saved in the same location as this program.

USAGE: See docstrings + __main__ section

Simple threading decorator (Python)

2009-03-10T01:38:51-07:00

Python recipe 576684 by david.gaarenstroom (async, decorator, run_asynchronously, thread, threading). Revision 4.

When you're new at threading, using threads can be a bit daunting at first. If all you want is just to "run this function in parallel (= asynchronously) to the main program code", then this recipe can be of use. Simply use "@run_async" as a decorator for the function you want to run asynchronously. A call to that function will return immediately but the function itself will run in parallel.

Simple caching decorator (Python)

2010-12-01T00:43:01-08:00

Python recipe 577479 by Raymond Hettinger (cache, memoization).

Memoizing decorator. Has the same API as the functools.lru_cache() in Py3.2 but without the LRU feature, so it takes less memory, runs faster, and doesn't need locks to keep the dictionary in a consistent state.

Method signature type checking decorator for Python 3 (Python)

2015-05-15T09:25:08-07:00

Python recipe 572161 by Dmitry Dvoinikov (checking, decorator, interface, method, signature, sysadmin, type). Revision 8.

This recipe allows nice and clean validation for method parameters/return values. It uses function annotations available in Python 3 for the actual signature specification.

namedtuple.abc - abstract base class + mix-in for named tuples (Python)

2011-04-02T02:07:00-07:00

Python recipe 577629 by Jan Kaliszewski (abc, collections, dry, inheritance, namedtuple, shortcuts). Revision 7.

If you need

to define named tuple subclasses (including reusable abstract ones), adding/overriding some methods, in a convenient way;
to have the named tuple ABC (abstract base class) for isinstance/issubclass tests;
or simply would like to define your named tuple classes in a class-syntax-based and DRY way (without repeating type names...)

-- this recipe is for you.

Trace decorator for debugging (Python)

2011-01-24T18:40:51-08:00

Python recipe 577551 by Kevin L. Sitze (classes, debug, debugging, decorators, functions, inspection, modules, trace). Revision 2.

This package provides a decorator for tracing function and method calls in your applications. The tracing capabilities are managed through the logging package, and several mechanisms are provided for controlling the destination of the trace output.

It also provides functionality for adding decorators to existing classes or modules.

Longest common subsequence problem solver (Python)

2009-08-06T06:36:56-07:00

Python recipe 576869 by Shao-chuan Wang (algorithm, decorator, lcs, longest_common_subsequence). Revision 2.

Longest common subsequence problem is a good example of dynamic programming, and also has its significance in biological applications.

For more information about LCS, please see: http://en.wikipedia.org/wiki/Longest_common_subsequence_problem

Also, here, I use a 'cached' decorator to keep core algorithm neat. You can see how great the decorator could be. :)

Also note that, this recipe is just a demonstration of LCS and the usage of a python decorator. However, the memory is not used very efficiently. If the problem is very large-scaled, it may lead to stack overflow or memory error.

So, do not use this recipe to deal with large-scaled problems. ;)

Docstring inheritance decorator (Python)

2009-07-28T13:42:32-07:00

Python recipe 576862 by Shai Berger (decorator, descriptor, docstring, inheritance, metaprogramming).

In many cases, a subclass overrides a method in a parent class, just to change its implementation; in such cases, it would be nice to preserve the overridden method's docstring. The decorator below can be used to achieve this without explicit reference to the parent class. It does this by replacing the function with a descriptor, which accesses the parent class when the method is accessed as an attribute.

GAE User Session with HTTP Basic Authentication (Python)

2010-05-20T23:49:49-07:00

Python recipe 577235 by Berend (appengine, appspot, authentication, clients, gae, google, python, sessions, web, wsgi). Revision 6.

HTTP Basic is an unsecure but easy to implement authentication protocol. I think its good enough for a simple client in front of an SSL capable server. Google App-Engine supports SSL, and here is a recipe to set up the user-session using HTTP Basic.

gauth has the code from my not-really-a-recipe listing at: http://code.activestate.com/recipes/577217-routines-for-programmatically-authenticating-with-

Topological Sort (Python)

2012-09-27T12:21:23-07:00

Python recipe 578272 by Sam Denton (algorithm, sort, sorting, topological, toposort).

A topological sort (sometimes abbreviated topsort or toposort) or topological ordering of a directed graph is a linear ordering of its vertices such that, for every edge uv, u comes before v in the ordering. For instance, the vertices of the graph may represent tasks to be performed, and the edges may represent constraints that one task must be performed before another; in this application, a topological ordering is just a valid sequence for the tasks. A topological ordering is possible if and only if the graph has no directed cycles, that is, if it is a directed acyclic graph (DAG).

MineSweep (Python)

2010-09-16T14:45:36-07:00

Python recipe 577365 by Stephen Chappell (game, minesweep, python). Revision 2.

A friend was wondering how to implement MineSweep and decided to try making the game in Excel. Unfortunately, the game was not quite as interactive as he wanted; and he is unable to get around certain limitations at the moment since he is still new to programming. To show off the power of Python and demonstrate how easy it can be to develop a simple demonstration program in the language, the following program (and its iterations) were put together between the hours of 1 and 6 in the morning. It could certainly be much better in its implementation, but any serious work were to be put into MineSweep, starting over from scratch would be the best choice (especially with interface design).

Search sequences for sub-sequence (Python)

2011-08-19T05:17:00-07:00

Python recipe 577850 by Steven D'Aprano (find, searching, sequence, string, sublist, substring).

The list and tuple index() method and in operator test for element containment, unlike similar tests for strings, which checks for sub-strings:

>>> "12" in "0123"
True
>>> [1, 2] in [0, 1, 2, 3]
False

These two functions, search and rsearch, act like str.find() except they operate on any arbitrary sequence such as lists:

>>> search([1, "a", "b", 2, 3], ["b", 2])
2

Topological Sort (Python)

2010-09-28T19:22:27-07:00

Python recipe 577413 by Paddy McCarthy (algorithm, sorting, topological).

Given items that depend on other items, a topological sort arranges items in order that no one item precedes an item it depends on. In this example items are strings and dependencies are expressed in a dictionary whose keys are items and whose values are a set of dependent items. Note that the dict may contain self-dependencies (which are ignored), and dependent items that are not also dict keys, such as the item 'ieee'.

Synchronization Decorator for Class Methods (Python)

2010-03-13T10:11:07-08:00

Python recipe 577105 by Rodney Drenth (decorator, synchronize, threads).

Synchronizes access to methods of a class with either an instance or class specific lock.

Implementing the observer pattern yet again: this time with coroutines and the with statement (Python)

2006-11-10T16:38:01-08:00

Python recipe 498259 by Jim Baker (oop).

Implements the observer design pattern via generator coroutines, wrapped up to use the new 'with' statement of Python 2.5. Enables the loosely-coupled observation of any container implementing the dictionary protocol.

Most viewed recipes tagged "meta:requires=functools"

LRU and LFU cache decorators (Python)

Guard against an exception in the wrong place (Python)

A memoize decorator for instance methods (Python)

Py2.6+ and Py3.0+ backport of Python 3.3's LRU Cache (Python)

Finding the percentile of the values (Python)

Python reader + writer for spss sav files (Linux, Mac & Windows) (Python)

Simple threading decorator (Python)

Simple caching decorator (Python)

Method signature type checking decorator for Python 3 (Python)

namedtuple.abc - abstract base class + mix-in for named tuples (Python)

Trace decorator for debugging (Python)

Longest common subsequence problem solver (Python)

Docstring inheritance decorator (Python)

GAE User Session with HTTP Basic Authentication (Python)

Topological Sort (Python)

MineSweep (Python)

Search sequences for sub-sequence (Python)

Topological Sort (Python)

Synchronization Decorator for Class Methods (Python)

Implementing the observer pattern yet again: this time with coroutines and the with statement (Python)