Popular recipes by Josiah Carlson

Module to allow Asynchronous subprocess use on Windows and Posix platforms (Python)

2006-12-01T17:30:02-08:00

Python recipe 440554 by Josiah Carlson (sysadmin). Revision 10.

The 'subprocess' module in Python 2.4 has made creating and accessing subprocess streams in Python relatively convenient for all supported platforms, but what if you want to interact with the started subprocess? That is, what if you want to send a command, read the response, and send a new command based on that response?

Now there is a solution. The included subprocess.Popen subclass adds three new commonly used methods: recv(maxsize=None), recv_err(maxsize=None), and send(input), along with a utility method: send_recv(input='', maxsize=None).

recv() and recv_err() both read at most maxsize bytes from the started subprocess. send() sends strings to the started subprocess. send_recv() will send the provided input, and read up to maxsize bytes from both stdout and stderr.

If any of the pipes are closed, the attributes for those pipes will be set to None, and the methods will return None.

v. 1.3 fixed a few bugs relating to *nix support v. 1.4,5 fixed initialization on all platforms, a few bugs relating to Windows support, added two utility functions, and added an example of how to use this module. v. 1.6 fixed linux _recv() and test initialization thanks to Yuri Takhteyev at Stanford. v. 1.7 removed _setup() and __init__() and fixed subprocess unittests thanks to Antonio Valentino. Added 4th argument 'tr' to recv_some(), which is, approximately, the number of times it will attempt to recieve data. Added 5th argument 'stderr' to recv_some(), where when true, will recieve from stderr. Cleaned up some pipe closing. v. 1.8 Fixed missing self. parameter in non-windows _recv method thanks to comment. v. 1.9 Fixed fcntl calls for closed handles.

Simple web request benchmark (Python)

2006-10-31T00:27:13-08:00

Python recipe 442412 by Josiah Carlson (web). Revision 4.

Recently, there has been a discussion between myself and another individual as to the performance of particular web servers under certain situations. Being that web server testing frameworks have different ways of measuring how long a 'request' takes, I thought I would take the guesswork out of it and measure everything explicitly.

There are generally 5 portions to a web request: 1. create the connection 2. start sending the request 3. finish sending the request 4. start recieving the response 5. finish reading the response

This recipe measures the amount of time to perform each portion of a request to a single file many times, and prints the results in a somewhat reasonable fashion.

Asynchronous HTTP server (Python)

2006-02-06T19:16:33-08:00

Python recipe 440665 by Josiah Carlson (web). Revision 5.

A recipe version of the SimpleAsyncHTTPServer with a few additional options not previously available for basic Python-only web servers.

Emacs-like hotkey support for wxPython (Python)

2005-11-22T06:50:06-08:00

Python recipe 457407 by Josiah Carlson (programs). Revision 2.

Some people have a burning need for emacs-like multiple key sequence hotkeys. What do I mean? Ctrl+x Ctrl+s to save the currently open document and exit emacs.

Included as code is a sample wxPython program which implements multi-group keypress combinations, and will print 'hello!' in the statusbar if the combination Ctrl+Y Alt+3 Shift+B is entered. As you are entering a valid sequence of hotkeys, it will print your current combination in the status bar. If you make a mistake, it will print out your failed keyboard combination.

I use variants of the menu manupulation and keymap generation code in PyPE (http://pype.sf.net).

Yet another reinvention of a Python HTML generation mechanism (Python)

2005-11-22T07:25:44-08:00

Python recipe 440563 by Josiah Carlson (web). Revision 3.

The other day I was complaining about writing html, forms, etc., for Python cgi and/or web programming. I had pointed out a selection of three examples, the first of which ended up being very much like Nevow.stan . Thinking a bit about it, I realized that stan had issues in that you couldn't really re-use pre-defined tags with attributes via map, and keyword arguments were just too darn convenient to swap the calling and getitem syntax.

Instead, I hacked together a mechanism that supports: T.tagname("content", T.tagname(...), ..., attr1='value', ...) T.tagname(attr1='value', ...)("content", T.tagname(...), ...) x = T.tagname(attr1='value', ...) y = T.tagname(*map(x, ['content', ...])) ... and many other options.

Essentially, you can mix and match calls as much as you want, with three memory and sanity saving semantics: 1. Creating a new tag object via T.tagname, or any call of such, will create a shallow copy of the object you are accessing. 2. smallred = T.font(size='-1', color='red');bigred = smallred(size='+1') Works exactly the way you expect it to. If it doesn't work the way you expect it to, then your expectations are confused. 3. If you are adding content that sites within the tag, the content is replaced, not updated, like attributes.

This simple version handles auto-indentation of content as necessary (or desireable), auto-escaping of text elements, and includes an (I believe) nearly complete listing of entities which don't require closing tags.

I don't know where this is going, whether it can or will expand into something more, or what, but I believe what I have managed to hack together is better than other similar packages available elsewhere (including this recipe over here http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/366000 , which I discovered after writing my own). Funny how these things work out. Astute observers will note that I borrow nevow.stan's meme of using T.tagname for generating tag objects.

Automatically generate slots attribute for classes and assign attributes on instances (Python)

2005-07-02T20:03:32-07:00

Python recipe 435880 by Josiah Carlson (shortcuts). Revision 2.

There was recently a short discussion about some way to make certain initialization of instances easier (less typing). This was then followed by an expression of a desire for the automatic generation of the __slots__ attribute for classes (which is generally painful to do by hand for any nontrivial class). This recipe defines a metaclass and a function. The 'AutoSlots' metaclass automatically generates a __slots__ attribute during compilation, and the 'InitAttrs' function initializes instance variables during runtime.

A variant which only requires the metaclass would be convenient and would be graciously accepted.

Rabin-Miller probabilistic prime test (Python)

2005-04-25T02:49:23-07:00

Python recipe 410681 by Josiah Carlson (algorithms). Revision 3.

Included is a recipe for performing the Rabin-Miller probabilistic test for a composite witness. As provided by Paul Miller in the comments (not the Miller in Rabin-Miller), Rabin-Miller can only tell us if a value is definitely composite. In the case where a test value is not a witness for the compositeness of a potential prime, it can only lie with a probability of at most 1/4.

With this, we can attempt to catch a liar over some number of trials, and the probability of us not catching at least one liar after k trials (if the number is not actually prime) is at most 4**-k.

Included is an algorithm for generating a number of b bits for which no composite witness was found after k trials. Removing mathematical rigor will suggest that the probability of the value being prime after k trials is at least 1-1/4**k.

Union Find data structure (Python)

2005-07-02T18:33:13-07:00

Python recipe 215912 by Josiah Carlson (algorithms). Revision 2.

The union find data structure is primarily used for Kruskal's Minimum Spanning Tree algorithm, though can be used whenever one only needs to determine of two items are in the same set, and be able to combine sets quickly.

Length-limited O(1) LRU Cache implementation (Python)

2006-08-07T03:37:32-07:00

Python recipe 252524 by Josiah Carlson (algorithms). Revision 3.

Everyone knows about LRU. Here is an implementation that takes O(1) time to insert, update or delete.