Popular recipes by Andrew Dalke

lazy ordered unique elements from an iterator (Python)

2011-06-23T16:36:38-07:00

Python recipe 577768 by Andrew Dalke (iterator, itertools, unique). Revision 2.

This implements a "unique" filter. Its input is an iterator of hashable items. It returns an iterator containing only the unique items of the input, in input order. That is, list(unique("cabbage")) produces ["c", "a", "b", "g"]. The implementation is lazy. The function supports the "key" parameter, which provides an alternate form of comparison.

(Note: a better version of this is available from the itertools documentation as unique_everseen )

resizable ctypes arrays (Python)

2006-03-28T23:03:58-08:00

Python recipe 475199 by Andrew Dalke .

a resizeable list-like object using ctypes arrays for the primary data storage

arithmetic coder (Python)

2004-10-02T17:54:59-07:00

Python recipe 306626 by Andrew Dalke (algorithms). Revision 2.

A simple but slow way of compression using arithmetic coding.

list permutation order indices (Python)

2004-10-06T16:31:23-07:00

Python recipe 306862 by Andrew Dalke (algorithms). Revision 2.

Given a list, find the indices used to get the elements from the list in sorted order

proleptic Gregorian dates and strftime before 1900 (Python)

2004-10-02T11:25:58-07:00

Python recipe 306860 by Andrew Dalke (algorithms).

A strftime implementation that supports proleptic Gregorian dates before 1900

another way to read lines backwards (Python)

2004-09-30T14:52:49-07:00

Python recipe 306569 by Andrew Dalke (files).

Another way to read lines from file backwards from the end to the beginning

test if a file or string is text or binary (Python)

2003-01-11T01:13:40-08:00

Python recipe 173220 by Andrew Dalke (files). Revision 2.

Here's a quick test to see if a file or string contains text or is binary. The difference between text and binary is ill-defined, so this duplicates the definition used by Perl's -T flag, which is: <br/> The first block or so of the file is examined for odd characters such as strange control codes or characters with the high bit set. If too many strange characters (>30%) are found, it's a -B file, otherwise it's a -T file. Also, any file containing null in the first block is considered a binary file.

modal wxPython with Twisted for XML-RPC (Python)

2003-06-04T08:22:07-07:00

Python recipe 203471 by Andrew Dalke . Revision 2.

This expands on Uwe C. Schroeder's recipe titled "Using wxPython with Twisted Python" to show one way to implement a modal progress bar using Twisted to make an XML-RPC call.

OrderedMultiDict and UnorderedMultiDict (Python)

2003-01-09T16:13:57-08:00

Python recipe 173072 by Andrew Dalke .

This implements two types of dictionary-like objects where there can be more than one entry with the same key. One is OrderedMultiDict, which preserves the order of all entries across all keys. The other is UnorderedMultidict, which only preserves the order of entries for the same key.

Download MultiDict.py Example:

>>> import MultiDict
>>> od = MultiDict.OrderedMultiDict()
>>> od["Name"] = "Andrew"; od["Color"] = "Green"
>>> od["Name"] = "Karen"; od["Color"] = "Brown"
>>> od["Name"]
'Karen'
>>> od.getall("Name")
['Andrew', 'Karen']
>>> for k, v in od.allitems():
...     print "%r == %r", (k, v)
...
'Name' == 'Andrew'
'Color' == 'Green'
'Name' == 'Karen'
'Color' == 'Brown'
>>> ud = MultDict.UnorderedMultiDict(od)
>>> for k, v in ud.allitems():
...     print "%r == %r", (k, v)
...
'Name' == 'Andrew'
'Name' == 'Karen'
'Color' == 'Green'
'Color' == 'Brown'
>>>

ReseekFile (Python)

2003-01-09T16:09:13-08:00

Python recipe 173071 by Andrew Dalke .

Wrap a file handle to allow seeks back to the beginning

Sometimes data coming from a socket or other input file handle isn't what it was supposed to be. For example, suppose you are reading from a buggy server which is supposed to return an XML stream but can also return an unformatted error message. (This often happens because the server doesn't handle incorrect input very well.)

A ReseekFile helps solve this problem. It is a wrapper to the original input stream but provides a buffer. Read requests to the ReseekFile get forwarded to the input stream, appended to a buffer, then returned to the caller. The buffer contains all the data read so far.

The ReseekFile can be told to reseek to the start position. The next read request will come from the buffer, until the buffer has been read, in which case it gets the data from the input stream. This newly read data is also appended to the buffer.

When buffering is no longer needed, use the 'nobuffer()' method. This tells the ReseekFile that once it has read from the buffer it should throw the buffer away. After nobuffer is called, the behaviour of 'seek' is no longer defined.

For example, suppose you have the server as above which either gives an error message is of the form:

ERROR: cannot do that

or an XML data stream, starting with "

infile = urllib2.urlopen("http://somewhere/") infile = ReseekFile.ReseekFile(infile) s = infile.readline() if s.startswith("ERROR:"): raise Exception(s[:-1]) infile.seek(0) infile.nobuffer() # Don't buffer the data ... process the XML from infile ...

This module also implements 'prepare_input_source(source)' modeled on xml.sax.saxutils.prepare_input_source. This opens a URL and if the input stream is not already seekable, wraps it in a ReseekFile.