Flattening an arbitrarily deep list (or any iterator) (Python)

2012-04-03T17:13:35-07:00

Python recipe 578092 by Garrett (flatten, iterator, itertools, list, python, tuple). Revision 6.

What if you had a list like this: [1, -10, [1,2,[3,4]], xrange(200)], and you just wanted to go through each element in order (wanted it to return a simple list of [1,-10,1,2,3,4,1,2,3,4...199])

I've seen a couple of attempts to flatten arbitrarily deep lists. Many of them involve recursion, like this one: http://rightfootin.blogspot.com/2006/09/more-on-python-flatten.html

Recursion is generally considered non-pythonic (at least to my knowledge), so I have used one which just involves simple iterators instead. Also, recursion will fail if the list is too deep (so it wouldn't really be arbitrary, would it?).

peek ahead an iterator (Python)

2010-08-17T13:41:41-07:00

Python recipe 577361 by Quintijn Hoogenboom (iter, iterator, peek, peeking).

a merge of recipes 16.7 and 19.18 of the python cookbook (2nd edition).

you can peek ahead more steps if needed, or just use the preview variable for the next item of the list.

PHP 'Struct' port (PHP)

2010-03-28T01:22:31-07:00

PHP recipe 577160 by Jeff Griffiths (iterator, iterators, oop). Revision 2.

In Ruby, the Struct class is a convenient way to create a hash-like object on the fly and use it for your nefarious purposes. PHP 5+ can be convinced to do this type of things as well, it just doesn't have it out of the box. Here is a simple class that implements iterator and allows you to populate the internal data structure similar to how Ruby's Struct works. Syntactic sugar? Probably.

Note: I haven't bothered to implement the Ruby Struct API per se, Instead I just got something similar by implementing the Iterator interface and keeping things very PHP-like.

Wrap any iterable context manager so it closes when consumed (Python)

2012-11-19T20:10:35-08:00

Python recipe 578342 by Andrew Barnert (contextmanager, context_manager, generator, iterable, iterator, with_statement).

There are a few types in Python—most notably, files—that are both iterators and context managers. For trivial cases, these features are easy to use together, but as soon as you need to use the iterator lazily or asynchronously, a with statement won't help. That's where this recipe comes in handy:

send_async(with_iter(open(path, 'r')))

This also allows you to "forward" closing for a wrapped iterator, so closing the outer iterator also closes the inner one:

sync_async(line.upper() for line in with_iter(open(path, 'r')))

Odometer iterator (Python)

2014-09-27T22:35:47-07:00

Python recipe 578944 by Tomas Nordin (iterator, odometer, radix). Revision 2.

A suggestion of an odometer implementation. Each "roll" in the odometer is defined by the caller. It is a list holding any objects, only the length of the list is important.

Can be good to have if something need to be tested in all possible combinations.

Partition an iterable into n lists (Python)

2009-05-30T16:35:27-07:00

Python recipe 576785 by Ian Eloff (iterator, partition).

This could also be easily modified to return n iterators, but was outside of my needs. Handy for splitting up the workload for use with multiple threads/processes.

Concurrent buffer for generators (Python)

2010-05-05T22:47:33-07:00

Python recipe 576999 by Javier Ruere (concurrency, generator, iterator, threading). Revision 2.

A buffer that will partially consume an iterator in the background.

Very useful for reading files and merging the data using the excellent http://code.activestate.com/recipes/491285/

Cycling a sequence (Python)

2014-09-20T19:20:46-07:00

Python recipe 578942 by Tomas Nordin (cyclic_iterator, generator, iterable, iterator, python).

A post-it function to cycle through some sequence. Better use itertools.cycle if for any iterable.

Windowing an iterable with itertools (Python)

2010-04-15T18:45:41-07:00

Python recipe 577196 by Daniel Cohn (generator, iterable, iterator, loop, peeking, window).

Oftentimes a programmer needs to peek into an iterator without advancing it, a task for which many good solutions already exist. But what if the intrepid coder needs a fast and pythonic way to 'window' the data? This recipe demonstrates how to wrap any iterable with a class that adds two methods, prev and peek.

An analogue of enumerate for nested lists. (Python)

2012-12-11T16:59:14-08:00

Python recipe 578376 by John Crichton (enumerate, index, iterator, list). Revision 6.

A generator function which enables traversal and modification of deeply nested lists. Together with the supplied helper functions it could be useful when working with data stored in deeply nested lists, particularly when the level of nesting precludes a recursive approach.

lazy ordered unique elements from an iterator (Python)

2011-06-23T16:36:38-07:00

Python recipe 577768 by Andrew Dalke (iterator, itertools, unique). Revision 2.

This implements a "unique" filter. Its input is an iterator of hashable items. It returns an iterator containing only the unique items of the input, in input order. That is, list(unique("cabbage")) produces ["c", "a", "b", "g"]. The implementation is lazy. The function supports the "key" parameter, which provides an alternate form of comparison.

(Note: a better version of this is available from the itertools documentation as unique_everseen )

iter_except -- a useful variant of builtin.iter() (Python)

2010-03-27T02:04:22-07:00

Python recipe 577155 by Raymond Hettinger (converter, iterator, wrapper).

Variant of iter(func, sentinel) that looks for an exception rather than for a sentinel value. Good for making iterators from of APIs that advance over a data and return an exception when they are done.

Fast flatten() with depth control and oversight over which subtrees to expand (Python)

2010-11-26T11:10:01-08:00

Python recipe 577470 by Kevin L. Sitze (flatten, iterator, iterators, list, optimal_solution, python, sequence, tuple).

Extremely fast, non-recursive, depth limited flatten with powerful control over which subtrees are to be expanded. If this is what you need then look no further.

Caching iterable wrapper (Python)

2009-11-06T11:38:43-08:00

Python recipe 576941 by Ulrik Sverdrup (cached, generator, iterator). Revision 8.

This is a very simple wrapper of an iterator or iterable, such that the iterator can be iterated streamingly without generating all elements or any at all, but the object can still be iterated from the beginning as many times as wanted. In effect, it is a streamingly loaded list.

Forward iterator (Python)

2015-01-29T17:12:10-08:00

Python recipe 578560 by Andrea Corbellini (iterator, wrapper). Revision 2.

A proxy for iterators that lets you read ahead items without consuming the iterator.

Most viewed recipes tagged "iterator"