Welcome, guest | Sign In | My Account | Store | Cart

look ahead one item during iteration (Python recipe) by Wai Yip Tung
ActiveState Code (http://code.activestate.com/recipes/409825/)

Iteration is a fundamental Python idiom. It is simple and effective.

for n in iterable: # do something with n

But there are also cases when you might want to look ahead one item during iteration. For example, with a sorted list, one can eliminate duplicated items by dropping those equals to the next item. This generator based recipe that enumerate an item and its next in a list. For example,

>>> for i,j in pairwise([1,2,3]): print i,j
...
1 2
2 3
3 None
Python, 9 lines
Download
1
2
3
4
5
6
7
8
9
def pairwise(lst):
    """ yield item i and item i+1 in lst. e.g.
        (lst[0], lst[1]), (lst[1], lst[2]), ..., (lst[-1], None)
    """
    if not lst: return
    #yield None, lst[0]
    for i in range(len(lst)-1):
        yield lst[i], lst[i+1]
    yield lst[-1], None

A use case of pairwise is to remove duplicated items in a sorted list. Below is a very compact example to do this using pairwise and list comprehension.

>>> [i for i,j in pairwise([1,1,1,2,3,3]) if j==None or i!=j]
[1, 2, 3]

Compare this with various examples in the 'Remove duplicates from a sequence' recipe. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560

Note that you can easily customize the initial and the final item iterated, that is whether you want (None, item0) as the initial item and (itemN, None) as the last item. I find it most useful to have only (itemN, None) included. In this case the number of steps iterated is the same as in ordinary list iteration.

Alternatively this could be expressed even more concisely using zip(lst, lst[1:]+[None])

Also the pairwise recipe in the itertools module (Python Library Reference 5.16.3) does something similar.

However I find the generator recipe both easier to understand as well as most straightforward to control the inital and final item iterated.

Tags: algorithms

4 comments

Raymond Hettinger 19 years ago  # | flag

Remove duplicates from sorted data with itertools.groupby.

>>> from itertools import groupby
>>> list(k for k,g in groupby([1,1,1,2,3,3]))
[1, 2, 3]
Maxim Khesin 19 years ago  # | flag

small nit. in usage I would do [i for i,j in pairwise([1,1,1,2,3,3]) if i!=j or j==None]

instead of

[i for i,j in pairwise([1,1,1,2,3,3]) if j==None or i!=j],

putting the common case first. It is both faster due to shortcut eval, and slightly clearer - I usually THINK of the common case first. Of course groupby looks even more powerful, thatnks Raymond for pointing it out.

Steven Bethard 19 years ago  # | flag

If you just want to look ahead, consider one of the 'peek' recipes:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/304373

If you really do want to view your items pairwise, I'd suggest a solution that works with any iterable instead of just sequences:

def pairwise(itr):
   first, second = itertools.tee(itr)
   second.next() # remove first element of second
   second = itertools.chain(second, [None]) # add final None
   return itertools.izip(first, second)
Giles Brown 15 years, 3 months ago  # | flag

The pairwise listed immediately above doesn't work if itr is initially empty.
You need to wrap second.next() in a try/except StopIteration.

Created by Wai Yip Tung on Thu, 14 Apr 2005 (PSF)
◄ Python recipes (4591) ►
◄ Wai Yip Tung's recipes (9) ►

Required Modules

  • (none specified)

Other Information and Tasks