Welcome, guest | Sign In | My Account | Store | Cart

The 3 Python array packages are not able to create multidimensional arrays from iterators. This recipe allows you to do that.

Python, 22 lines
 ``` 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22``` ```# Public domain. import numpy def array_from_iter(it, typecode): try: ans = numpy.array([it.next()], typecode) except StopIteration: raise ValueError('iterator contains 0 items') shape0 = ans.shape[1:] for (i, x) in enumerate(it): ans.resize((i+2,)+shape0) ans[i+1] = x return ans def f(): yield [1,2,3] yield [4,5,6] print array_from_iter(f(), 'b') # Prints [[1,2,3],[4,5,6]] # Alternatively, the same code above works with numarray and Numeric. # Just replace 'numpy' with 'numarray' or 'Numeric'. ```

In some cases, you may have an iterator that returns a large number of lists, so that calling list() on your iterator will cause the Python interpreter to exceed the available physical memory. If your iterator of lists is "rectangular" (in the sense of a rectangular multidimensional array) and uses homogeneous types, then the array might fit in memory if constructed by Numeric.array, numarray.array, or numpy.array.

The three array packages do not support initialization from an iterator in general: for example, numarray.array(iter([[1,2,3],[4,5,6]])) fails. The above recipe will produce the correct result: array_from_iter(iter([[1,2,3],[4,5,6]]),'b') will produce the 2x3 array [[1,2,3],[4,5,6]].

Note that the iterator passed to array_from_iter() must have at least 1 element: if it has 0 elements, then ValueError is raised.

 Created by Connelly Barnes on Fri, 31 Mar 2006 (PSF)