Most viewed recipes tagged "pipline"http://code.activestate.com/recipes/tags/pipline/views/2016-03-16T14:45:02-07:00ActiveState Code RecipesCollection Pipeline in Python (Python) 2016-03-16T14:45:02-07:00Steven D'Apranohttp://code.activestate.com/recipes/users/4172944/http://code.activestate.com/recipes/580625-collection-pipeline-in-python/ <p style="color: grey"> Python recipe 580625 by <a href="/recipes/users/4172944/">Steven D'Aprano</a> (<a href="/recipes/tags/bash/">bash</a>, <a href="/recipes/tags/filter/">filter</a>, <a href="/recipes/tags/map/">map</a>, <a href="/recipes/tags/pipe/">pipe</a>, <a href="/recipes/tags/pipline/">pipline</a>). </p> <p>A powerful functional programming technique is the use of pipelines of functions. If you have used shell scripting languages like <code>bash</code>, you will have used this technique. For instance, to count the number of files or directories, you might say: <code>ls | wc -l</code>. The output of the first command is piped into the input of the second, and the result returned.</p> <p>We can set up a similar pipeline using Lisp-like Map, Filter and Reduce special functions. Unlike the standard Python <code>map</code>, <code>filter</code> and <code>reduce</code>, these are designed to operate in a pipeline, using the same <code>|</code> syntax used by bash and other shell scripting languages:</p> <pre class="prettyprint"><code>&gt;&gt;&gt; data = range(1000) &gt;&gt;&gt; data | Filter(lambda n: 20 &lt; n &lt; 30) | Map(float) | List [21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0] </code></pre> <p>The standard Python functional tools is much less attractive, as you have to write the functions in the opposite order to how they are applied to the data. This makes it harder to follow the logic of the expression.</p> <pre class="prettyprint"><code>&gt;&gt;&gt; list(map(float, filter(lambda n: 20 &lt; n &lt; 30, data))) [21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0] </code></pre> <p>We can also end the pipeline with a call to <code>Reduce</code> to collate the sequence into a single value. Here we take a string, extract all the digits, convert to ints, and multiply:</p> <pre class="prettyprint"><code>&gt;&gt;&gt; from operator import mul &gt;&gt;&gt; "abcd12345xyz" | Filter(str.isdigit) | Map(int) | Reduce(mul) 120 </code></pre>