Welcome, guest | Sign In | My Account | Store | Cart

Dictionary, list, set, and histogram types for relational algebra, functional programming, list-oriented programming, and modelling data flow.

Python, 630 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
# ftypes.py - Functional dictionary and list types for Python 2.1+
#
# Author:  Dave Benjamin <ramen@ramenfest.com>
# Version: 1.1.2

"""
Introduction
------------

The purpose of this module is to provide a dictionary and list type that can
aid in relational algebra, functional programming, list-oriented programming,
and perhaps even code obfuscation.

There is a certain dark side to this library: operator overloading. Almost
all of the useful features of these types are invoked through operators. My
rationale is that certain functional/relational methods have become so well-
known that they deserve to be liberated from the highly syntactic domain of
functions to the free and flowy rivers of infix notation. You may disagree.

Part of my inspiration for these ideas came from the late computer scientist
and mathematician Edsger Wybe Dijkstra (11 May 1930 -- 6 August 2002), who
argued that programs should be based on mathematical logic and methodology.
Throughout the process of learning about functional programming techniques,
I began to see the resemblence between pure mathematics and functional
algorithms, and wanted to follow this train of thought further.

The "map" function, for example, is so common and useful (to me, anyway)
that there ought to be a symbolic notation for it. Instead of always having
to write "map(func, list)", perhaps it should be "list MAP func", where "MAP"
could be substituted with the "M"-looking Greek letter of your choice. =)
In fear of accidently reinventing APL, I tried to ignore the temptation to
create such an operator in Python, but it seemed so logical and natural
after awhile that I really wanted to try it in practice.

As you will see, I have indeed implemented map as an operator (*), as well
as filter (/), reduce (()), sort (%), zip (**), and many other common
FP-related functions. As is usually the case with liberal operator
overloading, the choice in operator symbols is somewhat arbitrary. I am
reasonably happy with my choices so far, but I did not reach this without
sometimes painful self-deliberation.

Another factor that contributed to this code was a need for powerful tools
to deal with result sets from database queries. My "discovery" that result
sets could be mapped to lists of dictionaries was somewhat of an epiphany,
and enabled me to find very concise solutions to common database and web
templating problems. Not only can dictionaries represent database rows, but
they also function as namespaces for template evaluation.

Defining a result set (or, more formally, a "relation") as a list of
dictionaries allowed me to apply these types to the domain of relational
algebra: operators such as select (which is essentially the same operation
as filter), project (also implemented using the / operator), distinct
(implemented using the unary -), and union (+, merely list concatenation)
fit nicely into this list/dictionary model. The list constructor is
extended to allow for concise specification of literal result sets as
follows:

list(['id', 'fname', 'lname'     ],
     #---------------------------#
     [1234, 'Guido', 'van Rossum'],
     [1235, 'Alex',  'Martelli'  ],
     [1236, 'Tim',   'Peters'    ])

This constructor will return a list of three dictionaries, each containing
the keys "id", "fname", and "lname", pointing to the respective values for
each of the rows above. Since most database APIs can provide results in this
form, literal and actual result sets can be swapped freely. As a result,
you can test database code without the database, even through the interpreter
if you desire. This has been very useful.

The examples below should demonstrate the usage of the aforementioned
features, plus many others. You may wish to consult the code itself for
more ideas about how to use these types.

How-To
------

Import these types:
    from ftypes import *

Import these types without overwriting the original ones:
    from ftypes import list as flist, dict as fdict

Instantiate these types:
    dict()                                    -> {}
    dict({'a': 5, 'b': 6})                    -> {'b': 6, 'a': 5}
    dict(((1, 2), (3, 4)))                    -> {3: 4, 1: 2}
    dict(('a', 'b', 'c'), (1, 2, 3))          -> {'b': 2, 'c': 3, 'a': 1}
    list()                                    -> []
    list([1, 2, 3])                           -> [1, 2, 3]
 ++ list(['st', 'state'        ],
         ['AZ', 'Arizona'      ],
         ['CA', 'California'   ],
         ['PZ', 'Planet Zektar']) -> [{'st': 'AZ', 'state': 'Arizona'},
                                      {'st': 'CA', 'state': 'California'},
                                      {'st': 'PZ', 'state': 'Planet Zektar'}]
Do functional things:
    list([1, 3, 5, 7]) * (lambda x: x + 1)    -> [2, 4, 6, 8] (map)
    list([2, 3, 4, 5]) / (lambda x: x % 2)    -> [3, 5] (filter)
    list(range(5)).reduce(operator.add)       -> 10 (reduce)
    list('abcde') % (lambda x, y: cmp(y, x))  -> ['e','d','c','b','a'] (sort)
    list([0, '0', [], '[]']) / operator.truth -> ['0', '[]'] (any)
    list([1, 2, 3]) ** [4, 5, 6]              -> [[1, 4], [2, 5], [3, 6]] (zip)
    
    The map (*) and filter (/) operators are also available for the dict type.
    The given function will be applied to the dictionary's values.

Do relational things:
    states.st                                 -> ['AZ', 'CA', 'PZ'] (column)
    (states / (lambda x: x.st != 'CA')).st    -> ['AZ', 'PZ'] (select)
    (states / ['st'])                         -> [{'st': 'AZ'},
                                                  {'st': 'CA'},
                                                  {'st': 'PZ'}] (project)
    -list([1, 2, 2, 3, 6, 3, 2])              -> [1, 2, 3, 6] (distinct)
                                                  
    Note: The definition of states can be found above as indicated (++).
 
Other (maybe) useful tricks:    
    list([1, 2, 3]) / {1: 1, 3: 1}.has_key    -> [1, 3] (dict set filter)
    dict({'a': 5, 'b': 6}).a                  -> 5 (object-style dict lookup)
    dict({'a': 5, 'b': 6}.items())            -> {'b': 6, 'a': 5} (identity)
    dict({'a': 5, 'b': 6}).items() * list     -> [['b', 6], ['a', 5]] (cast)
    ~list([(1, 2), (3, 4)])                   -> [[1, 3], [2, 4]] (transpose)
    ~dict({1: 2, 3: 4})                       -> {2: 1, 4: 3} (dict transpose)
    dict().set('a', 5).set('b', 6).unset('a') -> {'b': 6} (mutator chaining)
    d = dict(); (5 + d.put('a', 6 + 7)) * d.a -> 234 (memoization)
    list(range(5)) * list(range(4)).get       -> [0, 1, 2, 3, None] (list get)
    list(['hello', 'world']).join(' ')        -> 'hello world' (string join)
    dict({'a': 5, 'b': 6}).eval('a + b')      -> 11 (eval within a namespace)

Callables:
    Dictionaries and lists can be made callable, ie. they can be invoked
    like functions. This behavior can be activated by supplying the named
    parameter "__call__" to the constructors. For example:
    
    list([1,2,3], __call__=list.reduce)(operator.add) -> 6
    
    I believe that this fits the definition of a "closure".
    
    The ability to add methods via keyword arguments is not restricted to
    __call__, by the way. You can in fact supply any method you would like
    to override as a keyword argument to the dict and list constructors.

Sets and Histograms:
    As a matter of convience, set and constructor functions have been
    provided, both returning dictionaries.
    
    Use set(1, 2, 3) as an alternative to dict({1: 1, 2: 1, 3: 1}).
    To convert a list "liszt" to a set, write set(*liszt). The binary
    operators &, |, and - have been overridden to function as set
    intersection, union, and difference, respectively. The "in"
    operator is an alias for has_key, as with more recent versions of
    Python's built-in dictionary, so it can be used to test for set
    containment.
    
    The histogram function counts the number of occurrences (frequency) of
    each element in a list. It takes a single list as its argument (unlike
    set(), which accepts a variable number of arguments) and returns a
    dictionary where the list elements are the keys and their values are
    their respective frequency counts.
    
    Both of these constructors deal only with hashable types. They will
    pass on any named parameters to dict().
    
Afterword
---------

Thanks to Guido for such a powerful and flexible language! I welcome any
ideas, contributions, and criticism from the community. Thanks also to
Alex Martelli for the fantastic "curry" implementation on ActiveState's
Python Cookbook, and to Tim Peters for starting the helpful discussion on
extracting unique elements from a list.

Peace!
Dave Benjamin <ramen@ramenfest.com>
"""

from __future__ import nested_scopes
from UserDict   import UserDict
from UserList   import UserList
from pprint     import pformat

__all__ = ['dict', 'list', 'odict', 'oset', 'set']

# List Class
# ----------

class list(UserList):
    def __init__(self, *args, **kwds):
        # Import keyword arguments into the object dictionary.
        # Callables are automatically curried so that they take
        # "self" as the first argument".
        for key, val in kwds.items():
            if callable(val):
                self.__dict__[key] = curry(val, self)
            else:
                self.__dict__[key] = val

        if len(args) == 0:
            # No arguments: empty list.
            UserList.__init__(self)

        elif len(args) == 1:
            # One argument: list.
            UserList.__init__(self, args[0])

        else:
            # Two arguments: list of dictionaries.
            UserList.__init__(self, [dict(args[0], row) for row in args[1:]])

    def copy(self):
        """Copy constructor."""
        return self[:]
    
    def column(self, key):
        """Get column."""
        return list([item[key] for item in self])

    def flip(self):
        """Convert list of dictionaries to dictionary of lists."""
        result = dict()
        if not self: return result
        for key in self[0].keys():
            result[key] = self.column(key)
        return result

    def get(self, idx, default=None):
        """Get item."""
        try:
            return self.data[idx]
        except IndexError:
            return default

    def join(self, sep=''):
        """String join with reversed semantics."""
        return sep.join(self)

    def reduce(self, func, *initial):
        """Reduce to a single value by iteratively applying a function."""
        if initial: return reduce(func, self, initial[0])
        return reduce(func, self)

    def __mul__(self, func_or_n):
        """Map/repeat (*)."""
        
        if callable(func_or_n):
            # Function: map operation.
            return list([func_or_n(x) for x in self])
            
        else:
            # Number: repeat operation.
            return list(self.data * func_or_n)

    def __div__(self, func_or_keys):
        """Filter/select/project (/)."""
        
        if callable(func_or_keys):
            # Function: select (filter) operation.
            return list([x for x in self if func_or_keys(x)])
        
        else:
            # Key list: project operation.
            return list([dict(x) / func_or_keys for x in self])

    def __mod__(self, func):
        """Sort (%)."""
        result = self[:]
        result.sort(func)
        return result

    def __pow__(self, other):
        """Zip (**)."""
        return list(zip(self, other)) * list

    def __invert__(self):
        """Transpose (~)."""
        if not self: return list()
        return list(zip(*self)) * list

    def __neg__(self):
        """Distinct (unary -)."""
        result = list()
        
        try:
            # Hash method (faster).
            seen = dict()
            for item in self:
                if item not in seen:
                    seen[item] = 1
                    result.append(item)
            
        except TypeError:
            # Linear method (more compatible).
            for item in self:
                if item not in result:
                    result.append(item)
                    
        return result

    def __getattr__(self, key):
        """Get column or attribute (.)."""

        if key == '__methods__':
            return UserList.__dict__.keys()

        if key == '__members__':
            if self.data and hasattr(self.data[0], 'keys'):
                return self.data[0].keys()
            else:
                return []

        if self.__dict__.has_key(key):
            return self.__dict__[key]

        if self.data:
            head = self.data[0]
            if hasattr(head, 'has_key') and head.has_key(key):
                return self.column(key)
            #if hasattr(head, key):
            if hasattr(head, '__dict__') and head.__dict__.has_key(key):
                return list([getattr(x, key) for x in self])
        
        raise AttributeError, key
    
    def __str__(self):
        """Built-in pretty-printer."""
        return pformat(self.data)

# Dictionary Class
# ----------------

class dict(UserDict):
    def __init__(self, *args, **kwds):
        # Import keyword arguments into the object dictionary.
        # Callables are automatically curried so that they take
        # "self" as the first argument".
        for key, val in kwds.items():
            if callable(val):
                self.__dict__[key] = curry(val, self)
            else:
                self.__dict__[key] = val
            
        if len(args) == 0:
            # No arguments: empty dictionary.
            UserDict.__init__(self)
        
        elif len(args) == 1:
            # One argument: dictionary or item list.
            
            if hasattr(args[0], 'items'):
                # Dictionary.
                UserDict.__init__(self, args[0])
                
            else:
                # Item list.
                UserDict.__init__(self)
                for key, val in args[0]:
                    self[key] = val
        else:
            # Two arguments: key and value lists.
            UserDict.__init__(self)
            for key, val in zip(args[0], args[1]):
                self[key] = val

    def copy(self):
        """Copy constructor."""
        return dict(self.data)
    
    def keys(self):
        """Returns keys as overloaded list."""
        return list(self.data.keys())
    
    def values(self):
        """Returns values as overloaded list."""
        return list(self.data.values())
    
    def items(self):
        """Returns items as overloaded lists of tuples."""
        return list(self.data.items())
    
    def eval(self, expr, vars={}):
        """Evaluate an expression using self as the namespace (())."""
        return eval(expr, self.data, vars)
        
    def flip(self):
        """Convert dictionary of lists to list of dictionaries."""
        return list(self.keys(), *~self.values())
    
    def set(self, key, val=1):
        """Assignment as method. Returns self."""
        self[key] = val
        return self

    def unset(self, key):
        """Deletion as method. Returns self."""
        del self[key]
        return self
    
    def put(self, key, val):
        """Assignment as method. Returns the assigned value."""
        self[key] = val
        return val
    
    def __and__(self, other):
        """Intersection (&)."""
        result = dict()
        for key in self.keys():
            if other.has_key(key):
                result[key] = self[key]
        return result

    def __or__(self, other):
        """Union (|)."""
        result = dict(self.data)
        result.update(other)
        return result

    def __add__(self, other):
        """
        Merge (+).
        
        The merge operation is similar to a union except that data is
        never overwritten. If three dictionaries with the same set of
        keys are merged, the resulting dictionary's values will be
        three-element lists.
        
        If you want destructive behavior, use the union (|) operator
        instead, since it pays no consideration to duplicate keys.
        """
        result = dict(self.data)
        
        for key in other.keys():
            if result.has_key(key):
                if hasattr(result[key], 'append'):
                    result[key].append(other[key])
                else:
                    result[key] = list([result[key], other[key]])
            else:
                result[key] = other[key]
                
        return result
    
    def __sub__(self, other):
        """Difference (-)."""
        
        result = dict()
        
        for key in self.keys():
            if not other.has_key(key):
                result[key] = self[key]
                
        return result
    
    def __mul__(self, func_or_n):
        """Map/repeat (*)."""
        
        result = dict()
        
        if callable(func_or_n):
            for key in self.keys():
                result[key] = func_or_n(key, self[key])
        else:
            for key in self.keys():
                result[key] = list([self[key]]) * func_or_n
                
        return result
    
    def __div__(self, func_or_keys):
        """Filter/extract (/)."""
        
        result = dict()
        
        if callable(func_or_keys):
            for key in self.keys():
                if func_or_keys(key, self[key]):
                    result[key] = self[key]
        else:
            for key in func_or_keys:
                result[key] = self[key]
                
        return result

    def __pow__(self, other):
        """Compose (**)."""
        result = dict()
        for key in self.keys():
            result[key] = other[self[key]]
        return result

    def __invert__(self):
        """Transpose (~)."""
        result = dict()
        for key in self.keys():
            result[self[key]] = key
        return result

    def __contains__(self, other):
        """Contains key (in)."""
        return self.has_key(other)
    
    def __getattr__(self, key):
        """Get field or attribute (.)."""

        if key == '__methods__':
            return UserDict.__dict__.keys()

        if key == '__members__':
            return self.keys()
        
        if self.__dict__.has_key(key) or self.data.has_key(key):
            return self[key]

        raise AttributeError, key
                                
    def __str__(self):
        """Built-in pretty-printer."""
        return pformat(self.data)

# Ordered Dictionary Class
# ------------------------

class odict(dict):
    def __init__(self, *args, **kwds):
        self._keys = {}
        dict.__init__(self, *args, **kwds)

    def __delitem__(self, key):
        dict.__delitem__(self, key)
        del self._keys[key]

    def __setitem__(self, key, item):
        dict.__setitem__(self, key, item)
        if not self._keys.has_key(key):
            self._keys[key] = max([0] + self._keys.values()) + 1

    def clear(self):
        dict.clear(self)
        self._keys = {}

    def copy(self):
        result = odict(self)
        result._keys = self._keys.copy()
        return result

    def keys(self):
        result = [(y, x) for x, y in self._keys.items()]
        result.sort()
        return list([x[1] for x in result])

    def values(self):
        return list(map(self.get, self.keys()))

    def items(self):
        return list(zip(self.keys(), self.values()))

    def popitem(self):
        try:
            keys = [(y, x) for x, y in self._keys.items()]
            keys.sort()
            keys.reverse()
            key = keys[0][1]
            
        except IndexError:
            raise KeyError('dictionary is empty')

        val = self[key]
        del self[key]

        return (key, val)

    def setdefault(self, key, failobj=None):
        dict.setdefault(self, key, failobj)
        if not self._keys.has_key(key):
            self._keys[key] = max([0] + self._keys.values()) + 1

    def update(self, other):
        dict.update(self, other)
        for key in other.keys():
            if not self._keys.has_key(key):
                self._keys[key] = max([0] + self._keys.values()) + 1

# Custom Dictionary Constructors
# ------------------------------

# Dictionary set constructor. Elements must be hashable.
# Example: set('a', 'b', 'c') -> {'a': 1, 'b': 1, 'c': 1}
set = lambda *x, **y: dict(x, [1] * len(x), **y)

# Ordered dictionary set constructor. Elements must be hashable.
oset = lambda *x, **y: odict(x, [1] * len(x), **y)

# Dictionary histogram constructor. Elements must be hashable.
# Example: histo(['a', 'b', 'b', 'a', 'b', 'c']) -> {'a': 2, 'b': 3, 'c': 1}
histo  = lambda x, **y: list(x).reduce(_histo, dict(**y))
_histo = lambda x,   y: x.set(y, x.get(y, 0) + 1)

# Comparators
# -----------

# Case-insensitive, reversed, and reversed case-insensitive comparators.
cmpi    = lambda x, y: cmp(x.lower(), y.lower())
revcmp  = lambda x, y: cmp(y, x)
revcmpi = lambda x, y: cmp(y.lower(), x.lower())

def reorder(key, order):
    """
    Returns a comparator that reorders a row set (list of dictionaries).
    The order is specified as a key (column) and a list of ordered values.
    """
    return lambda x, y, k=key, o=dict(order, range(len(order))): \
                  cmp(o.get(x[k]), o.get(y[k]))

# Helper Functions
# ----------------

def curry(*args, **create_time_kwds):
    """
    Bind arguments to a function.
    
    Author: Alex Martelli
    Source: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52549
    """    
    func = args[0]
    create_time_args = args[1:]
    def curried_function(*call_time_args, **call_time_kwds):
        args = create_time_args + call_time_args
        kwds = create_time_kwds.copy()
        kwds.update(call_time_kwds)
        return func(*args, **kwds)
    return curried_function

WHAT'S NEW

I have added support for the set and histo constructors for dictionary-based sets and histograms. Also, lists of dictionaries can be transformed into dictionaries of lists and vice versa using the list.flip/dict.flip methods.

The dict.set method's value argument now defaults to 1. This way, dicts can be treated like sets, hiding the actual values away as an implementation detail. Since the set() constructor also uses 1 for its values, you can be assured that dict.get, dict.has_key, and dict.__contains__ all behave the same in a boolean context.

I added an ordered dictionary class and an ordered set constructor. These should perform identically to their unordered counterparts with the additional property that key and value order remains intact. keys(), values(), items(), and popitem() are all implemented to retain the original order. Iterators are not yet supported.

Finally, the map (*) and filter (/) operators work on dictionaries now. They expect a function that accepts two arguments, a key and a value.

USING THESE TYPES IN ZOPE

If you would like to use these types within Zope and take advantage of the object-style lookup and other attribute-based features, you need to make the following modification to lib/python/AccessControl/ZopeGuards.py near line 94.

After: <pre> if Containers(type(object)) and Containers(type(v)): # Simple type. Short circuit. return v </pre> Add: <pre> from UserList import UserList from UserDict import UserDict if isinstance(object, UserList) or isinstance(object, UserDict): # Extended dictionary or list type. Short circuit. return v </pre> Copy ftypes.py to lib/python or another location in Zope's import path and create a file in the Extensions directory with the following contents:

<pre> import ftypes

ftypes.dict.__allow_access_to_unprotected_subobjects__ = 1 ftypes.list.__allow_access_to_unprotected_subobjects__ = 1

dict = lambda x, *y: ftypes.dict(x, *y) list = lambda x, *y: ftypes.list(x, *y) </pre> Now, create External Methods for dict and list, and you're set (no pun intended). Enjoy.

MORE ABOUT DIJKSTRA

Although I mention in the comments that this project has been inspired by Dijkstra, this is not to say that he would have been perfectly happy with my system of notation. He was a strong proponent of symbolic notation, but also mentioned that infix operators should be associative and symmetric. Neither of these properties I attempt to maintain, although I did try to assign similar functionality to operators of equal precedence. For an interesting read, take a look at EWD1300, "The notational conventions I adopted, and why": http://www.cs.utexas.edu/users/EWD/ewd13xx/EWD1300.PDF

On a more general level, however, I feel that this is a worthy response to his more general arguments that a) software should resemble mathematics, and b) mathematics demand concise symbolic notation. For instance, the use of the functional operators in modelling data flow (a la pipes-and-filters) allows you to provide a straightforward, self-documenting representation of the data path:

liszt * expand_items / remove_junk * simplify % case_insensitive_sort

Here, expand_items and simplify are map functions, remove_junk is a filter, and case_insensitive_sort is a comparator (like cmp). Once the behaviour of these operators is understood, the expression reads like a more specialized version of the UNIX command line, and the problem decomposes into well-defined units of functionality.

I'd like to do more experimentation along the lines of data flow modelling, such as finding a concise way to represent splits and joins in the data path without the use of temporary variables. I'd appreciate any suggestions.

1 comment

Dave Benjamin (author) 21 years, 5 months ago  # | flag

Download the source. The above source file is available for download here:

http://ramenfest.com/ftypes.py