A small function that walks over pretty much any Python object and yields the objects contained within (if any) along with the path to reach them. I wrote it and am using it to validate a deserialized data-structure, but you can probably use it for many things.
Python, 59 lines
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | from collections import Mapping, Set, Sequence
# dual python 2/3 compatability, inspired by the "six" library
string_types = (str, unicode) if str is bytes else (str, bytes)
iteritems = lambda mapping: getattr(mapping, 'iteritems', mapping.items)()
def objwalk(obj, path=(), memo=None):
if memo is None:
memo = set()
iterator = None
if isinstance(obj, Mapping):
iterator = iteritems
elif isinstance(obj, (Sequence, Set)) and not isinstance(obj, string_types):
iterator = enumerate
if iterator:
if id(obj) not in memo:
memo.add(id(obj))
for path_component, value in iterator(obj):
for result in objwalk(value, path + (path_component,), memo):
yield result
memo.remove(id(obj))
else:
yield path, obj
# optional test code from here on
import unittest
class TestObjwalk(unittest.TestCase):
def assertObjwalk(self, object_to_walk, *expected_results):
return self.assertEqual(tuple(sorted(expected_results)), tuple(sorted(objwalk(object_to_walk))))
def test_empty_containers(self):
self.assertObjwalk({})
self.assertObjwalk([])
def test_single_objects(self):
for obj in (None, 42, True, "spam"):
self.assertObjwalk(obj, ((), obj))
def test_plain_containers(self):
self.assertObjwalk([1, True, "spam"], ((0,), 1), ((1,), True), ((2,), "spam"))
self.assertObjwalk({None: 'eggs', 'bacon': 'ham', 'spam': 1},
((None,), 'eggs'), (('spam',), 1), (('bacon',), 'ham'))
# sets are unordered, so we dont test the path, only that no object is missing
self.assertEqual(set(obj for path, obj in objwalk(set((1,2,3)))), set((1,2,3)))
def test_nested_containers(self):
self.assertObjwalk([1, [2, [3, 4]]],
((0,), 1), ((1,0), 2), ((1, 1, 0), 3), ((1, 1, 1), 4))
self.assertObjwalk({1: {2: {3: 'spam'}}},
((1,2,3), 'spam'))
def test_repeating_containers(self):
repeated = (1,2)
self.assertObjwalk([repeated, repeated],
((0, 0), 1), ((0, 1), 2), ((1, 0), 1), ((1, 1), 2))
def test_recursive_containers(self):
recursive = [1, 2]
recursive.append(recursive)
recursive.append(3)
self.assertObjwalk(recursive, ((0,), 1), ((1,), 2), ((3,), 3))
if __name__ == '__main__':
unittest.main()
|
Example use: In one configuration mechanism I implemented, there exists an UNCONFIGURED sentinel that marks configuration items that are required but aren't set. If a loaded configuration has unconfigured items in it, it's considered invalid. I used objwalk to recursively traverse the configuration object and possibly raise an exception telling where in the configuration were unconfigured elements found.
This is great I got lucky and found it with Google and it does what I needed - iterate over nasty object
Thanks for making it
I pretty much inspired some of my code from your code.
You are duely quoted as the source of the code (the code is python public licence or same).
It helps doing fun stuff, like watching for a dict in a dict (of dict ( of dict ... ) ) ) matching criterion. So thanks.
https://github.com/jul/ADictAdd_iction/blob/master/vector_dict/VectorDict.py#L156 (I know it is the old version, I will use the better one if I am allowed to).
Please contact me if I violated any IP stuff, and I would gladly conform to any of your demand even getting rid of it, if you consider this solution.
Nah, use the IP as you wish, that's the idea.
cool. Btw, I was told on #python to wait an endless time for yield from PEP implementation, and that this kind of code is bad usage and should not be promoted.
(I use your function to flatten nested generators).
I did not reply, but I had the idea now and could not wait, and it fitted my needs. :)
Practicality beats purity :)
Thanks.