Welcome, guest | Sign In | My Account | Store | Cart

Generic throttling for string/sequence functions (Python recipe) by Anon User
ActiveState Code (http://code.activestate.com/recipes/168638/)

This call wrapper class enables throttling of function calls. You can control how many characters (or sequence elements) per second your function processes (with granularity of one sequence). The calls are asynchronous and the actual function call is done when the last operation has lasted/waited long enough to satisfy the characters/second limit.

Python, 44 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
import time, threading, Queue

class throttle:
    def __init__(self, func, cps, daemon = 0, resolution = 0.1):
        self.func = func
        self.cps = cps
        self.res = resolution
        self.queue = Queue.Queue()
        self.thr = threading.Thread(target = self._loop)
        self.thr.setDaemon(daemon)
        self.thr.start()

    def destroy(self):
        if self.thr.isAlive():
            self.queue.put(self.queue) # kills the thread
            if not self.thr.isDaemon():
                self.thr.join()

    def __call__(self, data):
        self.queue.put(data)

    def _loop(self):
        while 1:
            data = self.queue.get()
            if data == self.queue:
                return # the queue instance serves as a trigger object
            t = time.time()
            self.func(data)
            while len(data) / (time.time() - t + 0.0001) > self.cps:
                time.sleep(self.res)


if __name__ == '__main__':
    import sys
    
    p = throttle(sys.stdout.write, 30)

    p("HEAD KNIGHT: We are now... no longer the Knights Who Say 'Ni'.\n")
    p("KNIGHTS OF NI: Ni! Shh!\n")
    p("HEAD KNIGHT: Shh! We are now the Knights Who Say "
      "'Ecky-ecky-ecky-ecky-pikang-zoop-boing-goodem-zu-owly-zhiv'.\n")
    p("RANDOM: Ni!\n")

    p.destroy()

I wrote this to control the output of a multithreading IRC bot (a news agent), which must not output too fast to the channel to avoid unwanted flooding.

The throttle can function in daemon or non-daemon (default) mode. The latter requires a call to destroy() to join the thread. While the daemonic version doesn't require this, it can't guarantee that all submitted items are processed if the program exits too soon.

The class could be made synchronous approximately by removing all the Queue and threading stuff, and moving the last four lines from _loop() to __call__().

There is not much error handling: the given function should handle all its exceptions (like non-sequence arguments).

Tags: threads