Welcome, guest | Sign In | My Account | Store | Cart

Retry loop (Python recipe) by Ryan Nowakowski
Forked from Recipe 578163 (fixing bug on line 16)
ActiveState Code (http://code.activestate.com/recipes/578527/)

4

Encapsulates the logic of a retry loop using a generator function.

Python, 63 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# Simple version:

import sys, time

class RetryError(Exception):
    pass

def retryloop(attempts, timeout):
    starttime = time.time()
    success = set()
    for i in range(attempts): 
        success.add(True)
        yield success.clear
        if success:
            return
        if starttime + timeout < time.time():
            break
    raise RetryError

""" 
Usage:

for retry in retryloop(10, timeout=30):
    try:
        something
    except SomeException:
        retry()

for retry in retryloop(10, timeout=30):
    something
    if somecondition:
        retry()

"""

# Fancy version:

def fancyretryloop(attempts, timeout=None, delay=0, backoff=1):
    starttime = time.time()
    success = set()
    for i in range(attempts): 
        success.add(True)
        yield success.clear
        if success:
            return
        duration = time.time() - starttime
        if timeout is not None and duration > timeout:
            break
        if delay:
            time.sleep(delay)
            delay = delay * backoff

    e = sys.exc_info()[1]

    # No pending exception? Make one
    if e is None:
        try: raise RetryError
        except RetryError as e: pass

    # Decorate exception with retry information:
    e.args = e.args + ("on attempt {0} of {1} after {2:.3f} seconds".format(i, attempts + 1, duration),)

    raise

The retry loop is common in any code that deals with resources that might fail intermittently such as calls to remote servers. This generator function takes away the noise of retry loop implementation and encapsulates it into a structure that almost looks like a language feature and should be easy to understand for a reader even without looking at the implementation of the retryloop() generator.

The fancy version adds delay and exponential backoff. It also preserves the original exception which terminated the retry loop, decorating it with useful information about the retry loop.

Caveat: Calling retry() does not terminate the current attempt. You should use "retry() ; continue" if you have multiple try/catch statements in order to prevent it from trying the next step of an attempt even if one step has failed.

Tags: exception, generator, loop, retry, timeout

Add a comment

Sign in to comment