Welcome, guest | Sign In | My Account | Store | Cart

Wrap a string in a file-like object that calls a user callback whenever read() is called on the stream (Python recipe) by Ben Hoyt
ActiveState Code (http://code.activestate.com/recipes/578669/)

Wraps a string in a read-only file-like object, but also calls a user callback with the number of bytes read whenever read() is called on the stream. Used for tracking upload progress, for example for a progress bar in a UI application.

Python, 21 lines
Download
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
import cStringIO as StringIO

class ReadCallbackStream(object):
    """Wraps a string in a read-only file-like object, but also calls
    callback(num_bytes_read) whenever read() is called on the stream. Used for
    tracking upload progress. Idea taken from this StackOverflow answer:
    http://stackoverflow.com/a/5928451/68707
    """
    def __init__(self, data, callback):
        self._len = len(data)
        self._io = StringIO.StringIO(data)
        self._callback = callback

    def __len__(self):
        return self._len

    def read(self, *args):
        chunk = self._io.read(*args)
        if len(chunk) > 0:
            self._callback(len(chunk))
        return chunk

An example using urllib2:

import json
import urllib2

def callback(num_bytes_read):
    print 'callback:', num_bytes_read, 'bytes read'

data = 'x' * 20000
stream = ReadCallbackStream(data, callback)
request = urllib2.Request('http://httpbin.org/post', stream)
f = urllib2.urlopen(request)
response = json.loads(f.read())
print 'httpbin.org said we POSTed', len(response['data']), 'bytes'

And the output:

callback: 8192 bytes read
callback: 8192 bytes read
callback: 3616 bytes read
httpbin.org said we POSTed 20000 bytes
Tags: callback, file, progress, upload