Popular recipes by Vovan

Simple Web socket client implementation using Tornado framework. (Python)

2015-06-30T03:37:19-07:00

Python recipe 579076 by Vovan (client, tornado, web, websocket, websockets).

Simple Web socket client implementation using Tornado framework.

De-chunk and decompress HTTP body (Python)

2015-06-30T03:31:31-07:00

Python recipe 579075 by Vovan (chunked, content, dechunking, decompression, encoding, http, preprocess, transfer). Revision 2.

Example read_body_stream() usage:

with open(http_file_path, 'rb') as fh:
    print(b''.join(httputil.read_body_stream(
        fh, chunked=True, compression=httputil.GZIP))

Consumer Application Skeleton (Python)

2015-06-30T03:24:07-07:00

Python recipe 579074 by Vovan (application, consumer, daemon, framework). Revision 2.

Consumer Application Skeleton

This is very basic skeleton for data processing application implementing consumer pattern:

while is_running():
    task = get_next_task_from_queue()
    if task:
        submit_task_for_processing(task)
    else:
        sleep_for_a_moment()

Here's an example:

class ExampleApp(ConsumerAppBase):

    def _get_next_task(self):

        # Get next task from the queue.
        return self._queue.next()

    def _run_task(self, task):

        # This code's being executed in separate worker thread of
        # ThreadPoolExecutor
        return task / 2

    def _on_task_done(self, task, future):

        # Once worker thread finished - task results are available
        # in _on_task_done() callback as a concurrent.futures.Future object.
        self._log.info('Task done. Result: %s', future.result())