| Store | Cart

Re: [Tutor] When to use multiprocessing Managers?

From: eryksun <eryk...@gmail.com>
Sat, 1 Mar 2014 11:48:25 -0500
On Fri, Feb 28, 2014 at 6:31 AM, James Chapman <jam...@uplinkzero.com> wrote:
>> log_Q = multiprocessing.Queue()

This is a Queue from multiprocessing.queues. It uses system resources
(e.g. a semaphore for the queue capacity) that can be shared with
processes on the same machine.

A value `put` in a queue.Queue is available immediately:

    >>> import queue>>> q1 = queue.Queue()>>> try: q1.put('value'); q1.get_nowait()
    ... except queue.Empty: 'empty'
    ...
    'value'

On the other hand, a Queue from multiprocessing.queues writes to a
pipe using a background thread, so there can be a small delay:

    >>> import multiprocessing as mp>>> q2 = mp.Queue()>>> try: q2.put('value'); q2.get_nowait()
    ... except queue.Empty: 'empty'
    ...
    'empty'
    >>> q2.get_nowait()
    'value'

> or whether I create it like this:>> multimanager = multiprocessing.Manager()> log_Q = multimanager.Queue()

This is a queue.Queue wrapped by an AutoProxy. For example, its `get`
method calls _callmethod('get', *args, **kwds), which connects to the
manager, sends the request, and receives the result.

The docs demonstrate using a manager with remote processes:

http://docs.python.org/3/library/multiprocessing#using-a-remote-manager

You can also proxy the Queue type from multiprocessing.queues. In that
case, remote processes use a proxy, but local processes can use the
queue directly.

> Perhaps the manager would be important if I was writing to a Queue and> expecting all threads to see that message?

Only 1 thread will `get` the message.
_______________________________________________
Tutor maillist  -  Tut...@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Recent Messages in this Thread
James Chapman Feb 25, 2014 10:52 am
Steven DAprano Feb 25, 2014 10:45 pm
Danny Yoo Feb 25, 2014 10:55 pm
Steven DAprano Feb 25, 2014 11:05 pm
David Palao Feb 26, 2014 02:19 pm
James Chapman Feb 28, 2014 11:31 am
eryksun Mar 01, 2014 04:48 pm
James Chapman Mar 03, 2014 11:45 am
eryksun Mar 03, 2014 03:40 pm
Messages in this thread