| Store | Cart

Re: [Python-ideas] Learning from the shell in supporting asyncio background calls

From: Nick Coghlan <ncog...@gmail.com>
Sat, 11 Jul 2015 15:04:22 +1000
On 10 July 2015 at 21:51, Guido van Rossum <gui...@python.org> wrote:
> As I wrote on the issue, I'm -1 on this proposal. Not only does this API> encourage beginners to ignore the essential difference between synchronous> functions meant to run in a thread (using synchronous I/O and pre-emptive> CPU scheduling) and asyncio coroutines/tasks (which use overlapped I/O and> require explicit scheduling), it also encourages avoiding the "await"> primitive (formerly "yield from") in favor of a function call which cannot> be used from within a coroutine/task.

My apologies for the confusion - the revised proposal focuses on
coroutines, not threads. With the benefit of hindight, leaving the
implementation details out of the python-ideas post was clearly a
mistake, as my previous posts had been more focused on threads. I've
added the full implementation details in my reply to Oscar, which will
hopefully make the revised proposal clearer.

The blog post goes into detail on this - it specifically takes a
synchronous function, replaces it with an asynchronous coroutine using
the await syntax, and then uses run_in_background() to manipulate the
asynchronous version from the REPL.

The main operation I use with "run_in_foreground" in the post is
actually asyncio.sleep, as "run_in_foreground(asyncio.sleep(0))" was
the simplest way I found to single step the event loop, and it also
allows you to trivially say "run the event loop for 5 seconds", etc.
Concatenating some of the example code from the post together gives
this demonstration of the basic UX:

    >>> async def ticker():
    ...     for i in itertools.count():
    ...         print(i)
    ...         await asyncio.sleep(1)
    ...
    >>> ticker1 = run_in_background(ticker())>>> ticker1
    <Task pending coro=<ticker() running at <stdin>:1>>
    >>> run_in_foreground(asyncio.sleep(5))
    0
    1
    2
    3
    4

If there isn't a coroutine currently running in the foreground, then
background coroutines don't run either. All of the currently running
tasks can be interrogated through the existing
asyncio.Task.all_tasks() class method.

> This particular spelling moreover introduces a "similarity" between> foreground and background tasks that doesn't actually exist.

The concept behind the revised proposal is layering the simpler
foreground/background task representational model on top of the full
complexity of the asyncio implementation model.

The "run_in_foreground" naming is technically a lie - what actually
gets run in the foreground is the current thread's event loop.
However, I think it's an acceptable and useful lie, as what it does is
run the event loop in the current thread until the supplied future
produces a result, which means the current thread isn't going to be
doing anything other than running the event loop until the specified
operation is completed.

This approach *doesn't* expose the full power of asyncio and native
coroutines, but it exposes a lot of it, and it should be relatively
easy to grasp for anyone that's already familiar with background
processes in POSIX shell environments.

> The example suggests that this should really be a pair of convenience> functions in collections.futures, as it does not make any use of asyncio.

While that was true of the previous proposal (which always used the
executor), this new proposal only falls back to using run_in_executor
if asyncio.ensure_future fails with TypeError and the supplied
background task target is a callable.

Regards,
Nick.

P.S. If anyone reading this isn't already familiar with the concept of
representational models vs implementation models, then I highly
recommend http://www.uxpassion.com/blog/implementation-mental-representation-models-ux-user-experience/
as a good introduction to the idea

-- 
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Pyth...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Recent Messages in this Thread
Nick Coghlan Jul 10, 2015 10:49 am
Oscar Benjamin Jul 10, 2015 11:48 am
Nick Coghlan Jul 11, 2015 04:33 am
Guido van Rossum Jul 10, 2015 11:51 am
Nick Coghlan Jul 11, 2015 05:04 am
Nathaniel Smith Jul 11, 2015 05:16 am
Nick Coghlan Jul 11, 2015 10:17 am
Nick Coghlan Jul 12, 2015 02:48 am
Sven R. Kunze Aug 11, 2015 09:26 pm
Jonathan Slenders Aug 11, 2015 10:37 pm
Sven R. Kunze Aug 13, 2015 06:48 am
Jonathan Slenders Aug 11, 2015 10:59 pm
Nick Coghlan Aug 19, 2015 09:24 am
Sven R. Kunze Aug 20, 2015 03:27 pm
Stephen J. Turnbull Aug 21, 2015 02:51 am
Andrew Barnert via Python-ideas Aug 21, 2015 04:22 am
Nick Coghlan Jul 11, 2015 07:04 am
Messages in this thread