Re: [Python-Dev] How io.IOBase.readline() should behave when used on non-blocking obj and no data available?

Hello,

On Thu, 16 Oct 2014 13:34:06 +0200
Antoine Pitrou <soli...@pitrou.net> wrote:

> On Thu, 16 Oct 2014 03:54:32 +0300> Paul Sokolovsky <pmis...@gmail.com> wrote:> > Hello,> > > > io.RawIOBase.read() is well specified for behavior in case it> > immediately gets a would-block condition: "If the object is in> > non-blocking mode and no bytes are available, None is returned."> > (https://docs.python.org/3/library/io.html#io.RawIOBase.read).> > > > However, nothing is said about such condition for> > io.IOBase.readline(), which is mixin method in a base class,> > default implementation of which thus would use io.RawIOBase.read().> > Looking at 3.4.0 source, iobase.c: iobase_readline() has:> > > >         b = _PyObject_CallMethodId(self, &PyId_read, "n",> > nreadahead); [...]> >         if (!PyBytes_Check(b)) {> >             PyErr_Format(PyExc_IOError,> >                          "read() should have returned a bytes> > object, " "not '%.200s'", Py_TYPE(b)->tp_name);> > > > I.e. it's not even ready to receive legitimate return value of None> > from read(). I didn't try to write a testcase though, so may be> > missing something.> > > > So, how readline() should behave in this case, and can that be> > specified in the Library Reference?> > Well, the problem is that it's not obvious how to implement such> methods in a non-blocking context.> > Let's says some data is received but there isn't a complete line.> Should readline() return just that data (an incomplete line)? That> breaks the API's contract. Should readline() buffer the incomplete> line and keep it for the next readline() call? But then the internal> buffer becomes unbounded: perhaps there is no new line in the next> 4GB of incoming data...> > And besides, raw I/O objects *shouldn't* have an internal buffer.> That's the role of the buffered I/O layer.

Yes, sure, readline() is defined on io.IOBase which is underspecified
for buffered-ness, so should have behavior which can be implemented for
both buffered and unbuffered case.

You're right also in saying that readline on non-blocking stream can't
work always the same way as blocking version, and that it "breaks the
API's contract". But it should be possible to extend that contract
for non-blocking readline() in pretty natural way:

1) An invariant of readline() is that it doesn't modify stream data,
it just segments it. So, readline() + write() looped until EOF will
produce the same result as read(N) + write(). Non-blocking readline()
will still satisfy this.

2) Even with blocking readline(), it can return a string not ending
with end-of-line character(s). For blocking readline, this may happen
with just the last line of a stream, with non-blocking, it may happen
for any call. The point is that even with blocking readline(), the
caller should be ready to check that a line satisfies its "complete
line" criteria, for non-blocking case, it's just will be different set
of criteria and actions to satisfy them.


I guess, defining non-blocking readline() in such way is better then
let it be underspecified whether it's supported or not (and if yes,
then how), or prohibit it.


-- 
Best regards,
 Paul                          mailto:pmis...@gmail.com
_______________________________________________
Python-Dev mailing list
Pyth...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/python-dev-ml%40activestate.com