A generator function which takes a file object (assumed to be some sort of pipe or socket, open for reading), and yields lines from it without blocking. If there is no input available, it will yield an endless stream of empty strings until input becomes available again; caller is responsible for not going into a busy loop. (Newlines are normalized but not stripped, so if there is actually a blank line in the input, the value yielded will be '\n'.) The intended use case is a thread which must respond promptly to input from a pipe, and also something else which cannot be fed to select (e.g. a queue.Queue). Note that the file object is ignored except for its fileno.
Only tested on Unix. Only tested on 3.4; ought to work with any python that has bytearray, locale.getpreferredencoding, and fcntl.
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | def nonblocking_readlines(f):
"""Generator which yields lines from F (a file object, used only for
its fileno()) without blocking. If there is no data, you get an
endless stream of empty strings until there is data again (caller
is expected to sleep for a while).
Newlines are normalized to the Unix standard.
"""
fd = f.fileno()
fl = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
enc = locale.getpreferredencoding(False)
buf = bytearray()
while True:
try:
block = os.read(fd, 8192)
except BlockingIOError:
yield ""
continue
if not block:
if buf:
yield buf.decode(enc)
buf.clear()
break
buf.extend(block)
while True:
r = buf.find(b'\r')
n = buf.find(b'\n')
if r == -1 and n == -1: break
if r == -1 or r > n:
yield buf[:(n+1)].decode(enc)
buf = buf[(n+1):]
elif n == -1 or n > r:
yield buf[:r].decode(enc) + '\n'
if n == r+1:
buf = buf[(r+2):]
else:
buf = buf[(r+1):]
|
