tail -f in Python (Python recipe) by Erik Max Francis

ActiveState Code (http://code.activestate.com/recipes/157035/)

How to make it usable.

import time, os

#Set the filename and open the file
filename = 'security_log'
file = open(filename,'r')

#Find the size of the file and move to the end
st_results = os.stat(filename)
st_size = st_results[6]
file.seek(st_size)

while 1:
    where = file.tell()
    line = file.readline()
    if not line:
        time.sleep(1)
        file.seek(where)
    else:
        print line, # already has newline

What tail-10 would do. Sometimes you actually just want the last 10 lines of the file. This will work quite happily on 10Meg files.

I would love to know if there is a more efficient method.

import sys
def tail_lines(filename,linesback=10,returnlist=0):
    """Does what "tail -10 filename" would have done
       Parameters:
            filename   file to read
            linesback  Number of lines to read from end of file
            returnlist Return a list containing the lines instead of a string

    """
    avgcharsperline=75

    file = open(filename,'r')
    while 1:
        try: file.seek(-1 * avgcharsperline * linesback,2)
        except IOError: file.seek(0)
        if file.tell() == 0: atstart=1
        else: atstart=0

        lines=file.read().split("\n")
        if (len(lines) > (linesback+1)) or atstart: break
        #The lines are bigger than we thought
        avgcharsperline=avgcharsperline * 1.3 #Inc avg for retry
    file.close()

    if len(lines) > linesback: start=len(lines)-linesback -1
    else: start=0
    if returnlist: return lines[start:len(lines)-1]

    out=""
    for l in lines[start:len(lines)-1]: out=out + l + "\n"
    return out

print tail_lines('/etc/hosts',5,1)
sys.stdout.write(tail_lines('/etc/hosts',5))

Err, unless I'm missing something. This seems rather inefficient - suppose several lines are added to the file at once - this version will read them a line at a time; pausing for a second between each.

I'm doing more or less the same, but seeking to EOF then using readlines: infile,seek(0,2) while 1: lines=infile.readlines if not lines: time.sleep(1) else

print the line, or pattern match in it, etc.

Better? (New to Python, so I could be way off the mark!)

A simple addition... A nice addition to this is to use "yield" to make the code generic. That is,

def tail_f(file):
  interval = 1.0

  while True:
    where = file.tell()
    line = file.readline()
    if not line:
      time.sleep(interval)
      file.seek(where)
    else:
      yield line

Which then allows you to write code like:

for line in tail_f(open(sys.argv[1])):
  print line,

The original code will not sleep on each line, only if there are no lines to be had

if I attempt to insert the output into Qlistbox using

self.listboxname.insertItem(file)

I get "QListBox.insertItem() has an invalid type" do I need to somehow format it?

@ Ed Pascoe code. HI Iam tried using your code for the same requirement at my side. It is working fine and but I have some doughts I actually not understood the logic properly. I tried modifying your code and just removed some lines like " if file.tell()" condition block and "avgcharsperline" statement but I do get 10 lines some times, but some times it prints 8 or 6 or 3 etc... lines. I am little confused actually those above statements what they do could you please elaborate about that thank you.

@Shashidhar: The statements you removed from Ed Pascoe’s code using using file.tell() and adjusting the avgcharsperline value is there to make sure that enough data has been from the end of the file to hold the desired number of lines. Eliminating the logic means that a different number of lines will most likely be printed (as you're apparently finding out).