Welcome, guest | Sign In | My Account | Store | Cart

This hack runs a command sequentially on a list of files using two simultaneous threads. If one of the commands takes more than a set time, it's killed and the program goes for the next file. EDITED to add some exception handling.

Python, 45 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#process a filtered list of files by calling reapeatedly a 
#console app(no console opens) in two parallel threads with a timeout
#Antoni Gual May 2015

import os
import threading
import subprocess

def my_thread():
  global files,path,timeout,options
  myname= threading.currentThread().getName()
  while files:
     #create command to run
     nextfile=files.pop() 
     #print name of thread and command being run
     print('Thread {0} starts processing {1}'.format(myname,nextfile))
     f=path + nextfile + options
     try:
        #timeout interrupts frozen command, shell=True does'nt open a console
        subprocess.check_call(args= f , shell=True, timeout=timeout)
     except subprocess.TimeoutExpired:
        print('Thread {0} Processing {0} took too long' .format(myname,nextfile))
     except subprocess.CalledProcessError as e: 
        print ('Thread {0} Processing {1} returned error {2}:{3}'.format(myname,nextfile,e.returncode,e.output))
     except Exception as e:
        print ('Thread {0} Processing {1} returned error {2}'.format(myname,nextfile,type(e).__name__))
  print ('thread {0} stopped'.format(myname))
  
  
timeout=150    
#the patth to the console app
exe_path = '\"C:/Program files/Calibre2/ebook-convert.exe" ' 
file_path = './' # so it can be called from a console opened in the folder whrer files are
options = '\" .epub > nul'

#filter the files in file_path
extensions = ['mobi','lit','prc','azw','rtf','odf' ] ;
files = [fn for fn in os.listdir(file_path) if any([fn.endswith(ext) for ext in extensions])];

path=exe_path +' \"'+file_path
#runs the same thread twice, each with a name
t1= threading.Thread(target=my_thread, name='uno' )
t1.start()
t2= threading.Thread(target=my_thread,name='dos' )
t2.start()

I use to convert ebooks from different formats to epub, using the command line ebook-converter.exe from the freeware Calibre. When I tried to run it on a list of files from a Windows console command i found it next morning stalled on a particular file with 90% of the list unprocessed. Python allows to kill a process that takes too much time and go for the next file. Calibre runs in parallel several instances of the converter so i did the same.

Created by Antoni Gual on Tue, 19 May 2015 (MIT)
Python recipes (4591)
Antoni Gual's recipes (16)
Angosso (2)

Required Modules

  • (none specified)

Other Information and Tasks