Welcome, guest | Sign In | My Account | Store | Cart

Extract all the text of a PDF (or other supported container types) at very high speed. In general, text pieces of a PDF page are not arranged in natural reading order, but in the order they were entered during PDF creation. This script re-arranges text blocks according to their pixel coordinates to achieve a more readable output, i.e. top-down, left-right.

Python, 97 lines
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
#!/usr/bin/env python
"""
Created on Wed Jul 29 07:00:00 2015

@author: Jorj McKie
Copyright (c) 2015 Jorj X. McKie

The license of this program is governed by the GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007. See the "COPYING" file of this repository.

This is an example for using the Python binding PyMuPDF of MuPDF.

This program extracts the text of an input PDF and writes it in a text file.
The input file name is provided as a parameter to this script (sys.argv[1])
The output file name is input-filename appended with ".txt".
Encoding of the text in the PDF is assumed to be UTF-8.
Change the ENCODING variable as required.
-------------------------------------------------------------------------------
"""
import fitz                 # this is PyMuPDF
import sys, json

ENCODING = "UTF-8"

def SortBlocks(blocks):
    '''
    Sort the blocks of a TextPage in ascending vertical pixel order,
    then in ascending horizontal pixel order.
    This should sequence the text in a more readable form, at least by
    convention of the Western hemisphere: from top-left to bottom-right.
    If you need something else, change the sortkey variable accordingly ...
    '''

    sblocks = []
    for b in blocks:
        x0 = str(int(b["bbox"][0]+0.99999)).rjust(4,"0") # x coord in pixels
        y0 = str(int(b["bbox"][1]+0.99999)).rjust(4,"0") # y coord in pixels
        sortkey = y0 + x0                                # = "yx"
        sblocks.append([sortkey, b])
    sblocks.sort()
    return [b[1] for b in sblocks] # return sorted list of blocks

def SortLines(lines):
    ''' Sort the lines of a block in ascending vertical direction. See comment
    in SortBlocks function.
    '''
    slines = []
    for l in lines:
        y0 = str(int(l["bbox"][1] + 0.99999)).rjust(4,"0")
        slines.append([y0, l])
    slines.sort()
    return [l[1] for l in slines]

def SortSpans(spans):
    ''' Sort the spans of a line in ascending horizontal direction. See comment
    in SortBlocks function.
    '''
    sspans = []
    for s in spans:
        x0 = str(int(s["bbox"][0] + 0.99999)).rjust(4,"0")
        sspans.append([x0, s])
    sspans.sort()
    return [s[1] for s in sspans]

#==============================================================================
# Main Program
#==============================================================================
ifile = sys.argv[1]
ofile = ifile + ".txt"

doc = fitz.Document(ifile)
pages = doc.pageCount
fout = open(ofile,"w")

for i in range(pages):
    pg_text = ""                                 # initialize page text buffer
    pg = doc.loadPage(i)                         # load page number i
    text = pg.getText(output = 'json')           # get its text in JSON format
    pgdict = json.loads(text)                    # create a dict out of it
    blocks = SortBlocks(pgdict["blocks"])        # now re-arrange ... blocks
    for b in blocks:
        lines = SortLines(b["lines"])            # ... lines
        for l in lines:
            spans = SortSpans(l["spans"])        # ... spans
            for s in spans:
                # ensure that spans are separated by at least 1 blank
                # (should make sense in most cases)
                if pg_text.endswith(" ") or s["text"].startswith(" "):
                    pg_text += s["text"]
                else:
                    pg_text += " " + s["text"]
            pg_text += "\n"                      # separate lines by newline

    pg_text = pg_text.encode(ENCODING, "ignore")
    fout.write(pg_text)

fout.close()

2 comments

Jorj X. McKie (author) 8 years, 1 month ago  # | flag

Please note, that the script also works for the other file types that are supported by MuPDF: OpenXPS, CBZ, EPUB, ...

Jorj X. McKie (author) 8 years, 1 month ago  # | flag

For whom it may be of interest:

(Py-) MuPDF text extraction is (one of) the fastest ways to extract text from PDFs. Several dozen (sic!)times faster than pdfminer, still sveral times faster than XPDF utilities, etc.

Similar statements hold true for other PyMuPDF capabilities like image rendering.

Have a look at PyMuPDF's documentation appendix (hosted at PyPI) - you will find many details on performance comparisons.