This function can be used to check if a string contain only ASCII characters.
Python, 35 lines
It can seems inefficient to process the whole string with the translate method before doing any check but this has the great advantage to minimize loops in Python code. So I guess that this solution is faster in the common case than any clever algorithm written in Python.
Now if you have to process large files, it can be interesting to first check only a small part of the file, this will allow to reject most binary files.
The function considers a string to be ASCII if all of its characters are either ASCII printing characters (again, according to the X3.4 standard, not isascii()) or any of the following controls: bell, backspace, tab, line feed, form feed, carriage return, esc, nextline.
I include bell because some programs (particularly shell scripts) use it literally, even though it is rare in normal text. I exclude vertical tab because it never seems to be used in real text. I also include, with hesitation, the X3.64/ECMA-43 control nextline (0x85), because that's what the dd EBCDIC->ASCII table maps the EBCDIC newline character to. It might be more appropriate to include it in the 8859 set instead of the ASCII set, but it's got to be included in#something* we recognize or EBCDIC files aren't going to be considered textual. Some old Unix source files use SO/SI (^N/^O) to shift between Greek and Latin characters, so these should possibly be allowed. But they make a real mess on VT100-style displays if they're not paired properly, so we are probably better off not calling them text.
A string is considered to be ISO-8859 text if its characters are all either ASCII, according to the above definition, or printing characters from the ISO-8859 8-bit extension, characters 0xA0 ... 0xFF.
Finally, a string is considered to be international text from some other character code if its characters are all either ISO-8859 (according to the above definition) or characters in the range 0x80 ... 0x9F, which ISO-8859 considers to be control characters but the IBM PC and Macintosh consider to be printing characters.
The transcoding table and the previous discussion chapters come from the "file" package source code. Copyright (c) Ian F. Darwin, 1987. Written by Ian F. Darwin. Extensively modified by Eric Fischer in July, 2000, to handle character codes other than ASCII on a unified basis. Joerg Wunsch wrote the original support for 8-bit international characters. The following license apply to the table:
This software is not subject to any license of the American Telephone and Telegraph Company or of the Regents of the University of California.
Permission is granted to anyone to use this software for any purpose on any computer system, and to alter it and redistribute it freely, subject to the following restrictions: