Welcome, guest | Sign In | My Account | Store | Cart

Notice! PyPM is being replaced with the ActiveState Platform, which enhances PyPM’s build and deploy capabilities. Create your free Platform account to download ActivePython or customize Python with the packages you require and get automatic updates.

Download
ActivePython
INSTALL>
pypm install unidecode

How to install Unidecode

  1. Download and install ActivePython
  2. Open Command Prompt
  3. Type pypm install unidecode
 Python 2.7Python 3.2Python 3.3
Windows (32-bit)
0.04.13 Available View build log
0.04.9 Available View build log
0.04.7 Available View build log
0.04.6 Available View build log
0.04.5 Available View build log
0.04.1 Available View build log
Windows (64-bit)
0.04.13 Available View build log
0.04.9 Available View build log
0.04.7 Available View build log
0.04.6 Available View build log
0.04.5 Available View build log
0.04.1 Available View build log
Mac OS X (10.5+)
0.04.13 Available View build log
0.04.9 Available View build log
0.04.7 Available View build log
0.04.6 Available View build log
0.04.5 Available View build log
0.04.1 Available View build log
0.04.9
0.04.13Never BuiltWhy not?
0.04.9 Available View build log
Linux (32-bit)
0.04.13 Available View build log
0.04.9 Available View build log
0.04.7 Available View build log
0.04.6 Available View build log
0.04.5 Available View build log
0.04.1 Available View build log
0.04.9
0.04.13Never BuiltWhy not?
0.04.9 Available View build log
Linux (64-bit)
0.04.13 Available View build log
0.04.12 Available View build log
0.04.10 Available View build log
0.04.9 Available View build log
0.04.7 Available View build log
0.04.6 Available View build log
0.04.5 Available View build log
0.04.1 Available View build log
0.04.9
0.04.13Never BuiltWhy not?
0.04.9 Available View build log
0.04.13 Available View build log
0.04.12 Available View build log
0.04.10 Available View build log
0.04.9 Available View build log
 
Author
License
GPL
Imports
Lastest release
version 0.04.13 on Jun 1st, 2013

It often happens that you have text data in Unicode, but you need to represent it in ASCII. For example when integrating with legacy code that doesn't support Unicode, or for ease of entry of non-Roman names on a US keyboard, or when constructing ASCII machine identifiers from human-readable Unicode strings that should still be somewhat intelligeble (a popular example of this is when making an URL slug from an article title).

In most of these examples you could represent Unicode characters as "???" or "\15BA\15A0\1610", to mention two extreme cases. But that's nearly useless to someone who actually wants to read what the text says.

What Unidecode provides is a middle road: function unidecode() takes Unicode data and tries to represent it in ASCII characters (i.e., the universally displayable characters between 0x00 and 0x7F), where the compromises taken when mapping between two character sets are chosen to be near what a human with a US keyboard would choose.

The quality of resulting ASCII representation varies. For languages of western origin it should be between perfect and good. On the other hand transliteration (i.e., conveying, in Roman letters, the pronunciation expressed by the text in some other writing system) of languages like Chinese, Japanese or Korean is a very complex issue and this library does not even attempt to address it. It draws the line at context-free character-by-character mapping. So a good rule of thumb is that the further the script you are transliterating is from Latin alphabet, the worse the transliteration will be.

Note that this module generally produces better results than simply stripping accents from characters (which can be done in Python with built-in functions). It is based on hand-tuned character mappings that for example also contain ASCII approximations for symbols and non-Latin alphabets.

This is a Python port of Text::Unidecode Perl module by Sean M. Burke <sburke@cpan.org>.

Module content

The module exports a single function that takes an Unicode object (Python 2.x) or string (Python 3.x) and returns a string (that can be encoded to ASCII bytes in Python 3.x):

>>> from unidecode import unidecode
>>> unidecode(u'ko\u017eu\u0161\u010dek')
'kozuscek'
>>> unidecode(u'30 \U0001d5c4\U0001d5c6/\U0001d5c1')
'30 km/h'
>>> unidecode(u"\u5317\u4EB0")
'Bei Jing '

Requirements

Nothing except Python itself.

You will need a Python build with "wide" Unicode characters in order for unidecode to work correctly with characters outside of Basic Multilingual Plane. Surrogate pair encoding of "narrow" builds is not supported.

Installation

You install Unidecode, as you would install any Python module, by running these commands:

python setup.py install
python setup.py test

Source

You can get the latest development version of Unidecode with:

git clone http://www.tablix.org/~avian/git/unidecode.git

Support

Questions, bug reports, useful code bits, and suggestions for Unidecode should be sent to tomaz.solc@tablix.org

Subscribe to package updates

Last updated Jun 1st, 2013

Download Stats

Last month:15

What does the lock icon mean?

Builds marked with a lock icon are only available via PyPM to users with a current ActivePython Business Edition subscription.

Need custom builds or support?

ActivePython Enterprise Edition guarantees priority access to technical support, indemnification, expert consulting and quality-assured language builds.

Plan on re-distributing ActivePython?

Get re-distribution rights and eliminate legal risks with ActivePython OEM Edition.