[Python-Dev] codecs.oen [was: PEP 385: the eol-type issue]

Jim Jewett <jimjjewett <at> gmail.com> writes:
> > In python 3, why does codecs.open even still exist?

I don't remember anyone proposing to deprecate it, so I suppose that's the
(social) reason.

> So at this point, are there any differences beyond:

(c) The built-in open is probably a little more featureful, especially when it
comes to seek() and tell().

> (b)  The codecs version is much slower, because it hasn't seen the> optimization effort.

By the way, the built-in open would also benefit from an optimization of
codecs.py's IncrementalEncoder classes: they are just thin Python wrappers
around C function calls, and the overhead of calling a Python method is very
significant when doing a lot of small unicode writes with a non-optimized codec
(a couple of dominant codecs have been optimized by means of internal shortcuts
bypassing codecs.py: latin-1, utf-8, utf-16).

Regards

Antoine.


> (a)  The builtin open doesn't work on multi-byte line-endings other> than the multi-character CRLF.  (In other words, it goes by the> traditional Operating System conventions developed when a char was a> byte, but the Unicode standard allows for a few more possibilities,> which are currently rare in practice.)>