Re: [Python-Dev] Status of packaging in 3.3

On Thu, Jun 21, 2012 at 9:48 PM, Chris McDonough <chr...@plope.com> wrote:
> On 06/21/2012 04:45 AM, Nick Coghlan wrote:>> And, like it or not, setuptools has a serious PR problem due to the>> fact it monkeypatches the standard library, uses *.pth files to alter>> sys.path for every installed application by default, actually *uses*>> the ability to run code in *.pth files and has hard to follow>> documentation to boot. I *don't* trust that I fully understand the>> import system on any machine with setuptools installed, because it is>> demonstrably happy to install state to the file system that will>> affect *all* Python programs running on the machine.>>> I don't know about Red Hat but both Ubuntu and Apple put all kinds of stuff> on the default sys.path of the system Python of the box that's related to> their software's concerns only.  I don't understand why people accept this> but get crazy about the fact that installing a setuptools distribution using> easy_install changes the default sys.path.

Because the vendor gets to decide what goes into the base install of
the OS. If I'm using the system Python, then I expect sys.path to
contain the system paths, just as I expect gcc to be able to see the
system include paths. If I don't want that, I'll use virtualenv or a
completely separate Python installation.

However, when I install a new Python package into site-packages it
*should* just sit there and have zero impact on other Python
applications that don't import that package. As soon as someone
installs a *.pth file, however, that's *no longer the case* - every
Python application on that machine will now be scanning additional
paths for modules whether it wants to or not. It's unnecessary
coupling between components that *should* be completely independent of
each other.

Now, *.pth support in the interpreter certainly cannot be blamed on
setuptools, but encouraging use of a packaging format that effectively
requires them certainly can be.

It's similar to the reason why monkeypatching and global environment
variable modifications (including PYTHONPATH) are a problem: as soon
as you start doing that kind of thing, you're introducing coupling
that *shouldn't exist*. If there is no better solution, then sure, do
it as a near term workaround, but that isn't the same as accepting it
as the long term answer.

> Installing a distribution will change behavior whether or not sys.path is> changed as a result.  That's its purpose.

No it won't. An ordinary package will only change the behaviour of
Python applications that import a package by that name. Other Python
applications will be completely unaffected (as it should be).

> The code that runs in the .pth> *file* (there's only one that matters: easy_install.pth) just mutates> sys.path.  The end result is this: if you understand how sys.path works, you> understand how eggs work.  Each egg is addded to sys.path.  That's all there> is to it.  It's the same as manually mutating a global PYTHONPATH, except> you don't need to do it.

Yes, it's the same as mutating PYTHONPATH. That's a similarly bad
system global change. Individual libraries do not have the right to
change the sys.path seen on initialisation by every other Python
application on that system.

> And note that this is not "setuptools" in general.  It's easy_install in> particular.  Everything you've brought up so far I think is limited to> easy_install.  It doesn't happen when you use pip.  I think it's a mistake> that pip doesn't do it, but I think you have to make more accurate> distinctions.

What part of "PR problem" was unclear? setuptools and easy_install are
inextricably linked in everyone's minds, just like pip and distribute.

>> A packaging PEP needs to explain:>> - what needs to be done to eliminate any need for monkeypatching>> - what's involved in making sure that *.pth are *not* needed by default>> - making sure that executable code in implicitly loaded *.pth files>> isn't used *at all*>> I'll note that these goals are completely sideways to any actual functional> goal.  It'd be a shame to have monkeypatching going on, but the other stuff> I don't think are reasonable goals.  Instead they represent fears, and those> fears just need to be managed.

No, they reflect the mindset of someone with configuration management
and auditing responsibilities for shared systems with multiple
applications installed which may be written in a variety of languages,
not just Python. You may not care about those people, but I do.

> It'd also be useful if other core developers actually tried to use> setuptools in anger.  That'd be a good start towards understanding some of> its tradeoffs.  People can write this stuff down til they're blue in the> face, but if core devs  don't try the stuff, they'll always fear it.

setuptools (or, perhaps, easy_install, although I've seen enough posts
about eggs being uploaded to PyPI to suspect otherwise), encourages
the deployment of system configuration changes that alter the runtime
environment of every single Python application executed on the system.
That's simply not cool.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
Pyth...@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev-ml%40activestate.com