| Store | Cart

Re: [Distutils] Working toward Linux wheel support

From: Leonardo Rochael Almeida <leor...@gmail.com>
Tue, 8 Sep 2015 16:18:10 -0300
Hi,

Going back in time to this old post, but I think it becomes more relevant
now that Nate's work is being completed:

On 13 August 2015 at 22:47, Nathaniel Smith <n...@pobox.com> wrote:

> On Thu, Aug 13, 2015 at 12:30 PM, Leonardo Rochael Almeida> <leor...@gmail.com> wrote:> >> > On 13 August 2015 at 11:07, Nate Coraor <n...@bx.psu.edu> wrote:> >>> >> On Wed, Aug 12, 2015 at 9:05 PM, Nathaniel Smith <n...@pobox.com> wrote:> >>>> [...]> >>> (2) the special hard-coded tag "centos5". (That's what everyone> actually> >>> uses in practice, right?)> >>> >>> >> The idea here is that we should attempt to install centos5 wheels if> more> >> specific wheels for the platform aren't available?> >> >> > Just my opinion, but although I'm +1 on Nate's efforts, I'm -1 on both> the> > standard behavior for installation being the exact platform tag, and an> > automatic fallback to cento5.> >> > IMO, on Linux, the default should always be to opt in to the desired> > platform tags.> >> > We could make it so that the word `default` inside> > `binary-compatibility.cfg` means an exact match on the distro version, so> > that we could simplify the documentation.> >> > But I don't want to upgrade to pip and suddenly find myself installing> > binary wheels compiled by whomever for whatever platform I have no> control> > with, even assuming the best of the package builders intentions.> >> > And I certainly don't want centos5 wheels accidentally installed on my> > ubuntu servers unless I very specifically asked for them.> >> > The tiny pain inflicted by telling users to add a one-line text file in a> > very well known location (or two lines, for the added centos5), so that> they> > can get the benefit of binary wheels on linux, is very small compared to> the> > pain of repeatable install scripts suddenly behaving differently and> > installing binary wheels in systems that were prepared to pay the price> of> > source installs, including the setting of build environment variables> that> > correctly tweaked their build process.>> I think there are two issues here:>> 1) You don't want centos5 wheels "accidentally" installed on an ubuntu> server: Fair enough, you're right; we should probably make the "this> wheel should work on pretty much any linux out there" tag be something> that distributors have to explicitly opt into (similar to how they> have to opt into creating universal wheels), rather than having it be> something you could get by just typing 'pip wheel foo' on the right> (wrong) machine.>

I agree that generating something like "this linux binary wheel is
generically installable" should be opt-in, yes. But I also feel strongly
that installing such a generic wheel should also be opt in.

I guess that if we go into the direction of being able to generate wheels
with a libc tag rather than a distro tag, like Nate and Donald are now
discussing, then we could get both kinds of opt-in by specifying the libc
tag in `binary-compatibility.cfg`.


> 2) You want it to be the case that if I type 'pip install foo' on a> Linux machine, and pip finds both an sdist and a wheel, where the> wheel is definitely compatible with the current system, then it should> still always prefer the sdist unless configured otherwise: Here I> disagree strongly. This is inconsistent with how things work on every> other platform, it's inconsistent with how pip is being used on Linux> right now with private wheelhouses, and the "tiny pain" of editing a> file in /etc is a huge barrier to new users, many of whom are> uncomfortable editing config files and may not have root access.


Not having root access shouldn't be an issue, as there should be a
user-level and virtualenv level equivalents to a `binary-compatibility.cfg`
on `/etc`, and perhaps it could even be included in `requirements.txt` for
a project, so users of a project might not even have to bother setting up
`binary-compatibility.cfg`.

However, you make an excellent point: not handling binary wheels on Linux
by default (at least with exact platform tag matching) would mean having
different behavior between Linux and Mac/Windows.

Still, I wouldn't want a random binary wheel suddenly finding its way into
my servers, and I would like a way to opt out of it, for "reasons" (ex. I
might have special build flags, or a special compiler, or maybe I'm still
waiting for TUF before trusting other peoples binaries on my servers).

So I'd like to propose that the installation tooling (eg. `pip`, `distlib`)
should allow the user to specify which index servers to trust for receiving
binary wheels and which to trust only for pure python wheels or sdists.

It could trust them all by default, to maintain the current behavior (where
`all` means only pypi unless I specified more, obviously), but I'd like a
switch to limit this trust to a subset of the specified index servers.

Regards,

Leo

_______________________________________________
Distutils-SIG maillist  -  Dist...@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Recent Messages in this Thread
Nate Coraor Jul 16, 2015 05:41 pm
Nick Coghlan Sep 06, 2015 11:32 pm
Nick Coghlan Sep 05, 2015 02:56 am
Marcus Smith Sep 07, 2015 04:11 am
Nate Coraor Sep 03, 2015 05:22 pm
Nick Coghlan Sep 07, 2015 04:26 am
Daniel Holth Aug 20, 2015 07:22 pm
Donald Stufft Aug 20, 2015 07:25 pm
Donald Stufft Sep 02, 2015 11:45 pm
Daniel Holth Sep 03, 2015 12:15 pm
Nate Coraor Aug 24, 2015 03:03 pm
Wes Turner Aug 24, 2015 05:51 pm
Nick Coghlan Jul 17, 2015 08:22 am
Chris Barker Jul 17, 2015 03:36 pm
Antoine Pitrou Jul 17, 2015 03:46 pm
Chris Barker Jul 17, 2015 03:53 pm
Andrea Bedini Jul 18, 2015 07:00 am
Tres Seaver Jul 21, 2015 03:25 am
Leonardo Rochael Almeida Jul 21, 2015 03:07 pm
Marcus Smith Jul 17, 2015 04:50 pm
Olivier Grisel Jul 17, 2015 06:34 pm
Daniel Holth Jul 17, 2015 08:18 pm
Chris Barker - NOAA Federal Jul 18, 2015 01:13 am
Daniel Holth Jul 18, 2015 02:11 am
Paul Moore Jul 18, 2015 11:51 am
Nick Coghlan Jul 20, 2015 05:50 am
Chris Barker Jul 20, 2015 05:37 pm
Paul Moore Jul 20, 2015 06:37 pm
Nick Coghlan Jul 27, 2015 02:19 pm
Nate Coraor Jul 27, 2015 07:07 pm
Oscar Benjamin Jul 21, 2015 04:38 pm
Chris Barker Jul 24, 2015 06:52 pm
Oscar Benjamin Jul 28, 2015 03:02 pm
Wes Turner Jul 28, 2015 04:21 pm
Nate Coraor Aug 12, 2015 08:21 pm
Robert Collins Aug 12, 2015 11:49 pm
Nathaniel Smith Aug 13, 2015 01:05 am
Nate Coraor Aug 13, 2015 02:07 pm
Leonardo Rochael Almeida Aug 13, 2015 07:30 pm
Wes Turner Aug 13, 2015 07:43 pm
Nathaniel Smith Aug 14, 2015 01:47 am
Wes Turner Aug 14, 2015 01:50 am
Nathaniel Smith Aug 14, 2015 02:33 am
Wes Turner Aug 14, 2015 02:41 am
Leonardo Rochael Almeida Sep 08, 2015 07:18 pm
Donald Stufft Sep 08, 2015 07:22 pm
Leonardo Rochael Almeida Sep 08, 2015 07:39 pm
Nathaniel Smith Aug 14, 2015 01:25 am
Robert Collins Aug 14, 2015 01:31 am
Wes Turner Aug 14, 2015 01:38 am
Robert Collins Aug 14, 2015 01:44 am
Wes Turner Aug 14, 2015 01:44 am
Nathaniel Smith Aug 14, 2015 02:14 am
Wes Turner Aug 14, 2015 02:24 am
Robert Collins Aug 14, 2015 02:27 am
Nathaniel Smith Aug 14, 2015 07:38 am
David Cournapeau Aug 13, 2015 05:52 pm
Nathaniel Smith Aug 14, 2015 04:07 am
Chris Barker Aug 14, 2015 04:04 pm
David Cournapeau Aug 14, 2015 04:20 pm
Chris Barker Aug 14, 2015 04:00 pm
Leonardo Rochael Almeida Jul 20, 2015 01:42 am
Nick Coghlan Jul 20, 2015 06:00 am
Chris Barker Jul 20, 2015 05:39 pm
Marcus Smith Sep 06, 2015 04:09 pm
Nick Coghlan Sep 05, 2015 08:35 am
Nick Coghlan Sep 05, 2015 06:44 am
Nick Coghlan Sep 05, 2015 06:43 am
Nathaniel Smith Sep 05, 2015 06:46 am
Donald Stufft Sep 05, 2015 03:06 am
Wes Turner Sep 08, 2015 07:14 pm
Daniel Holth Sep 08, 2015 07:32 pm
Nathaniel Smith Sep 09, 2015 11:49 pm
Nate Coraor Sep 21, 2015 03:33 pm
Nate Coraor Sep 09, 2015 03:06 pm
Donald Stufft Sep 08, 2015 06:33 pm
Donald Stufft Sep 07, 2015 04:02 pm
Marcus Smith Sep 07, 2015 05:51 pm
Wes Turner Sep 07, 2015 10:16 pm
Nate Coraor Sep 03, 2015 02:04 pm
Daniel Holth Sep 03, 2015 01:56 pm
Antoine Pitrou Aug 20, 2015 07:51 pm
Nate Coraor Aug 20, 2015 07:40 pm
Donald Stufft Aug 20, 2015 07:19 pm
Antoine Pitrou Aug 20, 2015 07:14 pm
Steve Dower Aug 14, 2015 04:17 pm
Daniel Holth Aug 20, 2015 06:38 pm
Chris Barker Aug 14, 2015 08:16 pm
Alexander Walters Aug 14, 2015 10:32 pm
Nate Coraor Aug 20, 2015 06:26 pm
Nick Coghlan Sep 05, 2015 02:12 am
Daniel Holth Sep 01, 2015 01:57 pm
Wes Turner Aug 26, 2015 01:42 am
Nate Coraor Aug 27, 2015 07:21 pm
Ben Finney Sep 06, 2015 11:42 pm
Messages in this thread