Re: [Distutils] Proposal: using /etc/os-release in the "platform tag" definition for wheel files

On Sun, 30 Nov 2014 01:47:16 +1000
Nick Coghlan <ncog...@gmail.com> wrote:
> On 29 November 2014 at 01:51, Antoine Pitrou <soli...@pitrou.net> wrote:> > On Sat, 29 Nov 2014 01:27:44 +1000> > Nick Coghlan <ncog...@gmail.com> wrote:> >> >> >> > Is this not going to be a slippery slope?> >>> >> Only if folks publish Linux binaries themselves, and that's still a> >> bad idea (for the same reason publishing distro binaries is already a> >> rare thing for people to do).> >> > Well, let's not make this a matter of ideology. Everyone knows it's a> > bad idea to publish binaries, yet it's often better than nothing,> > especially if the software is tedious to compile.> > It's not a matter of ideology, but a matter of practicality. Debian> stable, RHEL/CentOS, Ubuntu LTS, SLES - distros like these move slow> enough (and have strong enough ABI compatibility guarantees) to be> practical for ISVs to target with prebuilt binaries.

It seems we disagree on the notion of "practicality" :-)

For me practicality means being able to build a single binary package
for all recent Linux distros in a best effort approach. Building a
different package for each distro version is far from practical for any
reasonable-sized project (i.e. not something sponsored by a
1000+-employee entity, with a dedicated build team and infrastructure).

> > Case in point: can I ask you (the mythical "we") to build packages for> > all major distros (including supported LTS releases), and the four most> > recent Python versions, of the following piece of software:> > https://github.com/numba/llvmlite> > No, that would be a service provided by the as yet hypothetical PyPI> build farm. If/when that happens, it will need to have a way of> tagging Linux wheels appropriately, though.

"If/when that happens" is not reassuring, especially in the light of
how many pie-in-the-sky improvements in the packaging ecosystems have
turned out :-/

(at Continuum we have started offering such a service, but it's
"generic Linux": http://docs.binstar.org/build-config.html#BuildMatrix)

> Nearer term (and what prompted me to start this thread), the Fedora> Environments & Stacks working group is investigating providing> prebuilt wheel files for the Fedora ecosystem, and potentially for> EPEL as well (see> https://fedoraproject.org/wiki/Env_and_Stacks/Projects/UserLevelPackageManagement> for the broader context of that effort). For other ecosystems, you'll> have to ask participants in those ecosystems.

That's asking software authors to complicate and slow down their
development process a lot. Also, there's no guarantee that Fedora or
Ubuntu or whatever would actually *accept* to help us, right?

> > I'm not sure I understand: distros provide their own packages, they> > don't (shouldn't) blindly pull binary wheels from PyPI. Why would they> > depend on the wheel tagging format?> > We don't plan to blindly pull anything from PyPI - we're looking at> the feasibility of publishing already reviewed software in ecosystem> native formats (with the two pilot projects focusing on Java JAR files> and Python wheel files).> > When I last mentioned that idea here, Marcus pointed out that doing> that with the generic "linux_x86_64" compatibility tag on the wheel> filenames would be problematic, as there'd be nothing preventing> anyone from pulling them down onto inappropriate systems, with no> obvious trace of the Fedora or EPEL specific assumptions that went> into building them.

Uh, there's a lot of hidden knowledge required to understand those
two paragraphs that I don't master. I don't know what "inappropriate
systems" are, what are "reviewed software", etc. ;-)

Also I don't understand why you're not recompiling as you would
normally do.

> While that's a valid concern, I also don't want to go invent our own> custom compatibility tagging convention just for Fedora & EPEL, but> rather work within the limits of what upstream Python packaging> natively supports.

Well, *allowing* distro tags in the platform tag is certainly ok. What
I'm afraid of is if that's made mandatory.

> Consider if the following could be included in the "pyvenv.cfg" file> in a virtual environment:> > [compatibility]> python=cp27,cp2,py2> abi=cp27mu> platform=linux_x86_64_epel_6> > Or for a Python 3 virtual environment:> > [compatibility]> python=cp34,cp3,py3> abi=cp34m,abi3> platform=linux_x86_64_epel_6> > If present, these pyvenv.cfg settings would override the normal PEP> 425 compatibility tag calculations (I'm OK with the idea of *needing*> to be in a virtual environment to gain the power to configure these> tags).

As long as it's on an opt-in basis, it certainly sounds ok.

Regards

Antoine.
_______________________________________________
Distutils-SIG maillist  -  Dist...@python.org
https://mail.python.org/mailman/listinfo/distutils-sig