| Store | Cart

Re: [Distutils] PEP for dependencies on libraries like BLAS (was: Re: Working toward Linux wheel support)

From: Wes Turner <wes....@gmail.com>
Fri, 21 Aug 2015 14:27:42 -0500
On Fri, Aug 21, 2015 at 1:30 PM, Wes Turner <wes....@gmail.com> wrote:

>> On Aug 21, 2015 12:41 PM, "Brett Cannon" <bre...@python.org> wrote:> >> >> >> > On Thu, 20 Aug 2015 at 10:16 Wes Turner <wes....@gmail.com> wrote:> >>> >>> >> On Aug 20, 2015 5:05 AM, "Nick Coghlan" <ncog...@gmail.com> wrote:> >> >> >> > [Catching up on distutils-sig after travel]> >> >> >> > On 13 August 2015 at 16:08, Nathaniel Smith <n...@pobox.com> wrote:> >> > > It seems like a reasonable effort at solving this problem, and I> guess> >> > > there are probably some people somewhere that have this problem, but> >> > > my concern is that I don't actually know any of those people. The> >> > > developers I know instead have the problem of, they want to be able> to> >> > > provide a small finite number of binaries (ideally six binaries per> >> > > Python version: {32 bit, 64 bit} * {windows, osx, linux}) that> >> > > together will Just Work on 99% of end-user systems. And that's the> >> > > problem that Enthought, Continuum, etc., have been solving for> years,> >> > > and which wheels already mostly solve on windows and osx, so it> seems> >> > > like a reasonable goal to aim for. But I don't see how this PEP gets> >> > > us any closer to that.> >> >> >> > The key benefit from my perspective is that tools like pyp2rpm, conda> >> > skeleton, the Debian Python packaging tools, etc, will be able to> >> > automatically generate full dependency sets automatically from> >> > upstream Python metadata.> >> >> >> > At the moment that's manual work which needs to be handled> >> > independently for each binary ecosystem, but there's no reason it has> >> > to be that way - we can do a better job of defining the source> >> > dependencies, and then hook into release-monitoring.org to> >> > automatically rebuild the downstream binaries (including adding new> >> > external dependencies if needed) whenever new upstream releases are> >> > published.> >>> >> JSON (JSON-LD) would likely be most platform compatible (and designed> for interoperable graph nodes and edges with attributes).> >>> >> JSON-LD does not require a specific library iff the @context is not> necessary.> >>> >> Notes about JSON-LD and interoperable software package metadata:> https://mail.python.org/pipermail/distutils-sig/2015-April/026108.html> >> >  What does JSON-LD have to do with this conversation, Wes? No discussion> of implementation has even begun, let alone worrying about data formats.> This is purely a discussion of problem space and what needs to be solved> and is not grounded in concrete design yet.>> Really?> Why would a language designed for graphs be appropriate for expressing> graphs and constraints?>> The problem is: setuptools packages cannot declare dependency edges to> things that are not Python packages; and, basically portage ebuild USE> flags for attributes.>> Now, as ever, would be a good time to learn to write a JSONLD @context> (for the existing fragmented packaging system standards (that often require> code execution to read/generate JSON metadata (because this is decidable))).>

I guess what I'm trying to say is:

* "why is this packaging metadata split?"
* shouldn't this all be in setup.py

  * couldn't we generate a proper RDF graph
    from each of the latest JSONLD serializations
    (e.g. https://pypi.python.org/pypi/<pkg>/jsonld)

    * what is missing?
       * install_requires_pkgs_apt
         install_requires_pkgs = { "apt": [...], "yum": [...], "dnf":
[...], "aur": [....] }
       * extras_require_pkgs = {
            "extraname": {  { "apt": [...], "yum": [...], "dnf": [...],
"aur": [....] },
            "otherextraname": { "apt": [...], "yum": [...], "dnf": [...],
"aur": [....] }
       * build flag schema
         buildflags_schema={"numpy17":"bool"}
         buildflags=[{"numpy17"}]

Because, from a maintainer/reproducibility standpoint,
how do I know that all of these packages (1) do (2) could exist?

* (3) SHOULD exist: tox, Docker builds


So, is JSON-LD necessary to add extra attributes to setup.py? Nope.

Would it be a good time to norm with a JSON-LD @context and determine how
to specify package groupings like [salt grains] without salt, in
setuptools?   Or just be declarative (e.g. with Portage and Conda (and
Pip/Peep, inevitably)).

_______________________________________________
Distutils-SIG maillist  -  Dist...@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Recent Messages in this Thread
Nathaniel Smith Aug 13, 2015 12:51 am
James Polley Aug 13, 2015 01:02 am
Olivier Grisel Aug 13, 2015 03:06 am
Robert Collins Aug 13, 2015 03:10 am
Nathaniel Smith Aug 13, 2015 06:08 am
David Cournapeau Aug 14, 2015 09:59 am
Reinout van Rees Aug 17, 2015 02:07 pm
Donald Stufft Aug 17, 2015 02:15 pm
Reinout van Rees Aug 17, 2015 08:56 pm
Nick Coghlan Aug 20, 2015 10:05 am
Wes Turner Aug 20, 2015 05:15 pm
Brett Cannon Aug 21, 2015 05:41 pm
Wes Turner Aug 21, 2015 06:30 pm
Wes Turner Aug 21, 2015 07:27 pm
Wes Turner Aug 21, 2015 07:34 pm
Wes Turner Aug 21, 2015 08:28 pm
Nick Coghlan Aug 22, 2015 10:33 am
Antoine Pitrou Aug 22, 2015 07:22 pm
Messages in this thread