PEP 825: Add "removing ordering information" to rejected ideas

[mgorny]

No description provided.

[read-the-docs-community]

Documentation build overview

📚 wheelnext-peps | 🛠️ Build #33254091 | 📁 Comparing a31921b against latest (0cde95d)

  🔍 Preview build  

740 files changed · ± 740 modified

± Modified

• index.html
• numerical/index.html
• pep-0000/index.html
• pep-0001/index.html
• pep-0002/index.html
• pep-0003/index.html
• pep-0004/index.html
• pep-0005/index.html
• pep-0006/index.html
• pep-0007/index.html
• and 730 more...

[rgommers]

Thanks @mgorny! This is pretty good, but it can be strengthened a bit more. I made a couple of inline suggestions. In addition, I think Paul's key point was slightly broader, it was conceptual rather than practical. I'd link https://discuss.python.org/t/pep-825-wheel-variants-package-format-split-from-pep-817/106196/111 explicitly. And then in the reply, also address the conceptual part:

Essentially Paul is saying that this data, when inserted in one wheel, is saying something about "other wheels" (even these quotes are in his post). I don't think that's the case. Rather, I'd frame it as: ordering is project-level metadata, and a copy of that is made into every wheel. Similar to other project-level metadata. There is no reference to specific other wheels. And even if, for example, variant wheels covering one provider are missing from a release completely, that is fine, nothing depends on specific other wheels existing.

So, I'd frame it in this order:

1. Conceptual: this is project-level metadata, and one can think of variants.json as a cache
2. Practical: the two advantages you listed.

[rgommers]

I'd say this point that the JSON file is a cache rather than first-order data source is good to put higher up, it's a separate point to make about the design that I think is fairly strong.

[rgommers]

How about "without requiring publishers to upload an additional file" to be more explicit? The average reader might not know exactly what "client-side logic" means here.

[rgommers]

Given how important this objection was/is, it might even be useful to extend it a bit more, addressing two questions directly:

"What about inconsistency between wheels built at different times with different tooling?"

I'd say the answer is something like: "This PEP requires consistency, tools SHOULD verify it, and the realistic failure mode is building different variants with different versions of build tools. That is a bad idea that can also affect every other piece of duplicated metadata (e.g., ending up with different Core Metadata versions if you do that).

Also, I'd quote this, also from post 111: "it becomes possible for installers to support overrides by simply having a --variant-ordering flag that takes a file of ordering data in the standard format."

I'd say that that doesn't depend on removing this metadata from individual wheels, it can be done either way.

[mgorny]

Conceptual: this is project-level metadata, and one can think of variants.json as a cache

I don't think "cache" is a right word. "Cache" implies it's only an optimization, and you can refetch the data. However, here you can't refetch it directly because there's no definite "source of data" you can access. I can't think of the correct term, though.

Also, I'd quote this, also from post 111: "it becomes possible for installers to support overrides by simply having a --variant-ordering flag that takes a file of ordering data in the standard format."

I'd say that that doesn't depend on removing this metadata from individual wheels, it can be done either way.

That's 💯 true, though it sounds a bit weird to be mentioning this in this context, since Paul's assumption that you cannot do that with the existing design was simply mistaken.

That said, rereading his post; perhaps what we could do is clarify the scope of individual parts of wheel variant metadata; i.e. that the variants dictionary in the wheel is scoped to this wheel, and everything else is scoped to the "group of wheels".

[rgommers]

That said, rereading his post; perhaps what we could do is clarify the scope of individual parts of wheel variant metadata; i.e. that the variants dictionary in the wheel is scoped to this wheel, and everything else is scoped to the "group of wheels".

That sounds like a useful edit to make, perhaps it helps more readers. I also like that we're thinking about how to improve the PEP itself; even if we think a comment is mistaken, it might be caused by an explanation not as easily digestible as we think it is.

[rgommers]

"Cache" implies it's only an optimization, and you can refetch the data. However, here you can't refetch it directly because there's no definite "source of data" you can access

It depends on who "you" is I guess. The source of truth is pyproject.toml, that gets translated into metadata in wheels, and then (possibly via the index server synthesizing that data) exposed as variants.json and as the new json metadata file inside wheels. Exposing it in two places is an optimization.

Not sure if "cache" is the ideal word either, but we should be able to explain it in words or in a little ascii diagram I'd think?

[mgorny]

The source of truth is pyproject.toml, that gets translated into metadata in wheels

Well, yes and no. It might be pyproject.toml but that's to be defined in a subsequent PEP, and I don't think this is actually going to be technically required. Either way, if you have a bunch of wheels on PyPI, you don't have a guaranteed access to the file either way.

[mgorny]

the realistic failure mode is building different variants with different versions of build tools

I'm not sure if I understand this right. Given that the input would be in pyproject.toml and the tools would be expected to copy it 1:1, why would two versions of build tools produce different variant metadata?

[mgorny]

Okay, I've pushed my first attempt at rewording it per your feedback. I've left the "scope" problem out, to be addressed separately.