Migrate overlapping intent attributes in closed and open list

[physikerwelt]

According to my import script, there are some entries that exists in both, open and closed list. We should merge the contents (as the open list is much better) and remove one of the entries.

Examples:

• floor closed open
• fractional-part
• golden-ratio
• identity-matrix

[physikerwelt]

As an alternative, we could abandon the core list and make an attribute that some of the concepts are more important than others.

[davidcarlisle]

@physikerwelt

as the open list is much better

Can I focus in on that and see if we can improve that situation...

Firstly, would I be right to assume that in your scripted use of this data you use the YAML files not the HTML display (so improving the latter would not help)?

Just looking at golden-ratio to keep things simple:

core has

      - concept: golden-ratio
        arity: 0
        en: "golden ratio"
        fr: "d"
        comments:
        - "Often denoted as φ"

open has

    - concept: golden-ratio
      arity: 0
      en: golden ratio
      property: constant
      area: "geometry"
      mathml:
       - "<math><mi intent='golden-ratio'>φ</mi></math>"
       - "<math><mi intent='golden-ratio'>ϕ</mi></math>"
      urls: 
       - "https://mathworld.wolfram.com/GoldenRatio.html"
       - "https://en.wikipedia.org/wiki/Golden_ratio"
       - "https://en.wikipedia.org/wiki/List_of_works_designed_with_the_golden_ratio"
       - "https://www.encyclopediaofmath.org/index.php/Golden_ratio"
      alias:
       - golden-number
       - golden-mean
       - golden-section
       - medial-section
       - divine-proportion
       - divine-section
       - golden-proportion
       - golden-cut

Which of the data do you miss?

We could immediately add the urls field to core.yml for example (whether or not we decide to expose the URLs in the html view)

alias is a bit more problematic in core (which was why I intentionally left it out when making the first version of the two lists from the initial google docs data). It would need some careful wording. In open it is documenting what systems do, so more documentation is never really a bad thing. However in core, it is is a specification of what systems should implement.

The intent system as currently specified doesn't have an alias mechanism so the core name is golden-ratio and other names are likely (but not necessarily) treated as unknown concepts. So it is OK for the open list alias column to effectively mean there is an open list entry for intent="golden-mean" that works like intent="golden-ratio" But we don't want to say that in core. the aliases still could be listed but described as something like "in other systems this concept may be known as .." However my initial thought was that open would have alias names allowing data from different systems to be combined, but when choosing to make a concept part of core, we'd choose a name and so the alias list wouldn't be needed.

[physikerwelt]

Firstly, would I be right to assume that in your scripted use of this data you use the YAML files not the HTML display (so improving the latter would not help)?

Yes. I'm using YAML.

We could immediately add the urls field to core.yml for example (whether or not we decide to expose the URLs in the html view)

That would be the game changer. If we had URLs a lot of information can be inferred. For example, from the Wikipdia Link we can derive the wikidata link and have a lot of machine-readable data. See https://www.wikidata.org/wiki/Q41690 This would simplify adding more language labels a lot.

I don't think we need alias

The MathML field is helpful as well. (If we had a list of tex macros that would be even better, but I think if the MathML is canoncal enough, we have good changes to guess the tex input.

[davidcarlisle]

@physikerwelt I don't think there is any reason we shouldn't add URL data to core.yaml originally they were embedded in comments but the open list has had them as a specific url field for some time, so it's easy to add them to core without initially affecting the display at all. If we get to a point where all entries have a url we could think about whether to display them

For the list of concepts you give at the top we can simply copy the data, for the rest of core we'd have to reconstruct it, but perhaps at least for wikipedia you already have a mapping?

[physikerwelt]

Yes, we could copy between the lists. However, I am wondering what the disadvantage the suggestion above

As an alternative, we could abandon the core list and make an attribute that some of the concepts are more important than others.

Even if it might be a bit more effort to generate the HTML pages, I guess it will be easier and less confusing to main in the long run.

[davidcarlisle]

I do not like the suggestion of merging the lists. open list is supposed to be open and essentially anyone can add to it. We don't have wikipedia ease of access where anyone can literally edit it, but that is the model: if someone asks for a new concept to be added we should add it modulo the lightest of syntax checks.

The core list is supposed to be a curated list where each added name is signed off by the working group.

The core list is a should requirement to implementors that those names should be be supported to some extent. That list should be relatively small, the open list has no implementation requirements and the entries are primarily for documentation and author advice on concept names to use.

[dginev]

@physikerwelt previous comments by senior group members indicated that the lifecycle of an Open list is out of scope for the WG and there would be no available maintainer bandwidth to support it. So I suspect there will either be multiple Open lists, or the one you see here will be hosted elsewhere.

[physikerwelt]

To summarize:
I started importing core to the MaRDI portal https://tinyurl.com/2bf2bh9r
(So editing in a wiki-style is easy to reach).
I used the concept name (e.g., golden-ratio) as the primary identifier in this import. Before continuing with open, I wanted to ensure that both lists interact in a defined way. Let me come up with some hypotheses:

1. Concept names are globally unique
2. The W3C math WG will maintain the core list. The details of the governance process are yet to be defined, but it would somehow follow the usual principles that this group has used in the past.
3. Concepts are final. (This implies that concepts in core can not overwrite be overwritten, or extended by open concepts)

Are those correct?

Given my experience with MathML content dictionaries (which are entirely different), I would like to reach a consensus on how to continue with the open list. My gut feeling is that our processes are quite efficient and that maintaining an open list based on pull requests (maybe supported by some guidelines) is a viable option.

[davidcarlisle]

To summarize: I started importing core to the MaRDI portal https://tinyurl.com/2bf2bh9r (So editing in a wiki-style is easy to reach). I used the concept name (e.g., golden-ratio) as the primary identifier in this import. Before continuing with open, I wanted to ensure that both lists interact in a defined way. Let me come up with some hypotheses:

1. Concept names are **globally** unique

We can not of course control what systems do but yes I think that you can assume that we wouldn't accept a PR adding a "competing" definition to the open list if the concept was already there.

2. The W3C math WG will maintain the core list. The details of the governance process are yet to be defined, but it would somehow follow the usual principles that this group has used in the past.

As you know WG work in timed charters so promising future govenance is a bit problematic for all W3C publications. However my experience is that even in periods when Math has not had a chartered Working Group (and has been at various times, A W3C "Interest Group" and "Community Group" or "no active group at all") Access to repositories has always been available. For example the unicode.xml file has been updated for every Unicode release since 1999, access to the git repository (or cvs repository before that) has always been available.

3. Concepts are final. (This implies that concepts in core can not overwrite be overwritten, or extended by open concepts)

Yes as above I don't think we should accept PR to the open list for names in core (and should accept your initial comment here that the fact that a few names are currently in both is a bug)

Are those correct?

More or less yes, as noted above.

Given my experience with MathML content dictionaries (which are entirely different), I would like to reach a consensus on how to continue with the open list. My gut feeling is that our processes are quite efficient and that maintaining an open list based on pull requests (maybe supported by some guidelines) is a viable option.

Maintaining an open list based on pull requests is exactly the current procedure so I don't think I understand your comment here (although I probably agree with it).

[dginev]

Maintaining an open list based on pull requests is exactly the current procedure

Exactly the current procedure, really? Was this list assembled via pull requests from scratch?

Here's the history of the open.yml file:
https://github.com/w3c/mathml-docs/commits/main/_data/open.yml

Where are the pull requests? Is this what you envision as "collaborative"? The file has a single contributor today.

[davidcarlisle]

Exactly the current procedure, really?

Yes.

Was this list assembled via pull requests from scratch?

Not it was assembed from google docs data (mostly from you) as you know, and the format modified to track whatever was the current syntax for intent over the discussions.

But the reason it's in github and in a standard format such as yaml and not part of any W3C TR publishing pipeline is explicitly because having it in a public git repository makes contributions to the open list easy and just requiring a PR (and light review by whoever has write permission to accept the PR)

No external contributions were really expected while the syntax of intent was under active development so it's not surprising that the only activity so far is initiated by the working group, and as a technical detail most (or for this file, all) git commits are in my name as role of editor whoever suggested the actual change that was being commited.

[dginev]

Exactly the current procedure, really?

Yes.

Maybe I am using the US English meaning of "current".

Good luck with getting those PR contributions, exciting "current" trend! I'm a little curious why that original seed contributor hasn't sent in any updates to the list recently, but it's probably nothing to worry about.

[davidcarlisle]

@dginev I can not understand what point you are trying to make, The list is in github and the text at the top of the file explicitly asks for contributions and points to the git source of the yaml file with the data.

That is implementing the working group plan of having an open list that is maintained in github and accepting updates via PR. That is the current procedure.

It is no surprise at at all that there have not been any external contributions to that list yet. The syntax of intent and more recently the exact format of the yaml and content and display of the open and core lists has been the subject of change due to working group meetings discussing this every week. Even if potential external contributors are working on lists that could be contributed in the end, there isn't a lot to be gained from them adding them at this stage, although if some were proposed I'm sure they'd be added.

In particular at some point I assume we'll add any concepts that mathcat knows about that don't end up in the core list, also by then perhaps @physikerwelt will have additional entries coming from wikidata, but until mathml4 is published and the initial version of the core list is finalised it's a bit premature to add to open other names that are not in core as "not in core" is a very variable property at present.

but it's probably nothing to worry about.

Exactly. It is completely as expected.

[dginev]

@dginev I can not understand what point you are trying to make

So much for my attempts at subtlety. Fine, here it is spelled out:

I do not think the github setup today is anywhere near sufficient for non-technical contributions to Intent Open. The current arrangement has 1 actual contributor (you), mostly massaging data that came from somewhere else. There is no current validation that collaborating in this way is feasible for anyone. I am also quite worried about your claimed comfort with the current state of things, as if this is a perfectly ordinary task which is just usual work done all the time. It isn't.

Back in 2022 I mentioned one possible start towards an interface to make such a list friendly for contributors. As I stated above, that will likely happen somewhere else, since I don't feel it is welcome here at all at present.

and accepting updates via PR. That is the current procedure.

Where are the Open list PRs? I don't even see one. That is not what "current" means. You may be "aspiring" to PRs, but that's a whole different story.

No external contributions were really expected

Apparently no internal contributions were expected either? external is a red herring here. No one is contributing to this list.

[davidcarlisle]

@dginev you are misunderstanding the meaning of "current procedure" it's necessary to specify procedures in advance of them being used. The current procedure is that contributions are via PR to github. There could be changes to that procedure, but that is the current one. This whole side discussion came about as @physikerwelt suggested that the procedure should be via github PR and I commented that that was the current one. I never claimed that there had been any external submission so far. It was just a minor comment not something to derail the main point of this issue which (correctly) points out that some names are in both lists, and we should fix that.