From 74ca5dafb93d492805286460fe7ca11737e90248 Mon Sep 17 00:00:00 2001 From: Lukas Kahwe Smith Date: Fri, 30 Oct 2026 19:59:59 +0100 Subject: [PATCH 1/4] doc(spec): follow redirects when detecting technical forks A repository renamed or transferred on its code hosting platform keeps answering at the old path through a redirect. Comparing url without following it makes such a project look like a technical fork of itself. --- docs/standard/forks.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/standard/forks.rst b/docs/standard/forks.rst index d6a2ade..56a7ed2 100644 --- a/docs/standard/forks.rst +++ b/docs/standard/forks.rst @@ -31,7 +31,9 @@ Parsers Parsers **SHOULD** identify a technical fork by noticing that the top-level ``url`` key does not point to the repository in which the -``publiccode.yml`` is found. +``publiccode.yml`` is found. Parsers **SHOULD** follow redirects before +comparing, since a repository that was renamed or transferred on its +code hosting platform is still the same repository. Parsers **MIGHT** identify a technical fork also through metadata that might be exposed by the code hosting platform (e.g. GitHub marks forks From 7137106320598698367cfcb01572ae9e21de9b86 Mon Sep 17 00:00:00 2001 From: Lukas Kahwe Smith Date: Fri, 30 Oct 2026 19:59:59 +0100 Subject: [PATCH 2/4] doc(spec): add the mirrors key Lets a project declare the repository locations it endorses, so parsers can tell its own mirrors from technical forks. The key is read only from the publiccode.yml at url, and a listed repository counts only if its own publiccode.yml points back, so neither a clone nor the listing project can claim a location on its own. Refs #293 --- docs/standard/forks.rst | 75 +++++++++++++++++++++++++++++++++-- docs/standard/schema.core.rst | 35 ++++++++++++++++ 2 files changed, 107 insertions(+), 3 deletions(-) diff --git a/docs/standard/forks.rst b/docs/standard/forks.rst index 56a7ed2..9b5cb12 100644 --- a/docs/standard/forks.rst +++ b/docs/standard/forks.rst @@ -31,9 +31,10 @@ Parsers Parsers **SHOULD** identify a technical fork by noticing that the top-level ``url`` key does not point to the repository in which the -``publiccode.yml`` is found. Parsers **SHOULD** follow redirects before -comparing, since a repository that was renamed or transferred on its -code hosting platform is still the same repository. +``publiccode.yml`` is found, unless the repository is a mirror (see +`Mirrors`_). Parsers **SHOULD** follow redirects +before comparing, since a repository that was renamed or transferred on +its code hosting platform is still the same repository. Parsers **MIGHT** identify a technical fork also through metadata that might be exposed by the code hosting platform (e.g. GitHub marks forks @@ -110,3 +111,71 @@ Moreover, authors **SHOULD** evaluate the following changes: ``description/features`` key. Existing features **SHOULD NOT** be edited or removed from this list unless they have been removed from the variant, to allow parsers to easily compare feature lists. + +Mirrors +------- + +A mirror is a copy of the repository that the authors of the software +endorse. It contains the same ``publiccode.yml``, so its top-level +``url`` points to another repository, like a technical fork. + +.. _parsers-2: + +Parsers +~~~~~~~ + +Parsers **SHOULD** compare repository URLs after normalising them, for +example ignoring a trailing slash, a ``.git`` suffix, or differences in +case where the code hosting platform ignores them. + +When the top-level ``url`` does not point to the repository in which the +``publiccode.yml`` is found, parsers **SHOULD** fetch the +``publiccode.yml`` at ``url``. If that file lists the repository under +``mirrors``, parsers **SHOULD** treat it as the same software rather than +as a technical fork. Parsers **MUST NOT** rely on ``mirrors`` found in +any other copy of the file. + +A repository listed under ``mirrors`` is a mirror only if the +``publiccode.yml`` found there has its top-level ``url`` pointing to the +repository that lists it. Otherwise parsers **MUST NOT** treat the two +as the same software. + +A mirror marked ``deprecated`` is the same software only if a parser +confirmed it under the rule above at some point while it was reachable, +whether before or after it was marked. Parsers **SHOULD** keep that +confirmation once the mirror disappears, so that references to it still +resolve, and **SHOULD** warn when a reference resolves through it. +Parsers **SHOULD NOT** offer what is found at a deprecated mirror as the +software's code. + +A mirror whose ``publiccode.yml`` does not point back yet may still be +being set up, or may lag behind. Parsers **SHOULD** keep existing entries +unchanged meanwhile and check again on later crawls. How long to wait +before reporting a conflict, and when to intervene by hand, is up to each +catalog. + +If fetching either ``publiccode.yml`` fails, parsers **SHOULD** keep the +last known result and check again later, rather than treating the +failure as a change. A repository never confirmed as a mirror is treated +as a technical fork, as today. + +Checking this key means fetching URLs taken from a ``publiccode.yml``. +Parsers **SHOULD** apply the same restrictions as for any other URL they +fetch, such as HTTPS only, no private network addresses and a limited +number of redirects, and **MAY** check only the first entries. + +.. _authors-2: + +Authors +~~~~~~~ + +Authors **MAY** list the mirrors they endorse under ``mirrors``, most +preferred first and deprecated entries last, since parsers may check +only the first entries. They **MUST** remove a mirror whose location is +no longer under their control, or whose operator they no longer trust, +because whoever controls that location could otherwise present it as an +endorsed mirror. + +If development happens on an instance that parsers cannot reach, ``url`` +points to the public repository, and the internal instance is not listed +under ``mirrors``. diff --git a/docs/standard/schema.core.rst b/docs/standard/schema.core.rst index b74e849..f63592f 100644 --- a/docs/standard/schema.core.rst +++ b/docs/standard/schema.core.rst @@ -97,6 +97,41 @@ variant, descending from the specified repositories. The single string form is deprecated, use a one element array instead. +Key ``mirrors`` +~~~~~~~~~~~~~~~ + +- Type: array of objects +- Presence: optional + +A list of alternative repository locations endorsed by the authors of +the software, most preferred first and deprecated entries last. A +mirror holds the same software as +``url``; it is not a fork or a variant. + +Each item is an object with the following properties: + +- ``url`` - **mandatory** - a URL to the source code repository of the + mirror, in the same format as the top-level ``url`` key. +- ``status`` - either ``active`` (default) or ``deprecated``. + A mirror that is no longer maintained SHOULD be marked ``deprecated`` + rather than removed, so that parsers can still resolve references to + it and warn about them. A mirror whose location the authors no + longer control, for example after a domain has lapsed, or whose + operator they no longer trust, MUST be removed instead. + +Parsers MUST only take this key into account when it is read from the +``publiccode.yml`` found at the top-level ``url``. See +:ref:`forks-and-variants`. + +Example: + +.. code:: yaml + + mirrors: + - url: https://mirror.example.com/italia/medusa.git + - url: https://old-mirror.example.com/italia/medusa.git + status: deprecated + Key ``softwareVersion`` ~~~~~~~~~~~~~~~~~~~~~~~ From 193abeb345bf74b0ce9ed412696981943d892e2d Mon Sep 17 00:00:00 2001 From: Lukas Kahwe Smith Date: Fri, 30 Oct 2026 20:31:37 +0100 Subject: [PATCH 3/4] doc(spec): add the previousURLs key Lets a project list the earlier values of url, so catalogs can resolve references made against an old location and merge the old entry into the current one. Only the old location can confirm a move, through its own url or a redirect, and the whole chain of moves must hold. Listing a location under mirrors is not consent to a move. Refs #293 --- docs/standard/forks.rst | 71 +++++++++++++++++++++++++++++++++++ docs/standard/schema.core.rst | 16 ++++++++ 2 files changed, 87 insertions(+) diff --git a/docs/standard/forks.rst b/docs/standard/forks.rst index 9b5cb12..6e718a8 100644 --- a/docs/standard/forks.rst +++ b/docs/standard/forks.rst @@ -179,3 +179,74 @@ endorsed mirror. If development happens on an instance that parsers cannot reach, ``url`` points to the public repository, and the internal instance is not listed under ``mirrors``. + +Repository moves +---------------- + +When the software moves to a new repository location, ``url`` changes +and the earlier locations are listed under ``previousURLs``. + +.. _parsers-3: + +Parsers +~~~~~~~ + +Parsers **MUST NOT** rely on ``previousURLs`` found in any copy of the +file other than the one at ``url``. + +Confirmation flows forward from the current location. Each earlier +location confirms only the single move it made: to the location that +follows it in ``previousURLs``, or to the current ``url`` for the last +entry. It confirms that move when its last known ``publiccode.yml`` has +that location as its top-level ``url``, or when, instead of serving its +own ``publiccode.yml``, it redirects there or a parser has recorded it +doing so. Listing a location under ``mirrors`` does not confirm a move. +An entry counts as an earlier location of the software only if it and +every entry after it have confirmed their moves. + +Parsers **SHOULD** record an observed redirect, since the code hosting +platform may later drop it, for example when someone creates a new +repository under the old name. Parsers **MAY** treat a recorded redirect +as a confirmed move even when the new ``publiccode.yml`` does not list +the earlier location under ``previousURLs``. + +Parsers **MUST NOT** merge an earlier location with the current entry +unless it counts under these rules or a recorded redirect confirms the +move. Once an earlier location has +confirmed its move, later changes at that location do not affect it. +Only the location the software moved to can confirm a further move, +including a move back. + +A claim that is not confirmed yet can be a move in progress, which may +take days. Parsers **SHOULD** keep existing entries unchanged meanwhile +and check again on later crawls. As for mirrors, how long to wait before +reporting a conflict, and when to intervene by hand, is up to each +catalog, and the same fetch restrictions apply. + +.. _authors-3: + +Authors +~~~~~~~ + +Authors moving the software to a new repository location **SHOULD**: + +- Before the move, add the new location under ``mirrors``, so that + parsers recognise it as a mirror during the move. +- After the move, set ``url`` to the new location, add the old + location to ``previousURLs`` and remove the new location from + ``mirrors``. If the old location stays online as a mirror, also list + it under ``mirrors``. +- At the same time, set the ``url`` in the ``publiccode.yml`` at the old + location to the new location, before archiving or deleting it. + Without this, parsers cannot confirm the move. A mirror that copies + the new repository does this by itself. +- Keep every earlier location in ``previousURLs``, oldest first, since + each one is confirmed against the one that follows it. + +After renaming or transferring the repository on its code hosting +platform, authors **SHOULD** update ``url`` and ``previousURLs`` +promptly, while the platform still redirects the old location. + +A fork that continues the software without the agreement of its authors +is not a move. It takes its own ``url`` and declares the original under +``isBasedOn``, as a software variant. diff --git a/docs/standard/schema.core.rst b/docs/standard/schema.core.rst index f63592f..da043b6 100644 --- a/docs/standard/schema.core.rst +++ b/docs/standard/schema.core.rst @@ -132,6 +132,22 @@ Example: - url: https://old-mirror.example.com/italia/medusa.git status: deprecated +Key ``previousURLs`` +~~~~~~~~~~~~~~~~~~~~ + +- Type: array of strings (URL) +- Presence: optional +- Example: ``["https://old-forge.example.com/italia/medusa.git"]`` + +Earlier values of the top-level ``url`` key, oldest first, for software +that has moved to a different repository location. Parsers use it to +resolve references made against an earlier location to the current +entry. Entries MUST NOT repeat the top-level ``url`` or each other. + +Parsers MUST only take this key into account when it is read from the +``publiccode.yml`` found at the top-level ``url``. See +:ref:`forks-and-variants` for how parsers verify this key. + Key ``softwareVersion`` ~~~~~~~~~~~~~~~~~~~~~~~ From f12068ae02d7bc0e13ae9cf3fc079b0bac9554ce Mon Sep 17 00:00:00 2001 From: Lukas Kahwe Smith Date: Fri, 30 Oct 2026 20:31:38 +0100 Subject: [PATCH 4/4] doc(spec): keep old locations reachable until catalogs have crawled them Because the whole chain of moves must be confirmed, a location shut down before catalogs have seen its forward pointer breaks the confirmation of every earlier location as well. --- docs/standard/forks.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/standard/forks.rst b/docs/standard/forks.rst index 6e718a8..338f834 100644 --- a/docs/standard/forks.rst +++ b/docs/standard/forks.rst @@ -242,6 +242,11 @@ Authors moving the software to a new repository location **SHOULD**: the new repository does this by itself. - Keep every earlier location in ``previousURLs``, oldest first, since each one is confirmed against the one that follows it. +- Keep the old location reachable, with its ``url`` pointing to the + new location or redirecting there, until the catalogs that matter + to the project have crawled it. In a chain of moves, a location + that disappears before it is crawled also breaks the confirmation + of every earlier location. After renaming or transferring the repository on its code hosting platform, authors **SHOULD** update ``url`` and ``previousURLs``