Possible solution for train date and version

[RivenSkaye]

This comment mentions parsing the static manifests page for acquiring an exact release date. While feasible, that resource seems to always be a week behind compared to the actual releases.

Parsing it is still possible though, and I do have something that mostly works hidden beneath

There are two dates that are relatively easy to acquire from there

1. The date assigned to the channel release, in ISO-8601 format (YYYY-MM-DD, currently 2023-06-26)
2. The date inside the manifest TOML, which lists when the last contributing commit was made as well as a version number (`version = "0.72.0-beta.4 (cfd3bbd8f 2023-06-08)"`, which seems to follow some semver logic wrt beta numbering being 0.x.y)

Assuming 1, it's as simple as

tmpvar=$(curl -s https://static.rust-lang.org/manifests.txt | grep -i beta | tail -1)
beta_date=${tmpvar:26:10}

2 becomes slightly more complex, since it requires grabbing whatever the last beta release is

tomlurl=$(curl -s https://static.rust-lang.org/manifests.txt | grep -i beta | tail -1)
versionline=$(curl -s $tomlurl | grep -i version | sed -n '2p')  # I tried grabbing both in one line, but something's tripping up curl
commit_date=$([[ $versionline =~ ([0-9][0-9]+\-[0-9]+\-[0-9+][^\)]) ]] ; echo ${BASH_REMATCH[1]})  # 2023-06-08
version=$([[ $versionline =~ ([0-9]+\.[0-9]+\.[0-9]+\-beta\.[0-9]+) ]] ; echo ${BASH_REMATCH[1]})  # 0.72.0-beta.4

Of course, whatever components aren't necessary can be stripped off before use, e.g.

relver=${version:2}
cat <<EOS > beta
[toolchain]
channel = "1.${relver%.*}"
EOS

That said, the current beta version reports as being 72.0-beta.4 (cfd3bbd8f 2023-06-08), but upon installing it rustup reports rustc 1.71.0-beta.4 (dbf31f17d 2023-06-24). The static list also doesn't list any nightly builds after 2023-07-02, which seems to imply

• it's always a full week behind
• beta version numbers are incremented sooner than stable (maybe @ breakage freeze?)
• beta versions start with a different major version identifier

So perhaps it's best to abandon this idea? Does make me wonder where rustup gets its manifest information from though

[RivenSkaye]

Gave it some rest over the weekend and probed around a bit to see if more recent data than a week old was available easily, and I did manage to find something. Namely by inspecting static.rust-lang/org/dist/{date}/ and seeing where it gets the data from. Successfully managed to grab today's beta release info and for an extra test I ran a search starting yesterday, which also produced a valid beta release (beta-5).

The versioning provided in the TOML doesn't change over it, but at least the most recent release date & commit info can be extracted. I did update version extraction to provide a static 1. prefix and a decremented beta number to match what rustup outputs.

relday=$(date -I)
while [[ $(curl -s "https://static.rust-lang.org/?prefix=dist/$relday&marker=" | grep -c 'beta') == '0' ]] ; do
    relday=$(date -I -d "$relday -1 day")
done
versionline=$(curl -s "https://static.rust-lang.org/dist/$relday/channel-rust-beta.toml" | grep -i version | sed -n '2p')
commit_date=$([[ $versionline =~ ([0-9][0-9]+\-[0-9]+\-[0-9+][^\)]) ]] ; echo ${BASH_REMATCH[1]})  # 2023-06-08
version=$([[ $versionline =~ ([0-9]+\.[0-9]+\.[0-9]+\-beta\.[0-9]+) ]] ; echo "1.$((${BASH_REMATCH[1]:2:2} - 1))${BASH_REMATCH[1]:4}")  # 1.71.0-beta.6

Matches the info provided by rustup run beta rustc --version, sans commit hash. The same thing should also work for nightly, except it doesn't require looping since they're built daily. If it needs to account for nightly not being built (yet) for some reason, it could be a reusable function. Just substitute grep 'channel\-rust\-beta\.toml' with grep "channel\-rust\-$1\.toml" and echo "$version ($commit_date)".
If this approach seems desirable, feel free to add & close or tell me to PR it myself

edit: removed a linecount; grep -c works fine

[RivenSkaye]

Final update from me. I should've gone through the entire file a bit more thoroughly as I stumbled across a pkg.rustc header on line 21100 (in today's nightly toml) providing the correct version, which is surprisingly easy to extract a proper version info from! No mangling or anything, just a simple grep. I've tested the provided code to work for all three release channels. In order to prevent (worst case) 42 probe attempts for finding stable releases, there's some special handling for that channel.

This also voids current assumptions that the version starts with a 1, although that's already possible using a grep -oP pattern similar to the ones provided in the code below. That would work on all three rustup run $toolchain rustc --version invocations. though stable would need a special pattern to ignore the space and everything after.

The biggest potential limiting factor here is support for Perl patterns (the -P flag existing at all), which isn't an issue on the ubuntu-latest image, since that comes with a recent version of GNU grep.

rustc_crawl() {
    ptrn='\[pkg\.rust\]\nversion = "\K[^"]+'
    relday=$(date -I)  # Today's date
    jump='-1 day'
    # Rust stable updates happen every 6 weeks, starting at 2015-12-10
    if [[ "$1" = 'stable' ]] ; then
        jump='+6 weeks'  # Different jumps for stable, don't hit the static lists too much.
        ptrn='\[pkg\.rust\]\nversion = "\K[^\s]+'  # Don't grab commit info for stable
        # Any arbitrary confirmed release date in the past works. 1.70.0 was still on 6 week jumps since 1.5.0
        # but cutting down on the loops is not a bad idea. Perhaps store the last release date in stable and jump once?
        start='2023-06-01'
        while [[ $(date -d "$start" '+%s') -lt $(date -d "$relday" '+%s') ]] ; do
            start=$(date -I -d "$start $jump")
        done
        if [[ $(date -d "$start" '+%s') -ne $(date -d "$relday" '+%s') ]] ; then  # If the next release is on a future date...
            grep -oP '= "\K[^"]+' stable  # Print out the last release version from stable...
            return 0  # and halt further function execution
        fi
    fi
    # stable and nightly releases should be found on the first check here, making it equivalent to an if,
    # but beta releases are a pain in the behind to track down, so we probe a few days for those
    while [[ $(curl -s "https://static.rust-lang.org/?prefix=dist/$relday" | grep -c "$1") == '0' ]] ; do
        relday=$(date -I -d "$relday $jump")
    done
    # Extract whatever version information upstream provides!
    curl -s "https://static.rust-lang.org/dist/$relday/channel-rust-$1.toml" | grep -ozP $ptrn | tr -d '\0'
}

[RivenSkaye]

@badboy Upstream now has numbered betas, which would make half the work done here useless. I could either edit the existing PR to incorporate it, or open a new one on a fresh fork. What would be your preference?

[badboy]

New PR sounds good to me.