Skip to content

Headless replay player - #1950

Open
MichalLabuda wants to merge 15 commits into
Return-To-The-Roots:masterfrom
MichalLabuda:headless-replay-player
Open

MichalLabuda wants to merge 15 commits into
Return-To-The-Roots:masterfrom
MichalLabuda:headless-replay-player

Conversation

@MichalLabuda

@MichalLabuda MichalLabuda commented Jun 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds replay-player, a headless tool that plays back a .rpl replay without a GUI, shows live
progress and checks the replay for desyncs.

The playback is a reusable HeadlessReplay class in s25Main, so other tools can build on it.
Replays and maps are loaded through one code path, SetupGameWorld, which the game client,
HeadlessReplay and ai-battle share.

replay-player

replay-player [--replay] <file.rpl> [--verbose]
replay-player --help | --version
  • Prints the replay info (map, version, type, map size, seed, GF range, Lua, fish fix) and the
    player roster, then a status table that updates every second: GF / total GFs, game clock,
    wall clock, GF/s, and Country / Buildings / Military / Gold per player.
  • Plays back up to and including the replay's last GF, as the game does, and stops at the first
    desync.
  • Exit code 0 if no desync was found, 1 on a desync or a load error. On a desync it prints the GF
    and both checksums; --verbose also dumps the async log of the RNG.
  • The replay path accepts placeholders such as <RTTR_USERDATA>. Lua output goes to the log file
    in the user data folder instead of stdout.

HeadlessReplay

HeadlessReplay replay(path);   // throws std::runtime_error if the replay cannot be loaded
replay.Run([](const HeadlessReplay& r) { /* called after every GF */ });
if(const auto& desync = replay.getDesync())
    ...                        // GF and expected/actual checksums of the first desync
  • RunGF() plays back one frame at a time. getGame(), getWorld() and getReplay() give read
    access to the state, e.g. for extracting training data from replays.
  • Playback stops at the first desync, before running that frame, so the world and the RNG async
    log still show the state that produced it. Every frame after a desync would desync too.
  • A corrupt replay whose commands are out of GF order is rejected with an exception.

One code path for setting up a game

SetupGameWorld (libs/s25main/GameSetup.h) sets up the world of a new game for
GameClient::StartGame, HeadlessReplay and ai-battle. It covers snapshot or map loading, Lua,
start pacts, the fish fix, the replay compat version and the leather-distribution fixup for old
replays (moved from GameClient::StartReplay). Map and Lua script can also be taken from the data
embedded in a replay. The fish-fix version rule lives in Replay::usesFishFix().

Other changes

  • In the game's replay mode, ReplayInfo keeps the GF of the first desync instead of counting
    async frames, and the notice at the end of the replay reports it.
  • extras/headlessConsole: the console output shared by ai-battle and replay-player
    (printConsole, clock and number formatting, the status table).
  • NullLocalGameState (ILocalGameState.h) replaces three identical local copies in ai-battle,
    replay-player and the autoplay test.
  • ai-battle sets up its world with SetupGameWorld instead of its own map, Lua and start-pact code.
  • ai-battle creates its players as PlayerState::AI instead of Occupied, which marks a human.
    Before, Lua's AIConstructionOrder refused orders for them (e.g. LuaFunctions.lua failed its
    assertion), IsHuman() was true, pact requests reached them as post messages, and the AUTOFLAGS
    addon placed flags on their roads. Replays recorded by ai-battle now also store the AI info.
    This dates back to when ai-battle was added.
  • The autoplay test plays its replays through HeadlessReplay and gains a test case for a replay
    with out-of-order commands.

Visible changes

  • The notice at the end of a replay in the game changes from "Notice: Overall asynchronous frame
    count: %u" to "Notice: The replay was out of sync from GF %u on." The translations will be
    updated in a follow-up PR to the languages repository.
  • ai-battle's status table has the wider replay-player layout.

Testing

  • Every commit since the master merge builds on its own (Windows, MSVC Debug, all targets
    including tests and extras) and passes the clang-format 10 check.
  • Test_autoplay passes: the 200k GF replay started from a savegame, the 300k GF sea replay
    started from a map, and the new out-of-order case.
  • Linux (Ubuntu 22.04, gcc 11 with the CI's Debug settings including -Werror): the build, all
    tests, Test_autoplay, clang-tidy 18 on the changed files, and local runs of the CI's static
    analysis and test coverage checks pass.
  • Windows: all other tests pass, apart from two libsiedler2 cases (LoadBobFile,
    ReadOriginalFiles) that fail only locally because the S2 game data sits in the build folder.
    They are unrelated to this PR.
  • ai-battle before and after switching to SetupGameWorld: same map, seed and GF count
    (plain, teams, Lua) gave identical replays and savegames, apart from the version and
    timestamp in the header.

@MichalLabuda

Copy link
Copy Markdown
Contributor Author

Replay-Player is very similar to AI-Battle and theoretically could be implemented as an additional mode to it instead of completely separate app. I've decided to implement it like this but I'm not entirely convinced that this is the way to go.

@Flamefire Flamefire left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but I'm wondering if we can unify loading the replay. Especially with the fish fix it becomes a maintenance burden. Can you investigate this?

And it doesn't make sense to count asyncs: Every NWF after an async will be async too. So you can stop right there. Or at least report it only once if you see any use for letting it continue.

@wichern

wichern commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Great idea having a headless replay! I have been wanting this feature myself.

My suggestion would be to make HeadlessReplay a class instead of just a collection of types and helper functions.
Then it could be used for other tools as well.

The binary in this PR checks replays. I would like to be able to create another binary, that uses the same class and extracts training data from replays.

How about something like this:

class HeadlessReplay
{
public:
    HeadlessReplay(const boost::filesystem::path& replay);
    bool Run();
};

@MichalLabuda
MichalLabuda marked this pull request as draft September 9, 2026 17:33
Every frame after the first desync desyncs too, so the count told
nothing beyond that the replay went out of sync. ReplayInfo now keeps
the GF the first desync was detected at, and the notice at the end of
the replay names it.
ai-battle, replay-player and the autoplay test each defined the same
ILocalGameState that acts as the host playing as player 0 and discards
all output.
Moves the savegame/map branch of GameClient::StartGame into a function
the headless tools can share: snapshot or map loading, Lua, start pacts,
fish fix and replay compat version, plus the fixup of the leather addon
distribution for old replays from StartReplay. When the MapInfo has no
file path, the map and Lua script are taken from the data embedded in
it. Failures are reported as GameSetupError.

Add Replay::usesFishFix() so the version rule lives in one place.
printConsole, the clock and number formatting and the status table move
into a library the replay-player can use too. The table takes the
replay-player's layout, which can also show the total GF count.
The replay-player had its own copy of the replay loading and playback
loop. HeadlessReplay loads a replay through SetupGameWorld, as
GameClient does, and plays it back frame by frame with RunGF() or
entirely with Run(callback). It stops at the first desync before running
that frame, so the world and the async log of the RNG still show the
state that produced it.

The replay-player is built on it and stops at the first desync instead
of counting them. It prints through headlessConsole, the rest of its
output moves to ReplayOutput.
The test had its own copy of the playback loop, which also required the
replay started from a map to predate the fish fix. A failure now names
the GF of the desync.
Replaces the hand-rolled map, Lua and start-pact setup with the loader
GameClient uses. The AI players are now created after InitAfterLoad, as
GameClient does. They used to receive the BQ changes it makes through
NodeNote::BQ; now an AIJH reads the finished BQs when it is created.
The game runs identically: the replays and savegames recorded before
and after the change differ only in the version and timestamp header.
They were created as PlayerState::Occupied, the state of a human player,
so code that checks for a human treated the AIs as humans. A Lua script
could not give them construction orders (AIConstructionOrder returned
false), IsHuman() in Lua was true for them, pact requests reached them
as post messages and the AUTOFLAGS addon placed flags on their roads.
Replays recorded by ai-battle now also store the AI info of the players.
A replay with a Lua script that logs aborted with "Could not open
logs/... for writing" unless the working directory happened to contain
a logs folder.
Playback ended after the last recorded command, so the final state was
a few frames short of the recorded game and a replay without commands
played nothing. Like GameClient, the frame with the last GF is executed
and the game stops after it. The GF counts shown by the replay-player
count frames accordingly.
A command for a GF that already passed can never be executed, so
playback silently ran on to the replay's last GF with the remaining
commands ignored. RunGF now throws in that case, in Debug and Release
alike. Tested with a recorded replay
whose second command claims an earlier GF than the first.
@MichalLabuda

Copy link
Copy Markdown
Contributor Author

@Flamefire @wichern Thanks for the reviews, both are addressed now:

  • Unified loading (@Flamefire): SetupGameWorld (GameSetup.h) sets up the world for GameClient, HeadlessReplay and ai-battle, so the fish fix and the other replay compatibility rules live in one place (Replay::usesFishFix()).
  • No more async counting (@Flamefire): playback stops at the first desync, and the game's replay mode reports only the GF where the replay went out of sync.
  • HeadlessReplay as a class (@wichern): it now lives in s25Main, with Run(callback), RunGF() and read access to the game, world and replay, so other tools such as a training-data extractor can build on it.

I've rewritten the PR description to cover the whole PR in its current state. The new work comes as 11 commits after the master merge, and each builds on its own, so they can be reviewed one by one.

This one turned out to be a big job. I did it with the help of Claude Opus 5.5 and Fable 5.1.

@MichalLabuda
MichalLabuda marked this pull request as ready for review October 9, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants