Skip to content

feat(samples): add the samples section, a standalone proxy and a gap report - #13

Merged
tela merged 6 commits into
mainfrom
feat/samples-section
Nov 4, 2026
Merged

tela merged 6 commits into
mainfrom
feat/samples-section

Conversation

@tela

@tela tela commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Motivation

The demos in apps/ are ours and are staged to show well. Real customer use cases and edge cases live in open source apps written by someone else, so running those against the app emulators shows where an app emulator differs from the real vendor. This adds the samples/ section, the rules for it, and the two tools a run needs. It adds no sample app.

Changes

  • feat(shared): a validator for a sample's descriptor (sample.json): a full 40-character commit SHA, an SPDX license, how each vendor is pointed at its app emulator, the command to run. A branch or tag is rejected because a sample must be reproducible.
  • feat(samples): the samples/ directory, its README and make check-samples. A sample may hold only sample.json, README.md and results/, so app code never lands in this repo and licenses do not mix.
  • ci: make check-samples in build-and-test and in the no-mistakes gate.
  • feat(shared): wire-proxy, the recording proxy as a standalone command. One listener per vendor, one shared recording, served at /events and written to --out on exit. It is the proxy the demo uses, so credentials in bodies are masked the same way.
  • feat(shared): gap-report, which says which recorded calls an app emulator does not cover, from the app emulators' twin-manifest.json.
  • docs(samples): how to record a run and read the report.

Findings that shaped the design:

  • A status cannot show a gap. Stripe's app emulator answers a route it lacks with 404 Unrecognized request URL, copying the real Stripe, so the report reads the manifests instead.
  • The manifests in WonderTwin-AI/wondertwin do not share a format. Xero lists routes (PUT /api.xro/2.0/Accounts), Plaid lists paths (item/get), Stripe lists resource names (payment_intents). The report reads all three and prints which one applies in a BASIS column. A resource-level covered only means some of that resource is implemented. One format in the manifests would make the answer exact.

Deviation from the plan: the checks run as a step in the existing workflow, not as a separate path-filtered one. A path-filtered check that a branch rule requires never reports on a PR that does not touch those paths. A separate workflow makes sense once a sample run needs its own toolchain.

Tests

gofmt -l .                                                      (no output)
go vet (every module)                                            (clean)
go test -race -count=1 (every module)                            (all ok)
make check-kits                                                  (ok)
make check-samples                                               (samples in ../samples are well formed)

Each commit was checked out on its own and passed vet and the shared tests.

Mutation checks: with the descriptor rules removed the validator tests fail, with stop not writing --out or since ignored four wire-proxy tests fail, and with method matching, the unchecked default or the id heuristic broken five gaps tests fail.

Live run: the ACME demo's full flow (28 calls, 18 distinct) against Stripe, Plaid and Xero app emulators built from WonderTwin-AI/wondertwin origin/main (8e4a7235), reported against the real manifests from that commit: 0 not covered. Through a standalone wire-proxy, a call to /v1/radar/value_lists was reported as a gap and the recording was written on SIGTERM. Against a Xero manifest without PUT /api.xro/2.0/Accounts, that call was reported as the gap, which is the failure behind the 405 seen against an older Xero app emulator.

Related

Follows localstack-samples/localstack-app-emulators#12. The first sample app is not in this PR.

tela added 6 commits November 4, 2026 14:30
A sample is someone else's open source app, so this repo must record
which commit ran and under which license, and must never hold the app's
code. Reject a branch as a commit, a missing license, an unpointed
vendor and any file that is not the descriptor, notes or results.
Samples are a different kind of thing from the demos in apps/, so they
get their own directory, rules and make check-samples. The check runs
the descriptor validator over samples/ and fails on a bad one.
A bad descriptor or app code inside a sample should fail the PR that adds
it, not a later run. Add the check to CI and to the no-mistakes gate,
which mirrors CI.
The recording proxy only ran inside the demo's binary. A sample app is
someone else's process, so run the same proxy on its own: one listener per
vendor, one shared recording, served over HTTP and written to a file on exit.
A status cannot say it: Stripe's app emulator answers a missing route with
a 404 that copies the real Stripe. Read each emulator's manifest instead.
The manifests list routes, paths or resource names, so match each the way
its kind allows and say how precise the answer is. A vendor with no
manifest is unchecked, never a gap.
Presenters of a sample need the two commands, and anyone reading a report
needs to know why a status cannot show a gap and why Stripe's answer is
less precise than Xero's.
@tela
tela merged commit fa963e5 into main Nov 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant