Repository navigation
feat(samples): add the samples section, a standalone proxy and a gap report - #13
Merged
Merged
Conversation
added 6 commits
November 4, 2026 14:30
A sample is someone else's open source app, so this repo must record which commit ran and under which license, and must never hold the app's code. Reject a branch as a commit, a missing license, an unpointed vendor and any file that is not the descriptor, notes or results.
Samples are a different kind of thing from the demos in apps/, so they get their own directory, rules and make check-samples. The check runs the descriptor validator over samples/ and fails on a bad one.
A bad descriptor or app code inside a sample should fail the PR that adds it, not a later run. Add the check to CI and to the no-mistakes gate, which mirrors CI.
The recording proxy only ran inside the demo's binary. A sample app is someone else's process, so run the same proxy on its own: one listener per vendor, one shared recording, served over HTTP and written to a file on exit.
A status cannot say it: Stripe's app emulator answers a missing route with a 404 that copies the real Stripe. Read each emulator's manifest instead. The manifests list routes, paths or resource names, so match each the way its kind allows and say how precise the answer is. A vendor with no manifest is unchecked, never a gap.
Presenters of a sample need the two commands, and anyone reading a report needs to know why a status cannot show a gap and why Stripe's answer is less precise than Xero's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The demos in
apps/are ours and are staged to show well. Real customer use cases and edge cases live in open source apps written by someone else, so running those against the app emulators shows where an app emulator differs from the real vendor. This adds thesamples/section, the rules for it, and the two tools a run needs. It adds no sample app.Changes
feat(shared): a validator for a sample's descriptor (sample.json): a full 40-character commit SHA, an SPDX license, how each vendor is pointed at its app emulator, the command to run. A branch or tag is rejected because a sample must be reproducible.feat(samples): thesamples/directory, its README andmake check-samples. A sample may hold onlysample.json,README.mdandresults/, so app code never lands in this repo and licenses do not mix.ci:make check-samplesinbuild-and-testand in the no-mistakes gate.feat(shared):wire-proxy, the recording proxy as a standalone command. One listener per vendor, one shared recording, served at/eventsand written to--outon exit. It is the proxy the demo uses, so credentials in bodies are masked the same way.feat(shared):gap-report, which says which recorded calls an app emulator does not cover, from the app emulators'twin-manifest.json.docs(samples): how to record a run and read the report.Findings that shaped the design:
404 Unrecognized request URL, copying the real Stripe, so the report reads the manifests instead.WonderTwin-AI/wondertwindo not share a format. Xero lists routes (PUT /api.xro/2.0/Accounts), Plaid lists paths (item/get), Stripe lists resource names (payment_intents). The report reads all three and prints which one applies in aBASIScolumn. A resource-levelcoveredonly means some of that resource is implemented. One format in the manifests would make the answer exact.Deviation from the plan: the checks run as a step in the existing workflow, not as a separate path-filtered one. A path-filtered check that a branch rule requires never reports on a PR that does not touch those paths. A separate workflow makes sense once a sample run needs its own toolchain.
Tests
Each commit was checked out on its own and passed vet and the shared tests.
Mutation checks: with the descriptor rules removed the validator tests fail, with
stopnot writing--outorsinceignored fourwire-proxytests fail, and with method matching, the unchecked default or the id heuristic broken fivegapstests fail.Live run: the ACME demo's full flow (28 calls, 18 distinct) against Stripe, Plaid and Xero app emulators built from
WonderTwin-AI/wondertwinorigin/main (8e4a7235), reported against the real manifests from that commit: 0 not covered. Through a standalonewire-proxy, a call to/v1/radar/value_listswas reported as a gap and the recording was written on SIGTERM. Against a Xero manifest withoutPUT /api.xro/2.0/Accounts, that call was reported as the gap, which is the failure behind the 405 seen against an older Xero app emulator.Related
Follows
localstack-samples/localstack-app-emulators#12. The first sample app is not in this PR.