Add CarMax mirror + task verifiers (site by @Violet24K, verifiers by reviewer)#57
Open
DEM1TASSE wants to merge 7 commits into
Open
Add CarMax mirror + task verifiers (site by @Violet24K, verifiers by reviewer)#57DEM1TASSE wants to merge 7 commits into
DEM1TASSE wants to merge 7 commits into
Conversation
…com. - 13 SQLAlchemy models (User / Store / Vehicle / SavedVehicle / Comparison + ComparisonItem / Reservation / TestDrive / Appraisal / FinancePreQual / Order / Review / Article) - 59 routes covering search / browse / detail / research / compare / saved / sell-my-car / pre-qual / reserve / test-drive / checkout / account / articles / FAQ / MaxCare / stores / auth - Token-overlap scored search with multi-field weighting - 141 deterministically-seeded vehicles across 31 templates - 12 real CarMax store locations - 5 benchmark users with pre-populated saved/reservation/test-drive/ appraisal/order data - 20 WebVoyager tasks in tasks.jsonl (6 Easy / 9 Medium / 5 Hard, including 2 disambiguation tasks) - Idempotent seed at function level; byte-identical reset verified
Per-task deterministic verifiers (verify_<N>.py) + shared verify_lib.py under sites/carmax/verify/, plus verifier_path + judge_rubric on each task in tasks.jsonl. Each verifier checks navigation (anti knowledge-shortcut), DB after-state for stateful tasks, and the answer against frozen ground truth. Validated against the react agent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 3, 2026
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the CarMax mirror + its verifiers into
mainas one PR, per the maintainer'srequest to merge via the main repo rather than the cross-fork PR #24.
Authorship is preserved — both contributors stay credited. This branch is Zihao Li's
CarMax site commits (from PR #24) with the reviewer's verifier commit on top; each commit
keeps its original author (site = @Violet24K, verifiers = the reviewer). Please merge with a
regular merge (not squash) so both signatures are kept.
Contents:
sites/carmax/site (Flask app, templates, seed, 20 tasks) — @Violet24Ksites/carmax/verify/verify_<0..19>.py+verify_lib.pyandverifier_path/judge_rubricin
tasks.jsonl— reviewer deliverable (deterministic verifier per task: navigation +DB after-state + answer; validated against
agent_demo/agent.py)Known conflict (please resolve on merge):
mainalready merged Merriam-Webster atindex 15 / port 40015, and CarMax was authored at the same slot. On merge, register CarMax as
the next site — index 16 / port 40016 — in
websyn_start.sh,control_server.py,Dockerfile(EXPOSE …-40016), and bump thewebport insites/carmax/tasks.jsonl(40015 → 40016). The verifiers are port-agnostic (they match URL paths, not the host port).
Supersedes the cross-fork PR #24 and the verifier-only PR #56.