Skip to content

Add Target mirror + task verifiers (site rebuilt from real target.com data)#72

Open
1171-jpg wants to merge 1 commit into
aiming-lab:mainfrom
1171-jpg:target-mirror
Open

Add Target mirror + task verifiers (site rebuilt from real target.com data)#72
1171-jpg wants to merge 1 commit into
aiming-lab:mainfrom
1171-jpg:target-mirror

Conversation

@1171-jpg

@1171-jpg 1171-jpg commented Jul 21, 2026

Copy link
Copy Markdown

Adds the Target mirror as the 17th WebHarbor site, together with its 20 benchmark tasks
and the deterministic grading contract for them.

Supersedes #51, which was reassigned to me for rebuild. The catalog, imagery, copy, tasks and
verifiers here are all new.

Why a rebuild

The existing Target mirror could not be brought to standard by patching: its catalog was
carried over from another site's product set, all imagery was generated placeholders, and
the seed DB was not byte-identical reproducible. Each of those requires replacing the
content domain rather than amending it.

Mirrored site

Targethttps://www.target.com/
17th WebHarbor site, served on port 40016.

Port assignment

main gives port 40015 to merriam_webster, so Target takes index 16 / port 40016,
registered in all four places: websyn_start.sh, control_server.py,
Dockerfile (EXPOSE … 40016) and the web field of every tasks.jsonl row. The verifiers
are port-agnostic — they match URL paths, not the host port.

Verified after the move: 17/17 sites healthy and returning 200, byte-identical reset holds
for both target and merriam_webster, and reset-all leaves 17/17 alive.

Seeded rows per major model

model rows model rows
products 1,932 protection_plans 852
categories 18 deals 48
brands 562 support_articles 8
reviews 2,350 users 4
store_inventory 5,796 orders 16
stores 15 order_items 24
pickup_slots 75 wishlist_items 16

Products by category: Home Decor 120 · Women's Clothing 116 · Shoes 116 · Kids' Clothing 115 ·
Men's Clothing 115 · Bedding 112 · Toys 113 · Furniture 111 · Pets 109 · Beauty 108 ·
Personal Care 104 · Sports & Outdoors 102 · Baby 101 · Household Essentials 100 · Grocery 99 ·
Kitchen & Dining 98 · Video Games 98 · Electronics 95.

All product names, prices, ratings, rating counts, photographs, specification tables and guest
reviews are scraped from target.com. Nothing in the catalog is invented.

Site capabilities

  • Search — scored token-overlap (not strict AND), so multi-word and partial queries return
    ranked results. /search carries a facet rail (category / brand / price / rating / deals /
    fulfilment / sort); each facet's options are computed from every other facet, so the rail
    can never offer a combination that returns nothing.
  • Listings — 24 products per page with filters preserved across pages, matching target.com.
  • Two-level category navigation — target.com's mega-menu shape (10 sections, 18 categories).
    Click-to-open rather than hover, because a browser-driving agent clicks reliably but hovers
    fragilely; /categories carries the same tree for agents that never open the menu.
  • Product detail — real specification tables, per-attribute guest ratings, "% would
    recommend", protection plans, store inventory and scraped guest reviews.
  • Comparison — shows the specification rows at least two products share plus protection-plan
    pricing, and says so explicitly when the selected items come from different categories.
  • Cart and checkout — quantity accumulates on repeat adds (capped at 5), delivery and store
    pickup flows, completed checkout steps are navigable so an address can be corrected without
    restarting, and the confirmation page carries the pickup store / pickup time / shipping
    address / total.
  • Account — order history, order detail, rewards, wish list, profile editing, support
    requests, and product review submission. Forms validate server-side.

Tasks

20 tasks in sites/target/tasks.jsonl (WebVoyager schema) across 10 functional areas:
help centre, search + filter, product detail, multi-step comparison, account, orders, cart,
checkout (delivery and pickup), wish list, customer service, and user-generated reviews.

7 are stateful and verified against the database rather than the answer text. 4 require ≥7
steps (full delivery checkout, full pickup checkout, and two multi-page comparisons — e.g.
"across the Red Baron frozen pizzas, which has the lowest sodium per serving", which cannot
be answered without opening several product pages).

Answers are anchored on facts that appear only on a detail page — nutrition rows, specification
tables, per-attribute guest ratings, protection-plan pricing — and each was checked against the
rendered search-results DOM to confirm it does not leak there. Two tasks were rewritten during
this check: one asked for price and rating (both printed on the search card) and one asked for
a figure that the search page prints verbatim in the article summary.

There are no "ask me which one" tasks. design-tasks encourages disambiguation tasks, but
CONTRIBUTING rejects tasks that presuppose a human in the loop, and the merriam_webster
reference carries none — so the two disambiguation tasks drafted earlier were re-anchored onto
questions with a single correct answer.

Grading contract

sites/target/verify/verify_lib.py plus one verifier per task, with verifier_path and
judge_rubric on every tasks.jsonl row. No answer key; ground truth lives only inside
the verifiers, and the rubrics were scanned against every answer value to confirm they state
rules rather than answers.

Deterministic-first: a navigation gate (a correct answer with no matching page visit is a
knowledge shortcut), answer-value extraction from free text, and SQLite after-state diffing for
the stateful tasks. LLM helpers are anchored on frozen ground truth and short-circuit under
--no_llm.

Validation

case result
no-op run (homepage only, empty answer, clean DB) all 20 FAIL
correct completion PASS
correct answer with no on-site navigation FAIL
wrong answer FAIL
self-reported success with unchanged DB FAIL
right product read off an unfiltered/unsorted listing FAIL
right pickup time booked at the wrong store FAIL
wish-list item removed, but the wrong one FAIL
review posted with the wrong star rating FAIL

41 scenarios across all 20 tasks; every one behaves as expected. Target--17 was additionally
validated against a real Playwright-driven run (real trajectory, real screenshots, real
before/after databases) rather than a synthetic fixture.

Byte-identical reset

$ curl -X POST http://localhost:8201/reset/target
{"pid":172,"ready":true,"site":"target"}

$ docker exec wh-test md5sum /opt/WebSyn/target/instance/target.db \
                             /opt/WebSyn/target/instance_seed/target.db
c89a8e061c256c9ac3ac26182d450dcd  /opt/WebSyn/target/instance/target.db
c89a8e061c256c9ac3ac26182d450dcd  /opt/WebSyn/target/instance_seed/target.db

seed_data.py uses a fixed SEED_TIMESTAMP with explicit timestamps on every row and no RNG,
so consecutive builds produce an identical md5 — the models' created_at columns default to
datetime.utcnow, which would otherwise make the seed unreproducible. All 13 seed_*()
functions early-return as a whole. All 17 sites return 200, and reset-all leaves 17/17 alive.

Visual fidelity vs. the real site

Mirror rendered at localhost:40016 (compare against https://www.target.com/).

Home Category mega-menu
home megamenu
Product listing (paginated) Faceted search
products search
Detail — specifications Detail — guest ratings & reviews
specs reviews
More (categories, deals, compare, cart, pickup checkout)

categories
deals
compare
cart
pickup

Paired HuggingFace assets PR

https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/46 — uploads target.tar.gz (instance_seed/target.db + static/images: 1,932 product
photos and 18 category images, 15 MB).

Merge the HF dataset PR before this one so a clean-clone build can fetch the assets;
.assets-revision is pinned to that merge SHA in this branch.

Notes for human review

  • Clothing and footwear carry no long-form description. target.com collapses that copy
    behind a "Show more" control whose button resists programmatic clicking. Those categories
    have real names, prices, ratings, photographs and full specification tables, but no prose
    description. No task depends on one.
  • Grocery products have no guest reviews, verified independently against the live site
    (0 of 8 sampled under the gentlest scrape settings) rather than assumed from a failed scrape.
    Review-based tasks avoid that category.
  • Stores have no photography. No store imagery was scraped and the guide forbids
    placeholders, so store pages render without a hero image rather than with a fabricated one.
  • Category-specific facets are not implemented. target.com shows attribute facets per
    category (Roast Type, Flavor, …). The scraped specification data would support them, but it
    needs per-category configuration and unit normalisation; the generic facets cover the tasks.
  • Pickup windows are shared across stores. All 15 stores offer the same five windows, so
    the picker lists them once and the store is chosen separately, rather than repeating 75
    near-identical options.

…erifiers

Adds the Target mirror as the 17th site (index 16, port 40016), rebuilt from
real target.com data scraped with Playwright.

Catalog: 1,932 products across 18 categories, 562 brands, with real product
photography, 19,821 specification rows and 2,350 scraped guest reviews.

Tasks: 20 WebVoyager tasks in sites/target/tasks.jsonl, each anchored on a
fact that only appears on a detail page.

Grading contract: sites/target/verify/ with verify_lib.py and one deterministic
verifier per task, recorded as verifier_path + judge_rubric on every task row.
Ground truth lives only in the verifiers, never in tasks.jsonl.

Assets uploaded to HF as target.tar.gz (discussions/46).

Based on the Target site skeleton from aiming-lab#51 by @Lxr-max, which was reassigned
for rebuild after review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PLRyFZ4TjY9pC7uZkKJC6H
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant