Skip to content

Count visitors only when the cookie comes back - #72

Merged
kaihendry merged 2 commits into
mainfrom
visitor-confirmation
Nov 3, 2026
Merged

kaihendry merged 2 commits into
mainfrom
visitor-confirmation

Conversation

@kaihendry

@kaihendry kaihendry commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Why

visitors.sh jumped from ~1.3k to ~5.5k daily visitors on 26 Oct, while page hits stayed flat (~19–22k a day).

Since e7d9475, a page request without a cookie logs a freshly minted ID. So a client that never keeps cookies counts as a new visitor on every request. Since 26 Oct, one client has been reloading a stop page every 20 seconds (the page's own auto-refresh), 24/7, without cookies. That adds ~4,300 one-request IDs a day. Excluding that client, exact distinct IDs were ~1.4–1.5k a day both before and after.

Change

  • Pages still set the cookie, but a visitor is logged only when a request sends it back, on any route, as unique on that request's response line. A new browser does that straight away for the page's stylesheet, script, icon and manifest, so first visits still count. Bots and cookie-blocking clients don't count, and their response lines have no unique.
  • The manifest link gets crossorigin="use-credentials", because browsers fetch manifests without cookies otherwise, even from the same origin.
  • visitors.sh and the README filter on ispresent(unique), which also matches the older return visitor lines, so past figures are unchanged.
  • visitors.sh daily counts are exact (two stats steps). countDistinct is approximate at this cardinality: it undercounted ~1.5k-a-day days by 12–14%, which exaggerated the jump.

Considered instead

Counting daily uniques as HMAC(secret, date + IP + User-Agent). In the logs, ~46% of returning cookies use more than one IP within a day, and ~360 share ~137 IPs. A hash like that would track mobile-network behaviour more than people, still count bots once per IP, and add a second "visitor" identity alongside the cookie used for weekly retention.

After merge

  • Past figures in the logs don't change.
  • New daily visitors should return to roughly the pre-23-Sep level, so treat the deploy date as a break in the series.
  • One log event per request: no separate return visitor line.

Testing

  • go vet ./... and go test ./... pass.
  • Locally, the first page request logs no unique, and the stylesheet request that follows logs the new cookie on its response line. The rendered page has crossorigin="use-credentials" on the manifest link.
  • Both visitors.sh modes were run against the live log group; daily figures match the previous filter exactly.

🤖 Generated with Claude Code

Since e7d9475 a page request without a cookie logged a freshly minted ID,
so a client that never keeps cookies counted as a new visitor on every
request. One such client, reloading a stop page every 20 seconds around
the clock, has inflated daily visitors from about 1.4k to 5.7k since
26 October while page hits stayed flat.

Log the visitor when any request sends the cookie back; a new browser
does so at once for the page's stylesheet, script and icon, so first
visits still count. Add visitor_cookie to the response log so cookieless
traffic stays visible, and count daily visitors exactly rather than with
the approximate countDistinct.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rufiec commented Nov 2, 2026

Copy link
Copy Markdown

Claude Code review — 2 findings on commit 32fc3b9

🟡 Nit — main.go:433-458

visitor_cookie=false is not limited to cookieless clients. static/index.html:8 loads <link rel="manifest"> with no crossorigin attribute, and browsers fetch that request with credentials omitted, so every real browser's /static/manifest.json request reaches logRequest with no cookie and logs visitor_cookie=false.

🟡 Nit — main.go:432-436

logRequest now writes a separate "return visitor" log event for every request that carries a cookie (static CSS/JS, icon, /closest, 404s), on top of the "response" event it already writes for the same request. Logging unique as a field on the existing "response" line, and having visitors.sh filter on ispresent(unique) or visitor_cookie = 1, would carry the same data in one event.


Generated by Claude Code

Review feedback: a separate "return visitor" event for every request
with a cookie duplicated that request's response event, and browsers
fetch the manifest without cookies unless the link asks for them, so
ordinary browsers looked cookieless.

Record unique on the response line instead and drop visitor_cookie,
since no unique already means no cookie. visitors.sh and the README
filter on ispresent(unique), which also matches the older "return
visitor" lines, so past counts are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaihendry
kaihendry merged commit 22dff70 into main Nov 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants