Skip to content

feat(marketing): add agent-friendly page representations - #39

Open
rortan134 wants to merge 4 commits into
mainfrom
codex/agent-friendly-homepage-sitemap
Open

feat(marketing): add agent-friendly page representations#39
rortan134 wants to merge 4 commits into
mainfrom
codex/agent-friendly-homepage-sitemap

Conversation

@rortan134

@rortan134 rortan134 commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add content negotiation for Markdown variants of the localized homepage and sitemap.
  • Support Accept q-values and specificity, 406 responses, RSC bypass, and cache variation headers.
  • Add Markdown alternate links and shared sitemap metadata for human and agent consumers.

Validation

  • bun test (71 passing)
  • bun lint
  • bun run build
  • Live smoke checks for HTML, Markdown, XML, and 406 responses.

Summary by CodeRabbit

  • New Features

    • Added Markdown versions of localized homepage content and the public sitemap.
    • Added English and Spanish homepage Markdown with validation for unsupported locales.
    • Added content negotiation for HTML, Markdown, and XML responses.
    • Added a Markdown alternate link to homepage metadata.
    • Expanded sitemap coverage for public pages and localized URLs.
    • Added XML sitemap access through the API.
    • Added caching and revalidation for sitemap and Markdown responses.
  • Bug Fixes

    • Unsupported content formats now return a clear 406 response.

@pullfrog

pullfrog Bot commented Aug 3, 2026

Copy link
Copy Markdown

no API key found — this repo is configured to use google/gemini-3.1-pro-preview, which needs GEMINI_API_KEY (or GOOGLE_GENERATIVE_AI_API_KEY), but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using Gemini Pro𝕏

@vercel

vercel Bot commented Aug 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cache-app Ready Ready Preview Aug 3, 2026 8:02pm

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7f2dea46-a820-4a51-bbfc-0678ae1ae3c8

📥 Commits

Reviewing files that changed from the base of the PR and between 6bf80ac and a4e5881.

📒 Files selected for processing (10)
  • app/api/markdown/home/[[...locale]]/route.ts
  • app/api/markdown/sitemap/route.ts
  • app/api/sitemap/route.ts
  • app/metadata.ts
  • lib/common/accept.ts
  • lib/common/constants.ts
  • lib/marketing/constants.ts
  • lib/marketing/site-map.ts
  • lib/marketing/url.ts
  • proxy.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • app/api/markdown/sitemap/route.ts
  • app/api/sitemap/route.ts
  • lib/common/accept.ts
  • app/api/markdown/home/[[...locale]]/route.ts
  • proxy.ts

📝 Walkthrough

Walkthrough

Adds localized Markdown homepage and sitemap endpoints, centralized public route metadata, Accept negotiation for HTML, XML, and Markdown responses, and a text/markdown metadata alternate. The XML sitemap now uses the shared route catalog.

Changes

Markdown content negotiation

Layer / File(s) Summary
Accept negotiation contract and validation
lib/common/accept.ts, lib/common/accept.test.ts, lib/common/constants.ts
Adds media-type matching, quality ordering, wildcard handling, rejection behavior, and Vary: Accept support with tests.
Localized sitemap route catalog
lib/marketing/site-map.ts, lib/marketing/url.ts, lib/marketing/constants.ts, app/sitemap.ts, app/api/sitemap/route.ts, app/api/sitemap/route.test.ts, app/api/markdown/sitemap/route.ts, app/metadata.ts
Defines shared public routes, localized URLs, cache headers, and XML escaping. XML and Markdown sitemap generation use the shared catalog.
Markdown homepage and metadata
app/api/markdown/home/[[...locale]]/route.ts, app/api/markdown/home/[[...locale]]/route.test.ts, app/[locale]/page.tsx
Adds English and Spanish Markdown homepage responses, locale validation, endpoint tests, and a localized text/markdown alternate.
Proxy representation routing
proxy.ts
Negotiates homepage and sitemap representations, rewrites Markdown requests, handles locales, bypasses RSC requests, returns 406 responses, and matches /sitemap.xml.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant proxy
  participant negotiateContentType
  participant MarkdownHomeRoute
  participant SitemapRoute
  Client->>proxy: Request homepage or sitemap with Accept
  proxy->>negotiateContentType: Select supported media type
  negotiateContentType-->>proxy: Return HTML, XML, Markdown, or null
  proxy->>MarkdownHomeRoute: Rewrite localized Markdown homepage request
  proxy->>SitemapRoute: Route sitemap representation request
  proxy-->>Client: Return negotiated response or 406 response
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding agent-friendly representations for marketing pages.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/agent-friendly-homepage-sitemap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds negotiated Markdown representations for localized homepages and the sitemap while retaining HTML and XML defaults.

  • Parses Accept media ranges with quality and specificity handling.
  • Rewrites Markdown and text/xml requests to dedicated route handlers.
  • Adds localized Markdown alternate metadata and shared sitemap generation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; both previously reported representation issues are fixed in the current code.

Important Files Changed

Filename Overview
proxy.ts Adds content negotiation, RSC bypass, representation rewrites, 406 handling, and Vary: Accept; the prior text/xml rejection is fixed.
app/[locale]/page.tsx Adds a localized Markdown alternate that now targets the dedicated homepage Markdown route.
app/api/markdown/home/[[...locale]]/route.ts Serves validated English and Spanish homepage Markdown with explicit caching and content-type headers.
lib/common/accept.ts Implements media-range parsing and deterministic negotiation by specificity, quality, and header position.
lib/marketing/site-map.ts Centralizes public sitemap entries and safely renders escaped XML output.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Homepage or sitemap request] --> B{RSC request?}
    B -->|Yes| C[Continue to page renderer]
    B -->|No| D[Negotiate Accept header]
    D -->|Markdown| E[Rewrite to Markdown API route]
    D -->|text/xml sitemap| F[Rewrite to sitemap API route]
    D -->|HTML or application/xml| G[Continue to default renderer]
    D -->|No acceptable type| H[Return 406]
Loading

Reviews (4): Last reviewed commit: "refactor(marketing): simplify agent-frie..." | Re-trigger Greptile

Comment thread proxy.ts Outdated
Comment thread app/[locale]/page.tsx

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
proxy.ts (1)

15-64: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider splitting the negotiation branch out of proxy.

proxy currently handles RSC detection, Accept negotiation dispatch, Markdown rewrite construction, sitemap passthrough, and gtMiddleware delegation in one function. Static analysis flags this range as high complexity. Extracting the negotiation block (lines 28-56) into a named helper, such as resolveDocumentRepresentation(request, isHomepage, isSitemap) returning either a Response or null (meaning "fall through to gtMiddleware"), would keep proxy itself limited to routing dispatch and improve testability of the negotiation branch in isolation.

As per coding guidelines, "Keep functions small, self-contained, and limited to a single named responsibility."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@proxy.ts` around lines 15 - 64, Extract the Accept negotiation logic from
proxy into a named helper such as resolveDocumentRepresentation(request,
isHomepage, isSitemap), including media-type negotiation, Markdown rewrite
construction, sitemap passthrough, and Not Acceptable responses. Have the helper
return a Response for handled representations or null to indicate fall-through,
while keeping RSC detection and gtMiddleware delegation in proxy.

Sources: Coding guidelines, Linters/SAST tools

lib/common/accept.test.ts (1)

1-108: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add tests for case-insensitive Accept values and duplicate-type quality handling.

The suite covers quality precedence, specificity, wildcards, and defaults well. Two edge cases used by this critical routing path are not covered:

  • Mixed-case media types in the Accept header (e.g. "TEXT/HTML").
  • Duplicate entries for the same type with different q values (e.g. "text/html;q=0.2, text/html;q=0.9"), where negotiateContentType currently keeps the first-declared entry for a given specificity level rather than the highest-quality one.

Since this module decides every homepage/sitemap response representation and drives 406 responses, adding these cases locks in current behavior and catches future regressions cheaply.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/common/accept.test.ts` around lines 1 - 108, Add tests under the
negotiateContentType suite for case-insensitive media-type matching, such as a
mixed-case TEXT/HTML Accept value, and for duplicate entries of the same media
type with differing q values. Assert that matching ignores case and duplicate
entries select the highest quality value, using the existing SUPPORTED_TYPES and
default arguments.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/api/markdown/home/`[[...locale]]/route.ts:
- Around line 81-87: Update the locale handling in the route’s request function
to validate the complete optional locale array with a Zod v4 schema derived from
SUPPORTED_LOCALES. Accept only no locale or one supported locale segment, reject
unsupported locales and extra segments with a 404 Response, and select
HOME_MARKDOWN using the validated locale while preserving the default en-US
behavior.

In `@proxy.ts`:
- Around line 80-98: Normalize the locale segment in both isHomepagePath and
getHomepageLocale by removing a trailing slash from pathname.slice(1) before
comparing it with SUPPORTED_LOCALES. Preserve the root-path handling and
DEFAULT_LOCALE fallback, while ensuring paths such as /en-US/ resolve to the
same locale as /en-US.

---

Nitpick comments:
In `@lib/common/accept.test.ts`:
- Around line 1-108: Add tests under the negotiateContentType suite for
case-insensitive media-type matching, such as a mixed-case TEXT/HTML Accept
value, and for duplicate entries of the same media type with differing q values.
Assert that matching ignores case and duplicate entries select the highest
quality value, using the existing SUPPORTED_TYPES and default arguments.

In `@proxy.ts`:
- Around line 15-64: Extract the Accept negotiation logic from proxy into a
named helper such as resolveDocumentRepresentation(request, isHomepage,
isSitemap), including media-type negotiation, Markdown rewrite construction,
sitemap passthrough, and Not Acceptable responses. Have the helper return a
Response for handled representations or null to indicate fall-through, while
keeping RSC detection and gtMiddleware delegation in proxy.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4db8ab95-e50d-403f-ae3e-3c671a0a68a2

📥 Commits

Reviewing files that changed from the base of the PR and between 358a637 and 144f8de.

📒 Files selected for processing (8)
  • app/[locale]/page.tsx
  • app/api/markdown/home/[[...locale]]/route.ts
  • app/api/markdown/sitemap/route.ts
  • app/sitemap.ts
  • lib/common/accept.test.ts
  • lib/common/accept.ts
  • lib/marketing/site-map.ts
  • proxy.ts

Comment thread app/api/markdown/home/[[...locale]]/route.ts Outdated
Comment thread proxy.ts Outdated
@pullfrog

pullfrog Bot commented Aug 3, 2026

Copy link
Copy Markdown

no API key found — this repo is configured to use google/gemini-3.1-pro-preview, which needs GEMINI_API_KEY (or GOOGLE_GENERATIVE_AI_API_KEY), but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using Gemini Pro𝕏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/marketing/site-map.ts`:
- Around line 74-92: The buildPublicSitemapEntries function currently assigns a
new Date on every sitemap request, making every route appear freshly modified.
Replace this per-call timestamp with real per-route modification dates from
PUBLIC_STATIC_ROUTES, or otherwise reuse a single stable module-level timestamp
across calls while preserving each entry’s lastModified field.

In `@proxy.ts`:
- Around line 95-98: Update createNotAcceptableResponse to derive availableTypes
from the same SITEMAP_MEDIA_TYPES or HOMEPAGE_MEDIA_TYPES arrays used by
negotiateContentType, joining the selected array into the response string so the
406 message stays synchronized when supported media types change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: dd615e61-b11f-4912-a279-1eefba505da7

📥 Commits

Reviewing files that changed from the base of the PR and between 144f8de and c417167.

📒 Files selected for processing (10)
  • app/[locale]/page.tsx
  • app/api/markdown/home/[[...locale]]/route.test.ts
  • app/api/markdown/home/[[...locale]]/route.ts
  • app/api/sitemap/route.test.ts
  • app/api/sitemap/route.ts
  • app/sitemap.ts
  • lib/common/accept.test.ts
  • lib/common/accept.ts
  • lib/marketing/site-map.ts
  • proxy.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • lib/common/accept.test.ts
  • app/[locale]/page.tsx

Comment thread lib/marketing/site-map.ts Outdated
Comment thread proxy.ts Outdated
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@pullfrog

pullfrog Bot commented Aug 3, 2026

Copy link
Copy Markdown

no API key found — this repo is configured to use google/gemini-3.1-pro-preview, which needs GEMINI_API_KEY (or GOOGLE_GENERATIVE_AI_API_KEY), but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using Gemini Pro𝕏

@pullfrog

pullfrog Bot commented Aug 3, 2026

Copy link
Copy Markdown

no API key found — this repo is configured to use google/gemini-3.1-pro-preview, which needs GEMINI_API_KEY (or GOOGLE_GENERATIVE_AI_API_KEY), but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using Gemini Pro𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant