AGENTS.md

AGENTS.md

Project Overview

Talo is a self-hostable game development services platform providing leaderboards, player authentication, peer-to-peer multiplayer, event tracking and more.

Testing

pnpm test                # Run all tests with Vitest
pnpm test path/to/file   # Run specific test file

Tests run against fresh Docker containers. Environment variables from .env are combined with envs/.env.test.

Building + Linting

pnpm lint -- --type-check   # Run Oxlint + tsc

Database Migrations

pnpm migration:create    # Create new MikroORM migration
pnpm migration:up        # Run pending migrations

After creating a migration:

1. Rename from Migration[Timestamp].ts to [Timestamp][PascalCaseDescription].ts
2. Rename the exported class to match the description
3. Remove the override name
4. Import and add to migrations/index.ts (always at the end of the array)

Snapshot files (.snapshot-*.json) and migration files are generated by the commands and must never be manually edited. The only exception is adding backfill data between generated SQL statements.

ClickHouse migrations are created manually in src/migrations/clickhouse/ and registered in src/migrations/clickhouse/index.ts.

Architecture

Four-Tier Routing System

1. Protected Routes (/ prefix) - Web dashboard endpoints

• Auth: JWT signed with JWT_SECRET (user identity)
• Configured in: src/config/protected-routes.ts
• Routes in: src/routes/protected/

2. API Routes (/v1/ prefix) - Game-facing REST API

• Auth: JWT signed with game.apiSecret (per-game API key)
• Configured in: src/config/api-routes.ts
• Routes in: src/routes/api/

3. Admin API Routes (/admin/v1/ prefix) - Dashboard/ops endpoints authenticated by admin API keys

• Auth: game-specific admin API keys with scope-based authorization
• Configured in: src/config/admin-api-routes.ts
• Routes in: src/routes/admin/
• Scopes defined in the AdminAPIKeyScope enum in src/entities/admin-api-key.ts

4. Public Routes (/public/ prefix) - Unauthenticated endpoints

• Use cases: Webhooks, health checks, password reset
• Configured in: src/config/public-routes.ts
• Routes in: src/routes/public/

Admin API Route Pattern

Admin API routes mirror the game-facing API routes and share logic with protected dashboard routes:

• Extract a shared handler from the protected route (e.g. createStatHandler, listLeaderboardsHandler) that both the protected and admin routes call. The handler takes actor: User | AdminAPIKey, so the protected route passes ctx.state.user and the admin route passes ctx.state.key.
• Authorization via requireAdminScopes([AdminAPIKeyScope.X]) (see src/middleware/policy-middleware.ts).
• Game scoping comes free from the admin API key middleware (ctx.state.game is the key's game). Resources loaded by id must be scoped to it (404 for cross-game), using a per-tree common.ts loader (e.g. src/routes/admin/game-stat/common.ts). Resource loaders are duplicated per route tree (protected/api/admin each have their own loadStat); only generic middleware lives in src/middleware/.
• Docs are added as you go: each feature dir has a docs.ts exporting RouteDocs constants wired via docs: on the route config. Schema params get descriptions via .meta({ description }) (note: z.object().partial() strips meta — re-apply it, see optionalFields in src/routes/protected/game-stat/common.ts).
• Register the feature router in src/config/admin-api-routes.ts with a [feature]AdminRouter factory and docsKey.

Request Flow

Middleware executes in order (see src/index.ts).

Then route-specific middleware:

• API Routes: API key extraction → JWT auth → rate limiting → current player resolution → player auth validation → continuity checks
• Protected Routes: JWT auth → user authorization
• Admin API Routes: admin API key extraction → scope checks
• Public Routes: No authentication

Finally, route handlers execute.

All handlers receive ctx.em (EntityManager) for queries. Migrations run automatically on startup (except in test mode).

MikroORM Identity Map + Request Context

Each HTTP request has its own isolated EntityManager with an Identity Map - an in-memory cache that maintains a single instance of each entity throughout the request lifecycle.

Key behaviors:

• When you query the same entity multiple times within a request, you get the identical object reference
• Entities already loaded in the Identity Map are automatically populated into newly fetched entities
• If entity A is loaded with its relations, and later entity B references A, the already-loaded A (with its relations) is used

Practical implication: You don't need to explicitly load relations if they're already in memory from a previous query in the same request. For example:

// In API middleware, ctx.state.game is loaded
// Later in loadAlias middleware:
const alias = await ctx.em.repo(PlayerAlias).findOne({
  id: aliasId,
  player: { game: ctx.state.game }, // game already in Identity Map
})
// alias.player.game is automatically populated from the Identity Map
// No need to explicitly load it via `fields: ['player.game.id']`

Entity queries must always go via em.repo(EntityName) (e.g. em.repo(User).find(), em.repo(Player).findByCursor()) rather than em.find(EntityName, ...), em.findOne(EntityName, ...), or em.count(EntityName, ...). em.persist(), em.remove(), em.flush(), and em.clear() are EntityManager-level operations and are used directly.

Stale entities: a query returns the identity-map instance even if another transaction/request changed the row. To re-read the current DB state in place, use em.refresh(entity) / em.refreshOrFail(entity) rather than a fresh findOne with refresh: true. In tests, refresh an entity after a request commits to see its post-request state. Pass { filters: false } to refresh soft-deleted rows too.

WebSocket Layer

Real-time communication via custom WebSocket implementation in src/socket/:

• Connection state tracking (game, API key, scopes)
• Message routing and pub/sub patterns
• Socket tickets for authentication

Key Directories

src/
├── index.ts                 # App entry point, middleware pipeline
├── entities/                # MikroORM data models
├── routes/                  # Route handlers
│   ├── api/                 # Game-facing API endpoints (/v1/*)
│   ├── protected/           # Dashboard endpoints (/*)
│   ├── admin/               # Admin API endpoints (/admin/v1/*)
│   └── public/              # Unauthenticated endpoints (/public/*)
├── middleware/              # Request pipeline processors
├── config/                  # Route registration, providers, scheduled tasks
├── lib/                     # Shared utilities
│   ├── routing/             # Router factories and types
│   ├── docs/                # API documentation registry
│   ├── auth/                # JWT, API key handling
│   ├── props/               # Game live config, property validation
│   ├── billing/             # Stripe integration
│   ├── queues/              # BullMQ job management
│   └── clickhouse/          # Analytics database client
├── socket/                  # WebSocket implementation
├── tasks/                   # Background job definitions
├── migrations/              # Database schema migrations
│   └── clickhouse/          # ClickHouse-specific migrations
└── emails/                  # Email templates (Handlebars)

Common Patterns

ClickHouse DELETE/UPDATE mutations

Never issue a DELETE or UPDATE with a subquery against ClickHouse (DELETE FROM t WHERE id IN (SELECT ...)). ClickHouse rewrites these into mutations that reference a temporary table; if the server restarts the temp table is lost and the mutation retries forever with UNKNOWN_TABLE, eventually hitting the 1000-mutation cap and rejecting every mutation on that table. Instead: SELECT the ids first, then delete/update with chunked, parameterized id arrays (query_params). Keep each chunk small — the client sends params in the request URL, capped at 128KB per field and 1MB per URL — see src/lib/clickhouse/deleteEventProps.ts. Killed mutations stay visible in system.mutations with their failure reason, which is how these are diagnosed.

Adding a New API Endpoint

Use the /new-route skill for step-by-step guidance on creating routes.

Adding a New Entity

1. Create entity in src/entities/my-entity.ts with decorators
2. Register it in src/entities/index.ts
3. Run pnpm migration:create to generate migration
4. Rename and register migration in src/migrations/index.ts
5. MikroORM will auto-migrate on next startup

Error Handling

All errors caught by src/middleware/error-middleware.ts:

• Automatic HTTP status code mapping
• Sentry integration for production
• OpenTelemetry tracing context

Use return ctx.throw() pattern when you need TypeScript to narrow types after the throw:

const player = await em.repo(Player).findOne({ id })
if (!player) {
  return ctx.throw(404, 'Player not found') // return ensures type narrowing
}
// TypeScript knows player is not null here

Authentication vs Authorization

• Authentication: Handled by middleware (JWT validation, API key extraction)
• Authorization: Handled by middleware (user type gates, API scopes)

Important Conventions

• Entity names are singular (Player, not Players)
• Router functions are named [feature]Router or [feature]APIRouter
• Migration files: [Timestamp][PascalCaseDescription].ts
• Use lazy loading for entity relationships to avoid circular dependencies
• API endpoints require scope checks via requireScopes() middleware
• Admin API endpoints require scope checks via requireAdminScopes() middleware
• Protected endpoints require user type checks via userTypeGate() or ownerGate() middleware