Skip to content

Commit f4d5a3e

Browse files
waleedlatif1claude
andcommitted
fix(trigger): publish worker env from Secrets Manager instead of the build machine
`syncEnvVars` sourced the `FUNCTION_EXECUTION_ENV` vars from `process.env` on whatever machine ran the deploy, and dropped each one silently when unset. No deploy path sets them: the CI job exports only `TRIGGER_ACCESS_TOKEN` and `TRIGGER_PROJECT_ID`, and a build server reaches build-time env only through the `TRIGGER_BUILD_` prefix, which appears nowhere in this repo. So 14 of the 16 entries had never reached a worker, and every worker variable was being entered by hand instead. Sourcing from the build's ambient env was also unsafe rather than merely inert: the layer applies with `override: true`, so a local `deploy --env prod` would have published the developer's own `.env` into production. The list now resolves from the same `/{env}/sim/env-vars` secret the app container boots from, through the same `@sim/runtime-secrets` reader, so the two runtimes cannot drift and a new worker variable needs only a key in the secret and an entry in `WORKER_SECRET_KEYS`. `TRIGGER_DEV_ENABLED` is dropped from the list. `syncEnvVars` strips every `TRIGGER_`-prefixed key before building its layer, so that entry had no effect for the life of the config; `assertSyncableKeys` now fails config evaluation rather than letting another one look published. Run dispatch does not read it — the `init` hook marks the run process directly. Resolution never throws: `syncEnvVars` swallows a callback rejection and then publishes nothing, so a failed lookup degrades to the environment-independent constants and leaves the Trigger.dev environment as the previous deploy left it. A key absent from the secret is left untouched rather than blanked, since several are legitimately unset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017BBuD7nBqVURAah8t6jfND
1 parent a0f38db commit f4d5a3e

7 files changed

Lines changed: 396 additions & 57 deletions

File tree

.claude/rules/sim-architecture.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,8 +62,15 @@ Every export of a `'use client'` module becomes a *client reference* on the serv
6262
Server code runs in two runtimes with **different environments**. The app container loads the
6363
full env from `SIM_ENV_SECRET_ID` (Secrets Manager). Trigger.dev workers — which execute
6464
workflows, so every block handler and every tool call — get their env from the Trigger.dev
65-
dashboard; `trigger.config.ts` additionally syncs `DB_APP_NAME`, `TRIGGER_DEV_ENABLED`, and the
66-
`FUNCTION_EXECUTION_ENV` vars. The repo cannot see what the dashboard holds.
65+
dashboard, plus whatever `trigger.config.ts` publishes at deploy time: `DB_APP_NAME` and the
66+
`WORKER_SECRET_KEYS` list in `lib/core/config/trigger-env-sync.ts`, read from the *same*
67+
`/{env}/sim/env-vars` secret the app boots from. To give workers a new variable, add its key to
68+
that list and set it in the secret. The repo still cannot see what else the dashboard holds.
69+
70+
Two constraints on that list. `syncEnvVars` strips every `TRIGGER_`-prefixed key before it
71+
publishes, so such a variable can only be set in the dashboard (`assertSyncableKeys` fails the
72+
build rather than letting one look synced). And a key absent from the secret is left untouched
73+
rather than blanked, so removing it from the secret does not remove it from a worker.
6774

6875
So before replacing a worker's HTTP call to our own API with an in-process call, ask what env
6976
that work reads *on the app side*. Anything gated by a `require*Capability` helper is the sharp

.github/workflows/ci.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,10 +233,23 @@ jobs:
233233
if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
234234
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
235235
timeout-minutes: 15
236+
permissions:
237+
contents: read
238+
id-token: write
236239
steps:
237240
- name: Checkout code
238241
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
239242

243+
# syncEnvVars reads /dev/sim/env-vars from Secrets Manager during the
244+
# build to publish the worker environment. Without these credentials the
245+
# deploy still succeeds, but publishes only the environment-independent
246+
# constants and leaves the worker env at whatever the previous deploy set.
247+
- name: Configure AWS credentials
248+
uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
249+
with:
250+
role-to-assume: ${{ secrets.DEV_AWS_ROLE_TO_ASSUME }}
251+
aws-region: ${{ secrets.DEV_AWS_REGION }}
252+
240253
- name: Setup Bun
241254
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
242255
with:
Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
/**
2+
* @vitest-environment node
3+
*/
4+
import { beforeEach, describe, expect, it, vi } from 'vitest'
5+
6+
const { mockFetchSecretMap } = vi.hoisted(() => ({ mockFetchSecretMap: vi.fn() }))
7+
8+
vi.mock('@sim/runtime-secrets', () => ({ fetchSecretMap: mockFetchSecretMap }))
9+
10+
import {
11+
assertSyncableKeys,
12+
resolveTriggerEnvVars,
13+
SECRET_ID_BY_ENVIRONMENT,
14+
WORKER_SECRET_KEYS,
15+
} from '@/lib/core/config/trigger-env-sync'
16+
17+
const CONSTANTS = ['DB_APP_NAME']
18+
19+
function byName(vars: { name: string; value: string; isSecret: boolean }[]) {
20+
return new Map(vars.map((v) => [v.name, v]))
21+
}
22+
23+
describe('resolveTriggerEnvVars', () => {
24+
beforeEach(() => {
25+
vi.clearAllMocks()
26+
})
27+
28+
it('reads the secret mapped to the environment', async () => {
29+
mockFetchSecretMap.mockResolvedValue({})
30+
31+
await resolveTriggerEnvVars('prod')
32+
33+
expect(mockFetchSecretMap).toHaveBeenCalledWith('/production/sim/env-vars')
34+
})
35+
36+
it.each([
37+
['prod', '/production/sim/env-vars'],
38+
['staging', '/staging/sim/env-vars'],
39+
['preview', '/dev/sim/env-vars'],
40+
])('maps %s to %s', (environment, secretId) => {
41+
expect(SECRET_ID_BY_ENVIRONMENT[environment]).toBe(secretId)
42+
})
43+
44+
it('publishes the constants plus every key present in the secret', async () => {
45+
const secret = Object.fromEntries(WORKER_SECRET_KEYS.map(({ name }) => [name, `${name}-value`]))
46+
const resolved = byName(await resolveTriggerEnvVars('staging', async () => secret))
47+
48+
for (const key of CONSTANTS) expect(resolved.has(key)).toBe(true)
49+
for (const { name } of WORKER_SECRET_KEYS) {
50+
expect(resolved.get(name)?.value).toBe(`${name}-value`)
51+
}
52+
expect(resolved.size).toBe(CONSTANTS.length + WORKER_SECRET_KEYS.length)
53+
})
54+
55+
it('carries the secret flag through from the key table', async () => {
56+
const resolved = byName(
57+
await resolveTriggerEnvVars('staging', async () => ({
58+
REDIS_URL: 'redis://host',
59+
SANDBOX_PROVIDER: 'e2b',
60+
}))
61+
)
62+
63+
expect(resolved.get('REDIS_URL')?.isSecret).toBe(true)
64+
expect(resolved.get('SANDBOX_PROVIDER')?.isSecret).toBe(false)
65+
expect(resolved.get('DB_APP_NAME')?.isSecret).toBe(false)
66+
})
67+
68+
it('omits keys the secret does not carry, and keeps the ones it does', async () => {
69+
const resolved = byName(
70+
await resolveTriggerEnvVars('staging', async () => ({ REDIS_URL: 'redis://host' }))
71+
)
72+
73+
expect(resolved.get('REDIS_URL')?.value).toBe('redis://host')
74+
expect(resolved.has('E2B_API_KEY')).toBe(false)
75+
expect(resolved.size).toBe(CONSTANTS.length + 1)
76+
})
77+
78+
it('treats an empty value as unset so it cannot blank a working dashboard value', async () => {
79+
const resolved = byName(
80+
await resolveTriggerEnvVars('staging', async () => ({ REDIS_URL: '', E2B_API_KEY: null }))
81+
)
82+
83+
expect(resolved.has('REDIS_URL')).toBe(false)
84+
expect(resolved.has('E2B_API_KEY')).toBe(false)
85+
})
86+
87+
it('serializes a non-string secret entry', async () => {
88+
const resolved = byName(
89+
await resolveTriggerEnvVars('staging', async () => ({ E2B_ENABLED: true }))
90+
)
91+
92+
expect(resolved.get('E2B_ENABLED')?.value).toBe('true')
93+
})
94+
95+
it('publishes constants only for an unmapped environment, without reading a secret', async () => {
96+
const loadSecret = vi.fn()
97+
98+
const resolved = await resolveTriggerEnvVars('dev', loadSecret)
99+
100+
expect(loadSecret).not.toHaveBeenCalled()
101+
expect(resolved.map((v) => v.name)).toEqual(CONSTANTS)
102+
})
103+
104+
it('falls back to constants instead of rejecting when the secret cannot be read', async () => {
105+
const resolved = await resolveTriggerEnvVars('prod', async () => {
106+
throw new Error('AccessDeniedException')
107+
})
108+
109+
expect(resolved.map((v) => v.name)).toEqual(CONSTANTS)
110+
})
111+
112+
it('publishes no TRIGGER_-prefixed key, which the sync layer would strip silently', async () => {
113+
const resolved = await resolveTriggerEnvVars('staging', async () => ({
114+
REDIS_URL: 'redis://host',
115+
}))
116+
117+
expect(resolved.filter((v) => v.name.startsWith('TRIGGER_'))).toEqual([])
118+
expect(WORKER_SECRET_KEYS.filter(({ name }) => name.startsWith('TRIGGER_'))).toEqual([])
119+
})
120+
121+
it('rejects a key the sync layer would strip, naming it', () => {
122+
expect(() => assertSyncableKeys(['DB_APP_NAME', 'TRIGGER_DEV_ENABLED'])).toThrow(
123+
/TRIGGER_DEV_ENABLED/
124+
)
125+
})
126+
127+
it('accepts the keys this module actually publishes', () => {
128+
expect(() => assertSyncableKeys()).not.toThrow()
129+
})
130+
})
Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
import { createLogger } from '@sim/logger'
2+
import { fetchSecretMap } from '@sim/runtime-secrets'
3+
import { getErrorMessage } from '@sim/utils/errors'
4+
5+
const logger = createLogger('TriggerEnvSync')
6+
7+
/**
8+
* One variable to publish into the Trigger.dev environment being deployed.
9+
* Mirrors the shape `syncEnvVars` accepts.
10+
*/
11+
export interface SyncedEnvVar {
12+
name: string
13+
value: string
14+
isSecret: boolean
15+
}
16+
17+
/**
18+
* Values that are the same in every environment, so they need no secret lookup
19+
* and are published even when the lookup fails.
20+
*
21+
* Nothing here may start with `TRIGGER_`: `syncEnvVars` drops every key with
22+
* that prefix before it builds its layer, so such an entry looks published and
23+
* never is. `TRIGGER_DEV_ENABLED` used to sit in this list for exactly that
24+
* reason and had no effect for the life of the config; workers that need it get
25+
* it from the Trigger.dev dashboard, and run dispatch does not read it at all
26+
* because the `init` hook in `trigger.config.ts` marks the run process directly.
27+
*/
28+
const CONSTANT_ENV: readonly SyncedEnvVar[] = [
29+
{ name: 'DB_APP_NAME', value: 'sim-trigger', isSecret: false },
30+
] as const
31+
32+
/** Prefix `syncEnvVars` strips from any key it is handed. */
33+
const UNSYNCABLE_PREFIX = 'TRIGGER_'
34+
35+
/**
36+
* Environment a run needs for sandboxed work. Function block runs and the
37+
* document compiler share one provider selection, and the doc-template
38+
* variables decide whether a run reads a generated document through the doc
39+
* sandbox's artifact store or the isolated-vm fallback. The app authors
40+
* documents for whichever compiler it sees, so a worker missing the doc
41+
* template falls back to isolated-vm and tries to run Python or Node-style
42+
* sources as sandbox JavaScript. Reading a generated document under the doc
43+
* sandbox means loading its compiled artifact from the copilot storage
44+
* context, so that bucket has to be visible to the run as well.
45+
*
46+
* To give workers a new variable, add its key here and set it in the
47+
* `/{env}/sim/env-vars` secret. The next deploy publishes it; nothing has to be
48+
* entered in the Trigger.dev dashboard by hand.
49+
*/
50+
export const WORKER_SECRET_KEYS: readonly { name: string; secret: boolean }[] = [
51+
{ name: 'REDIS_URL', secret: true },
52+
{ name: 'REDIS_TLS_SERVERNAME', secret: false },
53+
{ name: 'SANDBOX_PROVIDER', secret: false },
54+
{ name: 'E2B_ENABLED', secret: false },
55+
{ name: 'E2B_API_KEY', secret: true },
56+
{ name: 'E2B_FUNCTION_TEMPLATE_ID', secret: false },
57+
{ name: 'E2B_FUNCTION_TEMPLATE_GENERATION', secret: false },
58+
{ name: 'MOTHERSHIP_E2B_DOC_TEMPLATE_ID', secret: false },
59+
{ name: 'DAYTONA_API_KEY', secret: true },
60+
{ name: 'DAYTONA_FUNCTION_SNAPSHOT_ID', secret: false },
61+
{ name: 'DAYTONA_DOC_SNAPSHOT_ID', secret: false },
62+
{ name: 'S3_COPILOT_BUCKET_NAME', secret: false },
63+
{ name: 'AZURE_STORAGE_COPILOT_CONTAINER_NAME', secret: false },
64+
{ name: 'GCS_COPILOT_BUCKET_NAME', secret: false },
65+
] as const
66+
67+
/**
68+
* Secret backing each Trigger.dev deploy target. `preview` is the `dev-sim`
69+
* branch CI deploys from the `dev` branch, so it reads the dev environment's
70+
* secret. A target absent from this map gets the constants and nothing else —
71+
* guessing a secret would risk publishing one environment's credentials into
72+
* another, which `syncEnvVars` would then apply with `override: true`.
73+
*/
74+
export const SECRET_ID_BY_ENVIRONMENT: Readonly<Record<string, string>> = {
75+
prod: '/production/sim/env-vars',
76+
staging: '/staging/sim/env-vars',
77+
preview: '/dev/sim/env-vars',
78+
} as const
79+
80+
/**
81+
* Resolves the variables to publish into one Trigger.dev environment, reading
82+
* them from the same Secrets Manager entry the app container boots from, so the
83+
* two runtimes cannot drift.
84+
*
85+
* Never throws. `syncEnvVars` swallows a callback rejection and then publishes
86+
* *nothing*, which would silently drop the constants too, so a failed or
87+
* incomplete lookup degrades to publishing what is known rather than aborting.
88+
* Keys the secret does not carry are logged by name and simply not published —
89+
* several are optional (Azure and GCS buckets on an S3 deployment, for one), so
90+
* their absence is normal rather than an error, and leaving them out preserves
91+
* whatever the Trigger.dev environment already held.
92+
*
93+
* @param environment Trigger.dev deploy target (`prod`, `staging`, `preview`).
94+
* @param loadSecret Secret reader, injectable for tests.
95+
*/
96+
export async function resolveTriggerEnvVars(
97+
environment: string,
98+
loadSecret: (secretId: string) => Promise<Record<string, unknown>> = fetchSecretMap
99+
): Promise<SyncedEnvVar[]> {
100+
const secretId = SECRET_ID_BY_ENVIRONMENT[environment]
101+
if (!secretId) {
102+
logger.warn(
103+
`No secret mapped for Trigger.dev environment "${environment}"; publishing constants only`
104+
)
105+
return [...CONSTANT_ENV]
106+
}
107+
108+
let entries: Record<string, unknown>
109+
try {
110+
entries = await loadSecret(secretId)
111+
} catch (error) {
112+
logger.error(
113+
`Failed to read ${secretId}; publishing constants only. Worker env is unchanged from the previous deploy.`,
114+
{ error: getErrorMessage(error) }
115+
)
116+
return [...CONSTANT_ENV]
117+
}
118+
119+
const resolved: SyncedEnvVar[] = [...CONSTANT_ENV]
120+
const missing: string[] = []
121+
122+
for (const { name, secret } of WORKER_SECRET_KEYS) {
123+
const value = normalizeSecretValue(entries[name])
124+
if (value === undefined) {
125+
missing.push(name)
126+
continue
127+
}
128+
resolved.push({ name, value, isSecret: secret })
129+
}
130+
131+
if (missing.length > 0) {
132+
logger.info(
133+
`${missing.length} worker env var(s) not set in ${secretId}; they are left untouched in Trigger.dev`,
134+
{ missing }
135+
)
136+
}
137+
138+
logger.info('Resolved Trigger.dev env vars', {
139+
environment,
140+
secretId,
141+
published: resolved.length,
142+
missing: missing.length,
143+
})
144+
145+
return resolved
146+
}
147+
148+
/**
149+
* Coerces a secret entry to an env var value, matching how container boot
150+
* hydrates `process.env`. An absent or empty value is treated as unset so a
151+
* blank secret entry cannot overwrite a working dashboard value with `''`.
152+
*/
153+
function normalizeSecretValue(value: unknown): string | undefined {
154+
if (value === undefined || value === null) return undefined
155+
const normalized = typeof value === 'string' ? value : JSON.stringify(value)
156+
return normalized === '' ? undefined : normalized
157+
}
158+
159+
/**
160+
* Guards the one silent failure this module cannot otherwise surface: a key the
161+
* sync layer strips is indistinguishable, in the deploy log, from one it
162+
* published.
163+
*
164+
* Called at module load rather than per resolve, so a key added to the wrong
165+
* list fails config evaluation — and therefore the deploy and the test run —
166+
* instead of reaching {@link resolveTriggerEnvVars}, whose contract is to
167+
* degrade rather than throw.
168+
*/
169+
export function assertSyncableKeys(
170+
names: readonly string[] = [
171+
...CONSTANT_ENV.map((v) => v.name),
172+
...WORKER_SECRET_KEYS.map((k) => k.name),
173+
]
174+
): void {
175+
const stripped = names.filter((name) => name.startsWith(UNSYNCABLE_PREFIX))
176+
177+
if (stripped.length > 0) {
178+
throw new Error(
179+
`syncEnvVars strips ${UNSYNCABLE_PREFIX}-prefixed keys, so these can never reach a worker and must be set in the Trigger.dev dashboard: ${stripped.join(', ')}`
180+
)
181+
}
182+
}
183+
184+
assertSyncableKeys()

0 commit comments

Comments
 (0)