feat(helm): migrate gateway configuration to gatewayConfig by gmenher · Pull Request #3384 · NVIDIA/OpenShell

gmenher · 2026-09-16T15:02:31Z

Summary

Migrate the Helm chart from field-by-field gateway.toml construction to the schema-v2 gatewayConfig boundary while preserving the non-secret ConfigMap boundary and Helm-owned Secret, volume, and resource wiring.

Related Issue

Closes #3060.

Compatibility

This implementation is backwards-compatible for 0.1.x:

Retains the deprecated sandboxRuntime, supervisor, upstreamProxy, and Kubernetes driver aliases for existing values files.
gatewayConfig is authoritative whenever both the schema-v2 field and its legacy alias are supplied.
Resolves aliases once and uses the same effective Kubernetes driver configuration for rendered TOML, validation, NetworkPolicy acknowledgement, and RBAC/workspace resources.
Validates schema-v2 proxy authentication Secret references and keeps private material outside gatewayConfig.

Changes

Add deterministic generic YAML-to-TOML rendering for the schema-v2 configuration boundary.
Preserve chart-owned deployment inputs and derive runtime values from their resource owners.
Cover legacy-to-schema-v2 compatibility, explicit precedence, workspace modes, NetworkPolicy acknowledgement, proxy CA wiring, and Secret-reference validation with Helm tests.
Add parser-backed rendering and resource-coherence validation.
Update Helm, Kubernetes, architecture, reference, and migration documentation.

Testing

mise run ci
mise run helm:test (170 gateway-chart tests and 5 workspace-chart tests)
mise run helm:lint across chart overlays
Gateway TOML parser and resource-coherence validation
CodeRabbit review; its Secret-reference finding is covered by a regression test

Checklist

Commits include DCO Signed-off-by trailers.
Documentation and generated Helm README are updated.
Compatibility and schema-v2 precedence are covered by tests.

copy-pr-bot · 2026-09-16T15:02:35Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

krishicks · 2026-09-17T19:08:08Z

/ok to test c5afa06

github-actions · 2026-09-18T15:03:03Z

Label test:e2e applied, but pull-request/3384 is at c5afa06 while the PR head is cb8ae8b. A maintainer needs to comment /ok to test cb8ae8bba78baed32b09240a49abbcf7f5509c92 to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

krishicks · 2026-09-18T15:03:08Z

/ok to test cb8ae8b

gmenher · 2026-09-18T16:56:09Z

@krishicks I pushed a new commit solving a conflict with e2e/rust/e2e-kubernetes.sh that was blocking the merge.

krishicks · 2026-09-18T18:58:59Z

/ok to test 5efe3e8

krishicks · 2026-09-21T16:09:02Z

/ok to test cfae1b6

krishicks · 2026-09-22T15:39:09Z

I added this to to 0.1.1 milestone as we're freezing what goes into 0.1.0. For this to actually land in 0.1.1 it would need to be implemented in a backwards-compatible way. Failing that this would need to be pushed to 0.2.0 which is the next release where breaking changes can get in.

gmenher · 2026-10-06T12:37:15Z

Rebased onto latest main and conflicts are resolved. @krishicks could I have the test suite re-executed with /ok to test when convenient? Thanks!

dvavili · 2026-10-06T17:13:18Z

I did some analysis as the goal is to have this rolled in a backwards compatible manner. Posting my analysis:

Compat check against 0.1.x: legacy server.oidc.* without role names loses RBAC on upgrade:

I rendered the chart at the merge-base (b8ffe52) and at this PR's head (de1451b) with the same legacy ci/values-*.yaml overlays, then compared the parsed gateway.toml and every other resource. All 18 overlays render on both. Nothing is removed or changed except the case below. The rest are additions that match gateway defaults (disable_tls = false, empty otlp.service_name, empty scopes_claim).

Regression: a values file that sets server.oidc.issuer and audience but no role names (for example values-gateway-tls.yaml) now renders:

[openshell.gateway.oidc]
admin_role = ""
user_role = ""
roles_claim = ""

The old template emitted roles_claim, admin_role and user_role only when non-empty (gateway-config.yaml L163-171), so the gateway fell back to realm_access.roles / openshell-admin / openshell-user.

With explicit empty strings:

Config-file values replace defaulted CLI args (cli.rs L1171-1178), so the defaults are lost.
Both roles empty puts AuthzPolicy in authentication-only mode, where any valid token is authorized (auth/authz.rs L20-26, L83-85).
So an existing install that was enforcing the default roles would accept any valid token from its issuer after upgrading. That is a silent loosening of an access control, which conflicts with the "backwards-compatible for 0.1.x" claim.

Repro:

helm template -t deploy/helm/openshell -f <base>/deploy/helm/openshell/ci/values-gateway-tls.yaml
# compare [openshell.gateway.oidc] between merge-base and this PR

Suggested fix: in the legacy OIDC translation, add roles_claim, admin_role and user_role only when the legacy value is non-empty, as the old template did and as this PR already does for the other optional fields. Please also add a Helm test: legacy server.oidc.issuer only → gateway.toml has no admin_role, user_role or roles_claim. Another test could pin that an explicit schema-v2 admin_role = "" is still honoured.

FrostGod · 2026-10-07T09:11:33Z

Optional but, IMO we should split this PR
the Zig wrapper fix and unrelated task cleanup from this PR, and restore the content-guard test invocation? The serializer, compatibility layer, resource wiring, documentation and migration tests belong together.

Signed-off-by: Gaizka Menendez Hernandez <gmenende@redhat.com>

gmenher · 2026-10-07T14:44:18Z

Thanks for the review and detailed analysis to both of you! @FrostGod & @dvavili . I addressed your points in the latest pushed commit. I’d really appreciate another look when you have a chance, and thanks again 🙂

TaylorMutch · 2026-10-07T21:22:45Z

/ok to test b7cde12

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

TaylorMutch

I validated the chart in a local Helm development environment and found three remaining migration issues. Details are inline.

TaylorMutch · 2026-10-08T05:25:59Z

@gmenher Thanks for working through the compatibility changes. I found the three remaining migration issues noted in my review and have fixes prepared locally. I'll push those fixes, along with regression tests, the CI values example covering ordered gateway interceptor and supervisor middleware arrays from #3060, and the small SPDX header reverts, so we can get this across the finish line.

The fixes have been checked with Helm unit tests, config-parser validation, and targeted tests in a local Kubernetes development environment, including credential continuity across an upgrade and the default pull policy for a :latest sandbox image.

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

TaylorMutch · 2026-10-08T05:49:18Z

/ok to test aa5e148

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

TaylorMutch · 2026-10-08T17:28:05Z

/ok to test a1a1933

…y server values (#12949) ## Failure The OpenShell v0.1.3 chart (NVIDIA/OpenShell#3384) moves gateway settings into `gatewayConfig`. NemoClaw still set four of them through `server.*` values, which work only through the chart's compatibility mapping: - `server.oidc` issuer, audience, claims, and roles - `server.auth.allowUnauthenticatedUsers` - `server.workspaceDefaultStorageSize` - `server.drivers.kubernetes.workspaceMode` ## Decision Set them in `gatewayConfig` (schema version 2), using the dotted TOML table keys the chart's own CI values use: ```yaml gatewayConfig: openshell.gateway.oidc: # issuer, audience, jwks_ttl_secs, roles_claim, # admin_role, user_role, scopes_claim, # dangerously_allow_insecure_http openshell.gateway.auth: allow_unauthenticated_users: false openshell.drivers.kubernetes: workspace_mode: shared workspace_default_storage_size: 2Gi ``` `server` keeps only values the templates read directly: TLS, the JWT signing secret, credential storage, telemetry, and `server.oidc.caConfigMapName`. The chart mounts the issuer CA only from that last value. The settings themselves are unchanged. ## Validation - The gateway value tests now require the `gatewayConfig` tables, and that `server` holds only those directly read keys. Both failed before the change and pass after it. All 84 Kubernetes tests pass. - `cargo ci` passed: 1,322 workspace tests and all 120 lifecycle tests. - Live / Kind passed on `linux_arm64` and `linux_amd64` with the real v0.1.3 chart ([run 38003924177](https://github.com/NVIDIA/NemoClaw/actions/runs/38003924177)). That includes `the_pinned_chart_renders_with_the_sdk_values` and `the_gateway_installs_authenticates_and_is_removed_keeping_storage`, which authenticates through the OIDC settings in `gatewayConfig`.

gmenher requested review from a team, derekwaynecarr, mrunalp and sjenning as code owners September 16, 2026 15:02

gmenher force-pushed the openshell/helm-gateway-config branch from c5afa06 to cb8ae8b Compare September 18, 2026 11:50

krishicks added the test:e2e Requires end-to-end coverage label Sep 18, 2026

krishicks previously approved these changes Sep 18, 2026

View reviewed changes

krishicks added this pull request to the merge queue Sep 18, 2026

github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 18, 2026

gmenher dismissed krishicks’s stale review via 5efe3e8 September 18, 2026 16:48

gmenher force-pushed the openshell/helm-gateway-config branch from cb8ae8b to 5efe3e8 Compare September 18, 2026 16:48

gmenher force-pushed the openshell/helm-gateway-config branch 2 times, most recently from 46d70d8 to cfae1b6 Compare September 21, 2026 12:58

This was referenced Sep 21, 2026

chore(vm): refresh runtime defaults and hardening #3446

Merged

bug(vm-driver): registry image prep expects bundle-style rootfs/ from umoci raw unpack — failure masked as success and broken disk cached forever #2358

Closed

krishicks added this to the OpenShell 0.1.1 milestone Sep 22, 2026

gmenher force-pushed the openshell/helm-gateway-config branch from cfae1b6 to df2e85c Compare September 23, 2026 11:06

This was referenced Sep 25, 2026

feat(ha): add production scaling signals and graceful gateway redistribution #3528

Open

feat(server): add HA capacity metrics, scoped locks, and graceful drain #3710

Closed

drew modified the milestones: OpenShell 0.1.1, OpenShell 0.1.5 Sep 28, 2026

EmilienM mentioned this pull request Sep 29, 2026

feat(server)!: harden gateway peer transport and expand HA conformance #3825

Draft

7 of 8 tasks

gmenher added 2 commits October 5, 2026 15:23

feat(helm): support gateway config arrays of tables

8394486

fix(helm): preserve sandbox identity aliases

4329258

gmenher force-pushed the openshell/helm-gateway-config branch from 4e1bcdb to 4329258 Compare October 5, 2026 14:28

FrostGod reviewed Oct 7, 2026

View reviewed changes

fix(helm): address gateway config review findings

b7cde12

Signed-off-by: Gaizka Menendez Hernandez <gmenende@redhat.com>

TaylorMutch self-assigned this Oct 7, 2026

TaylorMutch added 3 commits October 7, 2026 20:51

fix(helm): preserve gateway config migration settings

04bedc0

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

test(helm): validate extension arrays in CI values

7dd1e58

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

fix(helm): restore unconditional SPDX headers

41a2531

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

TaylorMutch reviewed Oct 8, 2026

View reviewed changes

Comment thread deploy/helm/openshell/templates/_helpers.tpl

Comment thread deploy/helm/openshell/templates/_toml.tpl Outdated

Comment thread deploy/helm/openshell/templates/_helpers.tpl Outdated

TaylorMutch added 3 commits October 7, 2026 22:32

fix(helm): restore route SPDX header placement

a864650

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

docs(helm): simplify configuration and migration examples

694de7f

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

docs(helm): remove redundant flag explanations

aa5e148

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

docs(helm): move chart configuration guidance to Kubernetes setup

a1a1933

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

TaylorMutch approved these changes Oct 8, 2026

View reviewed changes

TaylorMutch added this pull request to the merge queue Oct 8, 2026

Merged via the queue into NVIDIA:main with commit fe3942f Oct 8, 2026
107 of 113 checks passed

rhods-devops-app Bot mentioned this pull request Oct 9, 2026

Automated Sync from main to stable red-hat-data-services/openshell#22

Merged

cv mentioned this pull request Oct 9, 2026

refactor(sdk): pass gateway settings through gatewayConfig, not legacy server values NVIDIA/NemoClaw#12949

Merged

lunarwhite mentioned this pull request Oct 10, 2026

feat(helm): make gateway PVC size and StorageClass configurable #3854

Open

4 of 6 tasks

Conversation

gmenher commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Compatibility

Changes

Testing

Checklist

Uh oh!

copy-pr-bot Bot commented Sep 16, 2026

Uh oh!

krishicks commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

krishicks commented Sep 18, 2026

Uh oh!

gmenher commented Sep 18, 2026

Uh oh!

krishicks commented Sep 18, 2026

Uh oh!

krishicks commented Sep 21, 2026

Uh oh!

krishicks commented Sep 22, 2026

Uh oh!

gmenher commented Oct 6, 2026

Uh oh!

dvavili commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FrostGod commented Oct 7, 2026

Uh oh!

gmenher commented Oct 7, 2026

Uh oh!

TaylorMutch commented Oct 7, 2026

Uh oh!

TaylorMutch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

TaylorMutch commented Oct 8, 2026

Uh oh!

TaylorMutch commented Oct 8, 2026

Uh oh!

TaylorMutch commented Oct 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gmenher commented Sep 16, 2026 •

edited

Loading

dvavili commented Oct 6, 2026 •

edited

Loading