Repository navigation
Sync release-1.0 with upstream - #133
Merged
Merged
Conversation
Part of agent-substrate#3101. When local snapshots fill a node's disk, the kubelet evicts atelet and rejects every replacement while the node has DiskPressure. Uploads and detaches stop, so nothing frees the space. Only atelet frees what substrate writes under `/var/lib/ate`, because the kubelet doesn't count that hostPath toward any pod and its image GC doesn't reach it. The kubelet never evicts a pod at priority 2000000000 or above, and it admits one under DiskPressure. So atelet now runs at `system-cluster-critical`. In test runs based off of agent-substrate#3101 atelet was never evicted for either `system-node-critical` or `system-cluster-critical` and uploaded on schedule. `system-node-critical` would also give atelet `oom_score_adj` -997. With no memory limits on atelet and objectstore-plugin, a leak in either would then make the kernel kill workers first. Per discussion in this PR the manifest comment says to switch once both containers have memory limits. GKE admits critical pods outside kube-system only in a namespace with a `ResourceQuota` scoped to their class. The manifest adds one that allows both classes, so switching later needs only the `priorityClassName` change. Its `pods: 1G` limit matches GKE's own `gcp-critical-pods` quota. Clusters that don't restrict the critical classes, like kind, admit atelet without it. The kubelet enforces an emptyDir `sizeLimit` only by evicting the pod, so the `asset-staging` limit no longer applies. I removed it and the test that checked `--asset-staging-capacity` fits under it. An install upgraded across this change keeps the old `ate-node-critical` PriorityClass. Delete it with `kubectl delete priorityclass ate-node-critical`. I used an AI agent (Claude Code) to run the agent-substrate#3101 tests, make this change and draft this description. --------- Signed-off-by: Ivy <igooch@google.com> (cherry picked from commit 1b8f2e0) Signed-off-by: dberkov <dberkov@gmail.com>
…e#3457) Part of agent-substrate#3226. The API guide says a `durableDir` is created `0777`, like an emptyDir. On gVisor its root is `01777`. The volume is a tmpfs inside the sandbox, and its mount hint sets no mode, so it gets the tmpfs default. With the sticky bit, a user can't delete or rename another user's files in the volume's root. A durable dir created before this keeps `01777` when it is restored from a VOLUMES or ROOTFS snapshot, since the snapshot records its mode. Breaking Change: A gVisor actor whose template has a `durableDir` can't restore a MEMORY snapshot taken before this change. On a MEMORY restore, runsc checks that the sandbox spec's mount hints match the checkpoint, except `.source`. The new `dev.gvisor.spec.mount.<volume>.options` key doesn't match, so `runsc restore` fails and the actor goes to `CRASHED`. This includes MEMORY golden snapshots of these templates, and actors suspended while their worker rolls during the upgrade. No API restores such a snapshot without its memory. On upgrade: - Recreate affected templates so they take new golden snapshots. - Delete and recreate affected actors that hold a MEMORY snapshot from before the upgrade. A rollback breaks the same way. MEMORY snapshots taken after the upgrade fail to restore on the old worker. From v1.0.0, changes must keep upgrades and rollbacks working, so this has to land before then. AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [ ] Docs updated Signed-off-by: Nayana Bidari <nybidari@google.com> (cherry picked from commit 76cba32) Signed-off-by: dberkov <dberkov@gmail.com>
This adds an e2e suite and a workflow that upgrade a running install with actors on it, following `docs/upgrade.md` step by step. The workflow installs the newest earlier release on the branch with that release's own installer, and the suite starts counter actors that are running, paused, or suspended. One job runs them on gVisor and one on microVM. It then runs each runbook step with this checkout's `ate-setup` and a copy of the install record: podcertificate-controller, atelet, the WorkerPool's `workerImage`, the API server, ate-controller, atenet, the SandboxConfig, then `deploy ate-system --setup-csi=none`. While it runs, it acts as the actors' harness and suspends an actor as soon as its worker pod starts terminating. After each step, the step's component has to run new images and no actor may be `CRASHED`. Only the pool roll may roll the workers and evict actors, plus ate-controller if its pod template changed. At the end, every actor, plus one created after the upgrade, has to resume on a new worker with its counter intact. ----- ### Cadence Patch releases are cut by merging fixes into `release-X.Y` and tagging there. The workflow runs on every push to a `release-X.Y` branch and upgrades from the newest earlier tag on that branch, so the commit that becomes v0.6.1 is tested as `v0.6.0` → `v0.6.1` before it is tagged. A push with no earlier tag yet, such as the branch cut itself, is skipped with a notice. It also runs on demand. It doesn't run on pull requests or on pushes to main. After GA 1.0 we support upgrading to the next minor, for example 1.1.x → 1.2.y, so a push to `release-1.2` should also upgrade from the newest 1.1 release. That second lane comes with 1.0. -------- #### Tested - main → this PR, on GitHub runners in my fork: both lanes pass all 8 steps, in about 15 and 18 minutes. The microVM lane needs KVM, so I only runs it on GitHub runners. Not covered yet: rollback. I used Claude Code to help write the suite and to run the tests above. Part of agent-substrate#1784 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR --------- Signed-off-by: Haven Xia <haoyuxia@google.com> (cherry picked from commit 2d96fd6) Signed-off-by: dberkov <dberkov@gmail.com>
…can run the actor checks (agent-substrate#3358) Step 2 of agent-substrate#3095. A provider in its own repository cannot import the actor checks under `internal/`. - `New(ctx, Config)` builds the verifier through `internal/objectstoreplugin/actorauthz.StartVerifier`, as the in-tree exchange does. - A `.svc` issuer is trusted only through `IssuerCAFile`, in-tree too. An install whose `.svc` issuer certificate verified only through the system roots now fails at sidecar start. - `Authorize(ctx, jwt)` returns `ActorUID`, `Grants` and `Expiry`, or a gRPC status error. A grant access with no public `Access` value gets `Internal`. - `Close` closes the ate-api-server connection and idle issuer connections. - `ateapiauth.ClientConfig` loses `K8sClient`. Its four users add the `k8sresolver` option themselves, so the package does not link client-go. - `hack/verify/package-deps.sh` bans cloud SDKs and client-go here. `hack/verify/pkg-external-import.sh` builds the package from an outside module. Tested: `hack/verify-all.sh` passes. `go test -race ./...` fails only in `internal/atunnel/egress` and `internal/netns`, as at the merge base. No e2e. - [ ] Tests pass - [x] Docs updated --------- Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com> (cherry picked from commit 2289ade) Signed-off-by: dberkov <dberkov@gmail.com>
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bring
release-1.0through upstream4ee8f66ac, preserving the fork patches in one sync merge. This includes critical atelet scheduling, gVisor durable-directory permissions, object-store actor authorization, API documentation, and upgrade E2E coverage.The integration keeps atelet's insecure-mode behavior, applies upstream's priority/quota and asset-staging changes to Helm, and selects agentgateway in the new upgrade workflow.
Use a merge commit when merging this PR to preserve upstream ancestry.
Validation:
tools/apitoolrace tests passed.TestUnpackLayer_UnprivilegedKeepsUnpackingUserfailed withgid = 1000, want 1003, reproduced identically on the unchanged fork basef0f214ae8.AI use: Codex prepared the merge and integration fixes, ran validation, and performed an independent review.