Make the node model workload-aware so a job can run once per deployment - #218
Open
karldebisschop wants to merge 3 commits into
Open
Make the node model workload-aware so a job can run once per deployment#218karldebisschop wants to merge 3 commits into
karldebisschop wants to merge 3 commits into
Conversation
- workload_name: derive the owning workload from the pod's controlling
ownerReference with the Deployment pod-template-hash stripped, falling back
to the first reference then the pod name (StatefulSets, DaemonSets, Jobs,
bare pods); expose it as the default:workload node attribute
- number pods per namespace/workload/container, so each container name gets its
own sequence within a workload and a job can target a single replica by
filtering on index == 1. Numbering runs after the running/terminated filter so
a pod that is not emitted does not consume an index -- otherwise a workload
whose first pod is Terminated emits nodes numbered 2 and 3 and that filter
matches nothing
- RD_CONFIG_ONE_PER_WORKLOAD emits only the first kept pod for each workload
and container, doing once at the model level what an index filter does in
every job, and keeping the node count stable as replicas scale; off by
default
- render nodename from a ${attribute} template that can reference any node
attribute -- default:*, labels:*, index, Custom Mapping aliases and operator
defaults. main() renders it once the pod is known to be kept and has its final
index, so ${index} is meaningful. The default template preserves the
historical "<pod name>-<container name>"
- an explicit nodename from Custom Mapping ("nodename.selector=default:name",
the first example in the README) or from Default attributes still wins. The
template is the default naming strategy, not an override of the operator's;
overriding it would silently rename every node of an existing job, and
Rundeck treats a renamed node as a different node
- an attribute that is unset renders as an empty string rather than the literal
"None": an unscheduled pod has no podIP, and rendering that as "None" would
put the word in the node name and give every pending pod the same one
- resolve Custom Mapping sources against the assembled node attributes rather
than the default:* set alone, and after labels are merged, so
"service.selector=labels:app.kubernetes.io/name" now works and gives an
awkward label key a short alias usable in the template; an unresolvable
source is skipped instead of raising KeyError
- plugin.yaml: add Node Name Format and One Node Per Workload properties, note
that index is counted per namespace, and document label aliases in Custom
Mapping
- tests: workload derivation and controller-owner preference, per-workload
container numbering including the filtered-pod case, the one-per-workload
toggle, template rendering with labels, unset and missing attributes,
explicit-nodename precedence, and mapping from a label
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the Kubernetes pods resource model workload-aware so operators can target exactly one pod per Deployment (or other controller) for “run once per workload” maintenance jobs, while keeping the existing default behavior unchanged unless explicitly enabled.
Changes:
- Add a stable
default:workloadattribute derived from the pod’s owning controller (stripping the Deployment pod-template-hash) and anindexordinal per workload/container for deterministic replica selection. - Add configurable node naming via a
${attribute}template (nodename_format) and aone_per_workloadtoggle to collapse replicas at the model level. - Expand and update the unit test suite to cover workload derivation, indexing behavior, one-per-workload mode, and template rendering.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| plugin.yaml | Adds new plugin config options (nodename_format, one_per_workload) and updates mapping documentation. |
| contents/pods-resource-model.py | Implements workload derivation, per-workload indexing, node name template rendering, and one-per-workload emission behavior. |
| contents/tests/test_pods_resource_model.py | Adds tests covering new workload/index semantics, nodename templates, mapping-from-label behavior, and one-per-workload behavior. |
Suppressed comments (1)
plugin.yaml:95
- The implementation deduplicates on
namespace/workload/container(one node per workload per container). The config description currently says “single node per workload”, which can be misleading for multi-container pods where more than one node will still be emitted.
title: One Node Per Workload?
description: 'Emit a single node per workload (Deployment/StatefulSet/etc.) instead of one per pod, so a job runs once per deployment without every job needing an index filter. The emitted node keeps the first matching pod name in default:name; default:workload carries the stable workload name.'
default: "false"
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
177
to
183
| for key, value in mapping_array.items(): | ||
| if ".selector" in key: | ||
| attribute = key.replace(".selector", "") | ||
| custom_attribute = None | ||
| # take the values from default | ||
| if "default:" in value: | ||
| custom_attribute = default_settings[value] | ||
| custom_attribute = data.get(value) | ||
|
|
||
| if custom_attribute: | ||
| custom_attributes[attribute] = custom_attribute |
Comment on lines
+42
to
+44
| title: 'Node Name Format' | ||
| description: 'Template for the node name. Reference any node attribute with ${...}: default:*, labels:*, index, or an alias defined in Custom Mapping. An attribute that is unset renders as an empty string. index counts the emitted pods within a workload in a single namespace, so include ${default:namespace} when scanning all namespaces or two namespaces running the same workload will produce the same node name. Ignored if a nodename is set explicitly through Custom Mapping or Default attributes. The full pod name remains available as default:name for traceability.' | ||
| default: '${default:name}-${default:container_name}' |
Contributor
Author
There was a problem hiding this comment.
Both descriptions updated
Review findings on the pull request.
A Custom Mapping resolved its source and then applied a truth test, so a label
with an empty value -- legal in Kubernetes -- was dropped even though the source
existed. Test for `is not None` instead; a missing source still resolves to
None and is still skipped.
index counts pods within a namespace/workload/container group, so a template
that omits ${default:container_name} makes the containers of a multi-container
pod render the same node name, exactly as omitting ${default:namespace} does
across namespaces. The Node Name Format description covered only the namespace
case; it now covers both, and One Node Per Workload says that it emits one node
per container rather than one per pod.
karldebisschop
force-pushed
the
workload-indexing-and-node-naming
branch
from
August 10, 2026 23:05
dbeba93 to
e7e425d
Compare
A tag selector read its attribute by indexing and appended the value as-is, so two ordinary configurations took the entire node source down: no JSON, no nodes, a traceback instead. No value for this pod. Either the attribute is absent -- a label only some pods carry -- which raised KeyError, or it is present but unset, which raised TypeError when the tag list was joined. default:status_message is None on every healthy pod, so that selector failed on an entirely normal cluster. A cluster where 99 of 100 pods carry a label produced nothing at all. A value that is not a string. terminated is a bool, and joining it raised TypeError even with every pod labelled. Resolve the attribute with .get() and omit the tag when there is no value; str() it when there is. A pod that lacks the attribute now carries one tag fewer.
karldebisschop
force-pushed
the
workload-indexing-and-node-naming
branch
from
August 11, 2026 11:33
fe4a51a to
7179190
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A Deployment with three replicas produces three Rundeck nodes. For most jobs
that is exactly right — you want the command on every pod. But a large class of
maintenance work must run once per deployment, not once per pod, because the
replicas share state: a database migration, a cache rebuild, a data backfill, a
scheduled cleanup, anything that writes to the backend the replicas have in
common.
Running such a job on every pod is wasteful in the best case. In the worst case
it is a correctness bug: three replicas executing the same migration
concurrently against one database is a race that testing on a single-replica
environment will never reveal, and that surfaces in production as partial
writes, duplicate rows, or deadlocks.
Why this cannot be expressed today
The resource model names each node
<pod name>-<container name>, and exposesthe pod's labels. Neither gives you a way to select exactly one pod of a
deployment, durably:
billing-7f4b8bbfc6-2q2hdcontains apod-template-hash and a random suffix, both of which change on every rollout.
A node filter naming a specific pod stops matching the next time you deploy.
labels:app = billingselects all three replicas — which is the situation we are trying to avoid.
There is currently no stable identifier for "the deployment this pod belongs
to", and no deterministic way to pick one of its pods.
What this adds
default:workload— the owning controller with the Deploymentpod-template-hash stripped, so
billing-7f4b8bbfc6-2q2hdreportsbilling.Derived from the pod's controlling ownerReference rather than by parsing the
pod name, falling back to the first reference and then the pod name, so
StatefulSets, DaemonSets, Jobs and bare pods all resolve sensibly. Unlike the
pod name, this survives rollouts.
index— an ordinal withinnamespace/workload/container, assigned only topods that are actually emitted, so a filtered-out Terminated pod does not
consume a number and
index == 1always matches. Because the counter is scopedper workload, one filter selects one pod from each deployment:
One Node Per Workload (
RD_CONFIG_ONE_PER_WORKLOAD, default off) — does thesame thing at the model level for operators who would rather not rely on every
job remembering the filter. The node count then stays stable as replicas scale.
Node Name Format — node names become a
${attribute}template, so they canbe built from the stable identity rather than the ephemeral pod name, for
example
${default:namespace}-${default:workload}. The full pod name remainsavailable as
default:namefor traceability.Custom Mapping sources now resolve against any node attribute rather than the
default:*set alone. That makes a label with an awkward key usable in atemplate —
service.selector=labels:app.kubernetes.io/name, then${service}—and fixes mappings from
labels:silently producing nothing.Compatibility
Both new behaviors are opt-in and the defaults reproduce today's output:
${default:name}-${default:container_name}, which isthe historical
<pod name>-<container name>exactly.nodenamefrom Custom Mapping or Default attributes still winsover the template, so the documented
nodename.selector=default:namekeepsworking. Renaming a node makes Rundeck treat it as a different node, so this
ordering matters.
is enabled.
default:workloadandindexare additive attributes.An unset attribute renders as an empty string rather than the literal
None,so a pending pod with no
podIPcannot producemy-app-None— or give severalpending pods the same name.
Testing
The resource-model suite goes from 33 to 49 tests, covering workload derivation
and controller-owner preference, per-workload numbering including the
filtered-pod case, the one-per-workload toggle, template rendering with labels
and unset attributes, explicit-nodename precedence, and mapping from a label.