Conversation
Demand execution batch jobs failed at container start with:
CannotStartContainerError: ... Failed to create self-signed client-side
certificate ... File name too long
This is not a NAME_MAX/PATH_MAX limit. The EFS volume name flows through a
chain of generated names and lands in an OpenSSL buffer:
batch job definition name -> ECS task family
-> docker volume name ecs-<family>-<rev>-<volume>-<20 hex>
-> efs-utils TLS state dir
-> openssl CA database /var/run/efs/<state dir>/database/index.txt
openssl guards that path against a hard-coded 256 byte stack buffer (BSIZE in
apps/lib/apps.c), which caps the database path at 246 chars and, working back
up the chain, caps family + revision + volume name at 139. A volume name of
"dev-de-core-opt-fsap-0acb9f234e1b57786-scratch-vol" (50 chars) alongside a
92 char job definition name came to 143 -- over by 4.
Adds common/naming.py:
* condense_str(value, max_length, delimiter, hash_length) -- shortens a value
to a budget, keeping a readable prefix and appending a hash of the FULL
original so values sharing a prefix stay distinct. A value that already
fits is returned unchanged, so adopting it renames nothing.
* build_efs_volume_name(...) -- leads with the mount path basename
(scratch/shared/tmp) and takes uniqueness from a hash over the file system,
access point and mount path rather than spelling them all out. 50 -> 16 chars.
* check_ecs_volume_component_budget(...) -- raises while the job definition is
being built rather than minutes later at container start, where the error
names nothing responsible.
Capping execution_type would also have worked, but pushes an OpenSSL buffer
size onto callers; a descriptive execution type should not break EFS mounting.
With this change a 40 char execution_type fits comfortably (129/139) where the
20 char one previously failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #42 +/- ##
==========================================
+ Coverage 90.80% 90.87% +0.07%
==========================================
Files 26 27 +1
Lines 1446 1480 +34
Branches 137 143 +6
==========================================
+ Hits 1313 1345 +32
- Misses 96 97 +1
- Partials 37 38 +1
🚀 New features to boost your workflow:
|
Collaborator
Author
|
Superseded. Relocated per review:
Root-cause analysis and the character budget are preserved in the new commit message and the module docstring. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Demand execution batch jobs fail at container start, before any user code runs:
Root cause
Not a
NAME_MAX/PATH_MAXlimit — OpenSSL's hard-coded 256-byte stack buffer. The EFS volume name flows through a chain of generated names:openssl caguards that path withif (j + 6 >= BSIZE)whereBSIZEis 256 (apps/lib/apps.c), so the database path must be ≤ 246 chars. Working back up the chain:We were at 143 — over by exactly 4. The
/database/subdirectory is the easy piece to miss:config.confsits at 243 and writes fine, which is why the log shows the config being read and the certificate signed before it dies, and whyserial(247, guarded byj + 1) rotated successfully whileindex.txtdid not.Nothing fails at registration. It fails minutes later, in a message that names none of the code that chose the name.
Changes
New
common/naming.py:condense_str(value, max_length, delimiter, hash_length)— the general primitive. Shortens a value to a budget, keeping as much readable prefix as possible and appendingdelimiter+ a hash of the full original value, so two values sharing a long prefix still condense differently. A value that already fits is returned unchanged, so adopting this renames nothing. Deterministic, because an unstable name would register a new job definition revision on every run. Raises rather than silently truncating when the budget cannot fit the suffix plus a prefix character.build_efs_volume_name(...)— leads with the mount path basename (scratch/shared/tmp) so volumes stay identifiable, and takes uniqueness from a hash over (file system, access point, mount path) instead of spelling all three out. 50 → 16 chars.check_ecs_volume_component_budget(...)— raises while the job definition is being built, so an over-long name is an immediate legible error instead of aCannotStartContainerError. Reserves 3 revision digits, so a name cannot pass at revision 9 and fail at revision 10.Why not just cap
execution_typeCapping it at 8 chars also clears the budget and needs no deploy — it's what unblocked testing today. But it pushes an OpenSSL buffer size onto callers, and a descriptive execution type shouldn't be able to break EFS mounting. Its hard cap would be 16 chars, and 16 lands on exactly zero margin.
execution_typeocsdv452-filter-test(the failure)a-fairly-descriptive-execution-type-name(40)Testing
make format && make lint && make test— 177 passed, ruff and mypy clean. 25 new tests covering the no-op-when-it-fits property, determinism, prefix-collision resistance, budget validation, ECS charset safety, and uniqueness across access points and mount paths.Two existing tests asserted the old volume name format and were updated.
test_context_managernow asserts structure (the ids are moto-generated, so the hash can't be hardcoded) plus the invariant that the mount point'ssourceVolumematches the registered volume name;test_scaffoldingpins the new literal, since its ids are fixed.Notes
mainand passing against published core/aws-utils, so it can merge without waiting for R1/R2.containerPropertiesonly. Every existing family registers one new revision on its next run.dev-de— it adds{name}-part{i}EFS ecosystems with longer names. This change helps there too, but recompute before assuming margin.transitEncryption(sends all EFS traffic cleartext across the VPC — "fixes" this by deleting the failing code path).🤖 Generated with Claude Code