Skip to content

[TRTLLMINF-40][chore] Dedupe COMMON_SSH_OPTIONS to reference bloom's DEFAULT_CUSTOM_SSH_OPTIONS - #17706

Open
brnguyen2 wants to merge 2 commits into
NVIDIA:mainfrom
brnguyen2:user/brnguyen/dedupe-common-ssh-options
Open

[TRTLLMINF-40][chore] Dedupe COMMON_SSH_OPTIONS to reference bloom's DEFAULT_CUSTOM_SSH_OPTIONS#17706
brnguyen2 wants to merge 2 commits into
NVIDIA:mainfrom
brnguyen2:user/brnguyen/dedupe-common-ssh-options

Conversation

@brnguyen2

@brnguyen2 brnguyen2 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

jenkins/L0_Test.groovy defined COMMON_SSH_OPTIONS as a string literal that duplicated com.nvidia.bloom.Utils.DEFAULT_CUSTOM_SSH_OPTIONS in the bloom Jenkins shared library. This replaces the literal with a reference to the canonical constant:

COMMON_SSH_OPTIONS = Utils.DEFAULT_CUSTOM_SSH_OPTIONS

Utils (com.nvidia.bloom.Utils) is already imported (line 28), so no new import is added. All existing call sites that interpolate ${COMMON_SSH_OPTIONS} are unchanged.

Motivation

The two definitions had already drifted: bloom's DEFAULT_CUSTOM_SSH_OPTIONS gained -o ConnectTimeout=15 (from [TRTLLMINF-99], to stop SLURM frontend probes hanging), while this local literal still had no connect timeout. Referencing the canonical constant gives a single source of truth so the options can't diverge again.

Effect (not a no-op)

Because bloom's constant now carries ConnectTimeout=15, this dedupe adds a 15s connect timeout to the ssh/scp calls that use COMMON_SSH_OPTIONS in L0_Test.groovy (the result-download scp and the debug-VM ssh), bounding hung connects instead of relying on the OS default. This is safe:

  • The download scp already wraps in numRetries: 3, so a bounded-then-retried connect is strictly better than an unbounded hang.
  • COMMON_SSH_OPTIONS here is independent of bloom's SLURM-frontend-probe path (that path uses bloom's constant directly and has its own 30s backstop), so there is no interaction with the TRTLLMINF-99 probe invariant.

The other options (StrictHostKeyChecking, UserKnownHostsFile, TCPKeepAlive, ServerAliveInterval, ServerAliveCountMax) are unchanged.

Dev Engineer Review

  • Updated jenkins/L0_Test.groovy to use Utils.DEFAULT_CUSTOM_SSH_OPTIONS.
  • Removed the duplicate SSH option string.
  • Kept all existing call sites unchanged.
  • Added a waiver for TestGLM52NVFP4::test_nvfp4_nixl[cache_mgr_v1].
  • Referenced pending bug PR 17706.
  • No correctness, performance, API, configuration, error-handling, or regression concerns identified.

QA Engineer Review

  • Modified tests/integration/test_lists/waives.txt.
  • Added one waiver entry.
  • Removed no entries.
  • No test-db/ or qa/ files were modified.
  • No test code was added, modified, or removed.
  • Verdict: needs follow-up because CBTS coverage data is unavailable.
  • File a tracking bug for the CI-waived test(s) and record it with pr-babysitter waive set-bug NVIDIA/TensorRT-LLM#17706 <bug-url> — the waives entry carries a placeholder until then

@brnguyen2
brnguyen2 requested a review from a team as a code owner August 14, 2026 12:58
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e4228a0a-2b5f-4553-8676-f9e9f51e5c19

📥 Commits

Reviewing files that changed from the base of the PR and between 2462730 and 1984a29.

📒 Files selected for processing (2)
  • jenkins/L0_Test.groovy
  • tests/integration/test_lists/waives.txt
🚧 Files skipped from review as they are similar to previous changes (2)
  • jenkins/L0_Test.groovy
  • tests/integration/test_lists/waives.txt

Walkthrough

The PR uses the shared SSH options constant in Jenkins and adds a waiver for a GLM 5.2 NVFP4 NIXL integration test.

Changes

Test Configuration Updates

Layer / File(s) Summary
Use shared SSH options
jenkins/L0_Test.groovy
COMMON_SSH_OPTIONS now references Utils.DEFAULT_CUSTOM_SSH_OPTIONS.
Add NVFP4 test waiver
tests/integration/test_lists/waives.txt
The waiver list skips TestGLM52NVFP4::test_nvfp4_nixl[cache_mgr_v1] and references pending PR 17706.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 1984a

This change centralizes the SSH options and adds a bounded 15-second connection timeout to the affected CI transfers; no actionable merge-blocking risk remains.

Possibly related PRs

Suggested reviewers: yiqingy0

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the ticket, change type, and primary change to deduplicate COMMON_SSH_OPTIONS.
Description check ✅ Passed The description clearly explains the change, motivation, effect, and waived test tracking, but it omits an explicit Test Coverage section.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66282 [ run ] triggered by Bot. Commit: 9fd31e0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66282 [ run ] completed with state SUCCESS. Commit: 9fd31e0
/LLM/main/L0_MergeRequest_PR pipeline #53942 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66351 [ run ] triggered by Bot. Commit: 9fd31e0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66351 [ run ] completed with state SUCCESS. Commit: 9fd31e0
/LLM/main/L0_MergeRequest_PR pipeline #53994 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66372 [ run ] triggered by Bot. Commit: 9fd31e0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66372 [ run ] completed with state SUCCESS. Commit: 9fd31e0
/LLM/main/L0_MergeRequest_PR pipeline #54015 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66403 [ run ] triggered by Bot. Commit: 9fd31e0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66403 [ run ] completed with state FAILURE. Commit: 9fd31e0
/LLM/main/L0_MergeRequest_PR pipeline #54046 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66405 [ run ] triggered by Bot. Commit: 9fd31e0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66405 [ run ] completed with state FAILURE. Commit: 9fd31e0
/LLM/main/L0_MergeRequest_PR pipeline #54048 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

…DEFAULT_CUSTOM_SSH_OPTIONS

Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
…FP4::test_nvfp4_nixl[cache_mgr_v1] (pre-existing failure, tracking bug pending)

Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@brnguyen2
brnguyen2 force-pushed the user/brnguyen/dedupe-common-ssh-options branch from 2462730 to 1984a29 Compare August 15, 2026 03:29
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66426 [ run ] triggered by Bot. Commit: 1984a29 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66426 [ run ] completed with state FAILURE. Commit: 1984a29
/LLM/main/L0_MergeRequest_PR pipeline #54068 completed with status: 'UNSTABLE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants