feat(pypi): add pyproject.toml support to IntelliJ plugin#244
feat(pypi): add pyproject.toml support to IntelliJ plugin#244ruromero wants to merge 7 commits intoredhat-developer:mainfrom
Conversation
ruromero
left a comment
There was a problem hiding this comment.
Verification Report for TC-3853
| Check | Result | Details |
|---|---|---|
| Scope Containment | WARN | All 9 task-listed files present. plugin.xml is out-of-scope per file lists but required by Implementation Notes for annotator/inspection registration |
| Diff Size | PASS | 514 additions, 10 deletions, 10 files — proportionate to 3 new files + 7 modifications |
| Commit Traceability | PASS | Single commit references TC-3853 in body ("Implements TC-3853"), follows Conventional Commits, includes Assisted-by: Claude Code trailer |
| Sensitive Patterns | PASS | No passwords, secrets, API keys, or private keys detected |
| CI Status | WARN | build-linux, build-macos, main, SonarCloud pass; verify and build-windows pending |
| Acceptance Criteria | PASS | 6 of 6 criteria met (see details below) |
| Verification Commands | N/A | No verification commands specified in task |
Acceptance Criteria Details
| # | Criterion | Result | Evidence |
|---|---|---|---|
| 1 | Opening pyproject.toml triggers component analysis with inline diagnostics |
PASS | PyprojectCAAnnotator extends CAAnnotator, checks filename, registered in plugin.xml |
| 2 | Vulnerable packages in [project.dependencies] are highlighted |
PASS | parsePep621Dependencies() parses PEP 508 strings from project table's dependencies array |
| 3 | Vulnerable packages in [tool.poetry.dependencies] are highlighted |
PASS | parsePoetryDependencies() extracts name/version from poetry table key-value pairs |
| 4 | Quick-fix intention action available for recommended versions | PASS | PyprojectCAIntentionAction extends CAIntentionAction, handles both PEP 621 and Poetry formats |
| 5 | Stack analysis (RHDA Report) available via right-click | PASS | SaAction.java includes pyproject.toml in supported filenames; SaUtils.java maps it to python |
| 6 | README.md updated to list pyproject.toml as supported |
PASS | Multiple references added: prerequisites, component analysis, exhortignore, dev dependency sections |
Overall: WARN
Two CI checks (verify, build-windows) are still pending. One out-of-scope file (plugin.xml) is included but is justified by the Implementation Notes requirement to register the annotator and inspection. All acceptance criteria are satisfied. No blocking issues found.
This comment was AI-generated by sdlc-workflow/verify-pr v0.4.3.
|
/review |
PR Reviewer Guide 🔍(Review updated until commit a1c1727)Here are some key observations to aid the review process:
|
|
/review |
|
Persistent review updated to latest commit d420e28 |
|
@soul2zimate it is missing the license support introduced here #243 The license quick fix for pyproject.toml files can be introduced after your PR is merged. |
|
@ruromero roger! Let's hold any feature PR for the moment, I just want to include bug fix PR to address issue reported by QE at this stage. |
soul2zimate
left a comment
There was a problem hiding this comment.
In plugin.xml, there is already localInspection and externalAnnotator for TOML which supports Cargo.toml. This PR defines again for pyproject.toml. Does it cause conflict and false detection?
The annotators target different files so there is no conflict |
But IntelliJ still invokes both I suggest to add a filename guard for |
Done. Thanks for the suggestion. |
Review Summary by QodoAdd pyproject.toml support for Python dependency analysis in IntelliJ plugin
WalkthroughsDescription• Add pyproject.toml support for Python dependency analysis - Parse PEP 621 [project.dependencies] and [project.optional-dependencies] - Parse Poetry [tool.poetry.dependencies] format • Create PyprojectCAAnnotator, PyprojectCAInspection, and PyprojectCAIntentionAction classes • Update existing classes to recognize pyproject.toml as supported manifest • Add Cargo.toml validation and pyproject.toml documentation Diagramflowchart LR
A["pyproject.toml"] -->|"Parse PEP 621 & Poetry"| B["PyprojectCAAnnotator"]
B -->|"Extract dependencies"| C["Dependency Map"]
C -->|"Highlight vulnerabilities"| D["PyprojectCAInspection"]
D -->|"Quick fix available"| E["PyprojectCAIntentionAction"]
F["SaAction/SaUtils"] -->|"Register manifest"| A
G["ApiService"] -->|"Apply Python settings"| A
H["CAAnnotator/SAIntentionAction"] -->|"Map to python"| A
File Changes1. src/main/java/org/jboss/tools/intellij/componentanalysis/pypi/PyprojectCAAnnotator.java
|
Code Review by Qodo
|
| "requirements.txt", | ||
| "build.gradle", | ||
| "build.gradle.kts", | ||
| "Cargo.toml" | ||
| "Cargo.toml", | ||
| "pyproject.toml" |
There was a problem hiding this comment.
1. Missing pyproject.toml recognition tests 📎 Requirement gap ☼ Reliability
The PR updates SaAction/SaUtils to recognize pyproject.toml, but no automated tests were added/updated to verify this manifest gating behavior. This violates the checklist requirement to add tests for pyproject.toml recognition in these entry points.
Agent Prompt
## Issue description
`pyproject.toml` support was added to stack-analysis manifest gating, but the PR does not include automated tests to verify recognition in `SaAction` and `SaUtils` as required.
## Issue Context
The compliance checklist requires tests to prevent regressions where UI actions/SA routing stop triggering for `pyproject.toml`.
## Fix Focus Areas
- src/main/java/org/jboss/tools/intellij/stackanalysis/SaAction.java[40-44]
- src/main/java/org/jboss/tools/intellij/stackanalysis/SaUtils.java[27-31]
- src/main/java/org/jboss/tools/intellij/stackanalysis/SaUtils.java[61-66]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
[sdlc-workflow/verify-pr] Classified as code change request — the Jira task Test Requirements explicitly require "Test that pyproject.toml is recognized as a supported manifest in SaAction and SaUtils." Investigation confirms no tests exist for SaUtils/SaAction (pre-existing gap across all ecosystems). Combined sub-task created for all missing pyproject.toml tests.
| private void parsePep621Dependencies(TomlTable projectTable, Set<String> ignoredDeps, | ||
| Map<Dependency, List<PsiElement>> resultMap) { | ||
| List<TomlKeyValue> keyValues = PsiTreeUtil.getChildrenOfTypeAsList(projectTable, TomlKeyValue.class); | ||
| for (TomlKeyValue kv : keyValues) { | ||
| if (!"dependencies".equals(kv.getKey().getText())) { | ||
| continue; | ||
| } | ||
| TomlValue value = kv.getValue(); | ||
| if (!(value instanceof TomlArray array)) { | ||
| continue; | ||
| } | ||
| for (TomlValue element : array.getElements()) { | ||
| if (!(element instanceof TomlLiteral literal)) { | ||
| continue; | ||
| } | ||
| String depString = unquote(literal.getText()); | ||
| String name = extractPep508Name(depString); | ||
| if (name == null || ignoredDeps.contains(name.toLowerCase())) { | ||
| continue; | ||
| } | ||
| String version = extractPep508Version(depString); | ||
| Dependency dp = new Dependency(PYPI, null, name.toLowerCase(), version); | ||
| resultMap.computeIfAbsent(dp, k -> new LinkedList<>()).add(literal); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Parses PEP 621 [project.optional-dependencies] table. | ||
| * Format: dev = ["pytest>=7.0", "black"], security = ["certifi>=2023.7"] | ||
| */ | ||
| private void parseOptionalDependencies(TomlTable optDepsTable, Set<String> ignoredDeps, | ||
| Map<Dependency, List<PsiElement>> resultMap) { | ||
| List<TomlKeyValue> groups = PsiTreeUtil.getChildrenOfTypeAsList(optDepsTable, TomlKeyValue.class); | ||
| for (TomlKeyValue group : groups) { | ||
| TomlValue value = group.getValue(); | ||
| if (!(value instanceof TomlArray array)) { | ||
| continue; | ||
| } | ||
| for (TomlValue element : array.getElements()) { | ||
| if (!(element instanceof TomlLiteral literal)) { | ||
| continue; | ||
| } | ||
| String depString = unquote(literal.getText()); | ||
| String name = extractPep508Name(depString); | ||
| if (name == null || ignoredDeps.contains(name.toLowerCase())) { | ||
| continue; | ||
| } | ||
| String version = extractPep508Version(depString); | ||
| Dependency dp = new Dependency(PYPI, null, name.toLowerCase(), version); | ||
| resultMap.computeIfAbsent(dp, k -> new LinkedList<>()).add(literal); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Parses Poetry [tool.poetry.dependencies] table. | ||
| * Format: anyio = "^3.6.2" or anyio = {version = "^3.6.2", optional = true} | ||
| */ | ||
| private void parsePoetryDependencies(TomlTable poetryTable, Set<String> ignoredDeps, | ||
| Map<Dependency, List<PsiElement>> resultMap) { | ||
| List<TomlKeyValue> keyValues = PsiTreeUtil.getChildrenOfTypeAsList(poetryTable, TomlKeyValue.class); | ||
| for (TomlKeyValue kv : keyValues) { | ||
| String name = normalizeKeyName(kv.getKey().getText()); | ||
| if ("python".equalsIgnoreCase(name) || ignoredDeps.contains(name.toLowerCase())) { | ||
| continue; | ||
| } | ||
| String version = extractPoetryVersion(kv.getValue()); | ||
| Dependency dp = new Dependency(PYPI, null, name.toLowerCase(), version); | ||
| resultMap.computeIfAbsent(dp, k -> new LinkedList<>()).add(kv); | ||
| } | ||
| } |
There was a problem hiding this comment.
2. Pyproject dependency parsing untested 📎 Requirement gap ☼ Reliability
The PR adds new dependency extraction logic for PEP 621 [project.dependencies] and Poetry [tool.poetry.dependencies], but no automated tests were added to validate correct extraction. This violates the checklist requirements for tested parsing of both formats.
Agent Prompt
## Issue description
Dependency extraction for `pyproject.toml` (PEP 621 and Poetry) is implemented but lacks automated tests.
## Issue Context
The checklist requires tests verifying that `[project.dependencies]` (PEP 508 strings) and `[tool.poetry.dependencies]` (name->version mappings) are correctly extracted from TOML PSI.
## Fix Focus Areas
- src/main/java/org/jboss/tools/intellij/componentanalysis/pypi/PyprojectCAAnnotator.java[116-187]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
[sdlc-workflow/verify-pr] Classified as code change request — the Jira task Test Requirements explicitly require "Test that PyprojectCAAnnotator extracts dependencies from PEP 621 format" and Poetry format. No tests exist. Combined sub-task created for all missing pyproject.toml tests.
| @Override | ||
| protected void updateVersion(@NotNull Project project, Editor editor, PsiFile file, String version) { | ||
| if (version == null) { | ||
| return; | ||
| } | ||
|
|
||
| if (element instanceof TomlLiteral literal) { | ||
| // PEP 621: string literal in dependencies array like "anyio==3.6.2" | ||
| String depString = PyprojectCAAnnotator.unquote(literal.getText()); | ||
| String name = PyprojectCAAnnotator.extractPep508Name(depString); | ||
| if (name != null) { | ||
| String extras = PyprojectCAAnnotator.extractPep508Extras(depString); | ||
| String markers = PyprojectCAAnnotator.extractPep508Markers(depString); | ||
| StringBuilder newDep = new StringBuilder(name); | ||
| if (extras != null) { | ||
| newDep.append(extras); | ||
| } | ||
| newDep.append("==").append(version); | ||
| if (markers != null) { | ||
| newDep.append(" ").append(markers); | ||
| } | ||
| replaceVersionLiteral(project, file, literal, newDep.toString()); | ||
| } | ||
| } else if (element instanceof TomlKeyValue keyValue) { | ||
| // Poetry: key-value pair like anyio = "^3.6.2" | ||
| TomlLiteral valueLiteral = findVersionLiteral(keyValue); | ||
| if (valueLiteral != null) { | ||
| replaceVersionLiteral(project, file, valueLiteral, "==" + version); | ||
| } | ||
| } |
There was a problem hiding this comment.
3. Pyproject quick-fix untested 📎 Requirement gap ☼ Reliability
A new PyprojectCAIntentionAction is introduced to apply recommended versions, but no automated tests were added to verify its availability and correct version application for pyproject.toml. This violates the checklist requirement for intention action tests.
Agent Prompt
## Issue description
`PyprojectCAIntentionAction` was added but lacks tests verifying (1) it is offered for `pyproject.toml` and (2) it correctly updates versions for both PEP 621 literals and Poetry key/value entries.
## Issue Context
The repo already uses IntelliJ `BasePlatformTestCase` fixtures for intention-action tests (e.g., cargo). Add analogous tests for pyproject.
## Fix Focus Areas
- src/main/java/org/jboss/tools/intellij/componentanalysis/pypi/PyprojectCAAnnotator.java[93-96]
- src/main/java/org/jboss/tools/intellij/componentanalysis/pypi/PyprojectCAIntentionAction.java[37-66]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
[sdlc-workflow/verify-pr] Classified as code change request — the Jira task Test Requirements explicitly require "Test that PyprojectCAIntentionAction is available for pyproject.toml files." No tests exist. Combined sub-task created for all missing pyproject.toml tests.
| if ("requirements.txt".equals(manifestName) || "pyproject.toml".equals(manifestName)) { | ||
| if (settings.pythonMatchManifestVersions) { | ||
| System.setProperty("MATCH_MANIFEST_VERSIONS", "true"); | ||
| } else { | ||
| System.setProperty("MATCH_MANIFEST_VERSIONS","false"); | ||
| } | ||
| } | ||
| if (!"go.mod".equals(manifestName) && !"requirements.txt".equals(manifestName)) { | ||
| if (!"go.mod".equals(manifestName) && !"requirements.txt".equals(manifestName) && !"pyproject.toml".equals(manifestName)) { | ||
| System.clearProperty("MATCH_MANIFEST_VERSIONS"); | ||
| } |
There was a problem hiding this comment.
4. Apiservice pyproject tests missing 📎 Requirement gap ☼ Reliability
ApiService is updated to treat pyproject.toml like requirements.txt for component analysis submission and Python-specific request properties, but no automated tests were added to verify these settings are applied. This violates the checklist requirement for ApiService tests covering Python-specific behavior for pyproject.toml.
Agent Prompt
## Issue description
`ApiService` now includes `pyproject.toml` in Python-specific manifest handling, but there are no automated tests to confirm the correct request properties/system properties are set/cleared.
## Issue Context
Checklist requires parity with `requirements.txt` behavior (e.g., `MATCH_MANIFEST_VERSIONS` and Python virtualenv-related settings handling) for `pyproject.toml`.
## Fix Focus Areas
- src/main/java/org/jboss/tools/intellij/exhort/ApiService.java[101-106]
- src/main/java/org/jboss/tools/intellij/exhort/ApiService.java[259-268]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
[sdlc-workflow/verify-pr] Classified as code change request — the Jira task Test Requirements explicitly require "Test that ApiService applies Python-specific settings for pyproject.toml." No tests exist. Combined sub-task created for all missing pyproject.toml tests.
| private String findAssociatedDependency(PsiFile file, PsiComment comment) { | ||
| // Check if comment is on same line as a TomlKeyValue | ||
| TomlKeyValue keyValue = findKeyValueOnSameLine(file, comment); | ||
| if (keyValue != null) { | ||
| String keyName = keyValue.getKey().getText(); | ||
| // If in [tool.poetry.dependencies], the key IS the dependency name | ||
| if (isInPoetryDependencies(keyValue)) { | ||
| return normalizeKeyName(keyName); | ||
| } | ||
| // If key is "dependencies" in [project], check if on same line as an array element | ||
| // For PEP 621 array entries, the ignore comment is typically on the same line as the string literal | ||
| } | ||
|
|
||
| // Check array elements — PEP 621 format where comment is on the same line as a literal in the array | ||
| TomlLiteral literal = findLiteralOnSameLine(file, comment); | ||
| if (literal != null) { | ||
| String text = unquote(literal.getText()); | ||
| String name = extractPep508Name(text); | ||
| if (name != null) { | ||
| return name; | ||
| } | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| private boolean isInPoetryDependencies(TomlKeyValue keyValue) { | ||
| TomlTable parentTable = PsiTreeUtil.getParentOfType(keyValue, TomlTable.class); | ||
| if (parentTable != null) { | ||
| String tablePath = getTablePath(parentTable.getHeader()); | ||
| return "tool.poetry.dependencies".equals(tablePath); | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| private TomlKeyValue findKeyValueOnSameLine(PsiFile file, PsiComment comment) { | ||
| int commentLine = getLineNumber(file, comment); | ||
| Collection<TomlKeyValue> allKeyValues = PsiTreeUtil.collectElementsOfType(file, TomlKeyValue.class); | ||
| for (TomlKeyValue keyValue : allKeyValues) { | ||
| int keyValueLine = getLineNumber(file, keyValue); | ||
| if (commentLine == keyValueLine) { | ||
| return keyValue; | ||
| } | ||
| } | ||
| return null; | ||
| } |
There was a problem hiding this comment.
5. Poetry ignore comment misbind 🐞 Bug ≡ Correctness
PyprojectCAAnnotator.findAssociatedDependency() can associate a trustify-da-ignore comment on a Poetry inline-table dependency line with an inner key (e.g., version) instead of the dependency name, so the dependency is not actually ignored and diagnostics keep appearing.
Agent Prompt
### Issue description
`trustify-da-ignore` lookup in `PyprojectCAAnnotator` can return the wrong TOML key on Poetry inline-table dependency lines because it scans all `TomlKeyValue` elements on the same line and returns the first match.
### Issue Context
Poetry inline-table deps (supported by this PR) place nested key/values (`version`, `optional`, etc.) on the same line as the outer dependency key, so the ignore comment must be associated with the *outer* dependency key, not inner entries.
### Fix Focus Areas
- src/main/java/org/jboss/tools/intellij/componentanalysis/pypi/PyprojectCAAnnotator.java[218-263]
### Suggested implementation direction
- In `findAssociatedDependency`, when handling Poetry, prefer the `TomlKeyValue` whose **direct parent is the `TomlTable` entry list** (i.e., the dependency entry), not a nested inline-table key.
- Alternatively, derive the associated dependency by walking PSI from the comment to the nearest enclosing/top-level `TomlKeyValue` on that line, then verify it’s in `[tool.poetry.dependencies]` before returning its key name.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
[sdlc-workflow/verify-pr] Classified as code change request — investigation confirms a variant of this bug exists for multi-line Poetry inline-table dependencies where the trustify-da-ignore comment is placed on an inner key-value line. The single-line case works correctly due to PSI depth-first traversal order. Sub-task created for the multi-line case fix.
Add component analysis support for pyproject.toml manifest files, enabling inline vulnerability diagnostics for Python projects using PEP 621 ([project.dependencies]) and Poetry ([tool.poetry.dependencies]) formats. New classes: - PyprojectCAAnnotator: extracts dependencies from TOML PSI structure - PyprojectCAIntentionAction: quick-fix for version updates - PyprojectCAInspection: inspection registration Modified files: - SaAction, SaUtils: recognize pyproject.toml as supported manifest - CAAnnotator, SAIntentionAction: add pyproject.toml to python mapping - ApiService: apply Python-specific settings for pyproject.toml - plugin.xml: register annotator and inspection for TOML language - README.md: document pyproject.toml support Implements TC-3853 Assisted-by: Claude Code
- Fix PEP 621 annotation targeting: use individual TomlLiteral elements instead of parent TomlKeyValue so each dependency gets its own highlight - Fix quick-fix replacement: preserve package name in PEP 508 strings (e.g. "anyio==3.6.2" → "anyio==4.0.0" instead of just "==4.0.0") - Add [project.optional-dependencies] parsing for PEP 621 optional groups - Update README to reflect optional-dependencies are now analyzed Implements TC-3853 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Assisted-by: Claude Code
…on extraction The version extraction regex was matching across the ';' environment marker boundary, and the quick fix was dropping extras and markers when rewriting the dependency string. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ruben Romero Montes <rromerom@redhat.com>
Signed-off-by: Ruben Romero Montes <rromerom@redhat.com>
|
/review |
|
Persistent review updated to latest commit a1c1727 |
Verification Report for TC-3853 (commit a1c1727)
Overall: WARNSummary of issues requiring attention:
This comment was AI-generated by sdlc-workflow/verify-pr v0.6.1. |
Add 35 tests covering PEP 508 utility methods, PEP 621 and Poetry dependency extraction, optional dependencies, intention action version updates, availability checks, and SaAction/SaUtils manifest recognition. Implements TC-4172 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Assisted-by: Claude Code
…e tables When a trustify-da-ignore comment was placed on an inner key of a multi-line Poetry inline table, findAssociatedDependency() returned the inner key name (e.g., "version") instead of the outer dependency name. Added resolvePoetryDependencyEntry() to walk up from inner TomlKeyValue through TomlInlineTable to the actual dependency entry. Resolves: TC-4173 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|



Summary
pyproject.tomlmanifest files in the IntelliJ Dependency Analytics plugin[project.dependencies]and Poetry[tool.poetry.dependencies]sections for vulnerability scanningPyprojectCAAnnotator,PyprojectCAInspection, andPyprojectCAIntentionActionfor TOML languageSaAction,SaUtils,CAAnnotator,SAIntentionAction, andApiServiceto recognizepyproject.tomlpyproject.tomlsupport including ignore patterns and dev dependency exclusionImplements TC-3853
Test plan
pyproject.tomlappears in supported manifest files listpyproject.tomlwith PEP 621 dependencies and verify inline diagnostics appearpyproject.tomlwith Poetry dependencies and verify inline diagnostics appearpyproject.tomltrustify-da-ignorecomment works to suppress diagnostics./gradlew build./gradlew test --tests "org.jboss.tools.intellij.componentanalysis.pypi.*")🤖 Generated with Claude Code