fix: accept UTF-8 BOM in SKILL.md frontmatter - #99
Conversation
Schema read utf-8 and anchored frontmatter at ^---, so a Notepad BOM made a valid skill fail HIGH while unicode called the same BOM benign. Read utf-8-sig and allow an optional BOM before the opening fence. Fixes NVIDIA#91 Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
rng1995
left a comment
There was a problem hiding this comment.
The focused parser/schema suites pass (46 tests), with Ruff and diff checks clean. One Tier 1 consumer still fails to parse the newly accepted BOM form, leaving inconsistent results; details are inline. The shared Gitleaks failure is unrelated branch history.
Schema already accepted a BOM-prefixed SKILL.md, but quality still matched ^--- and skipped XML-tag checks. Read utf-8-sig and reuse the shared FRONTMATTER_PATTERN. Fixes NVIDIA#91 Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
# Conflicts: # CHANGELOG.md
|
Merged main. BOM handling in the quality parser should be covered now. Ready for re-review. |
rng1995
left a comment
There was a problem hiding this comment.
Re-reviewed the current head. BOM-prefixed frontmatter is now handled consistently by schema and quality parsing, so the original quality-gate mismatch is fixed. The focused frontmatter, schema, and quality suite passed (181 tests), with Ruff and diff checks clean. Approved. GitHub currently reports a CHANGELOG-only merge conflict, so the branch still needs an update before it can merge.
|
|
||
| try: | ||
| content = file_path.read_text(encoding="utf-8") | ||
| content = file_path.read_text(encoding="utf-8-sig") |
There was a problem hiding this comment.
[P1] Make security scanning honor BOM-aware frontmatter
I took another final review pass and found one remaining issue that should be fixed before merge. This utf-8-sig read makes BOM-prefixed frontmatter valid here, but _scan_file_for_pii() still reads the raw file with UTF-8 and _frontmatter_author_emails() requires the first raw line to equal ---. In a fresh real-CLI reproduction, byte-identical plain files exit 0 while BOM-prefixed files exit 1 with a HIGH finding for the valid metadata.author email, for both LF and CRLF. Can we make the security path BOM-aware too and add regression coverage for both line endings before we merge?
There was a problem hiding this comment.
Good catch. Frontmatter and quality parsing accept BOM now, but the security scan path still reads the raw first line in _frontmatter_author_emails(). I will wire that through the same BOM-aware read and add LF/CRLF regression coverage next.
There was a problem hiding this comment.
Fixed at c4e249d. _scan_file_for_pii now reads with utf-8-sig and _is_frontmatter_delimiter treats a BOM-prefixed opener as valid, so metadata.author emails stay exempt. Added LF and CRLF regression coverage in test_security.
|
Pushed the security-scan BOM fix at c4e249d. Ready for re-review. |
|
Thanks for the update. The current DCO check is failing because these two commits do not contain Signed-off-by trailers:
Please rewrite or amend both commits with your own Signed-off-by line, then force-push the updated branch using --force-with-lease. The DCO check and CI can rerun on the new head. |
Read SKILL manifests with utf-8-sig and treat BOM-prefixed --- fences as valid frontmatter so metadata.author emails are exempt in LF and CRLF files. Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
5ce9931 to
1cf459b
Compare
Summary
A valid SKILL.md that only differs by a UTF-8 BOM failed schema with HIGH frontmatter_format, while the unicode check called that BOM benign.
I read manifests with utf-8-sig (same as plugin JSON) and allow an optional BOM before the opening ---. Fixes #91.
Verification
make lintmake testmake buildRelease Impact
CHANGELOG.md