You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A successful documentation build does not currently explain which source documents reached the package, which were intentionally excluded, or which were lost to read/parse failures. That makes missing documentation difficult to distinguish from an unsuccessful search.
There is already a useful skippedFiles count; this proposal extends it into actionable diagnostics rather than replacing it.
At upstream commit 149f112:
findMarkdownFiles() applies filename, directory, language and ignore filters. Directory read failures are caught without a diagnostic.
readLocalDocsFiles() deduplicates content and catches file-read failures without returning their paths or reasons.
buildPackage() counts split/parse failures, but does not expose the affected paths or failure reasons. A file producing no sections is also different from a parser throwing.
fix(context): only skip repo-meta filenames at the scan root #125 demonstrates the practical consequence: valid documentation named security.md can disappear while the build succeeds. That PR fixes a filtering rule; this issue makes similar omissions diagnosable throughout the pipeline.
Proposed behavior:
Return structured ingestion diagnostics alongside the existing build result. Distinguish intentional exclusions, duplicate content, unreadable files/directories, split/parse failures, files producing no sections, and successfully indexed documents.
The CLI should print a concise summary and offer detailed or machine-readable output with relative paths and reasons. Keep diagnostics in the result rather than requiring library code to log directly. Avoid flooding the report with unrelated source-code files when scanning a repository.
Acceptance criteria:
A small fixture with exclusions, duplicate documents, a read failure, a parse failure, and an empty document produces distinct, inspectable outcomes.
Git, ZIP, and HTML-index sources can report outcomes through the same vocabulary.
The summary distinguishes discovered documentation, selected files, indexed documents, and indexed sections.
Representative Markdown, HTML, and RST fixtures assert that expected headings and examples survive ingestion.
Unexpected loss can fail an explicit strict validation mode; intentional exclusions remain distinguishable from failures.
Related: #125 and #137. The objective is extraction visibility and fidelity; retrieval ranking can be evaluated separately.
A successful documentation build does not currently explain which source documents reached the package, which were intentionally excluded, or which were lost to read/parse failures. That makes missing documentation difficult to distinguish from an unsuccessful search.
There is already a useful
skippedFilescount; this proposal extends it into actionable diagnostics rather than replacing it.At upstream commit
149f112:findMarkdownFiles()applies filename, directory, language and ignore filters. Directory read failures are caught without a diagnostic.readLocalDocsFiles()deduplicates content and catches file-read failures without returning their paths or reasons.buildPackage()counts split/parse failures, but does not expose the affected paths or failure reasons. A file producing no sections is also different from a parser throwing.security.mdcan disappear while the build succeeds. That PR fixes a filtering rule; this issue makes similar omissions diagnosable throughout the pipeline.Proposed behavior:
Return structured ingestion diagnostics alongside the existing build result. Distinguish intentional exclusions, duplicate content, unreadable files/directories, split/parse failures, files producing no sections, and successfully indexed documents.
The CLI should print a concise summary and offer detailed or machine-readable output with relative paths and reasons. Keep diagnostics in the result rather than requiring library code to log directly. Avoid flooding the report with unrelated source-code files when scanning a repository.
Acceptance criteria:
Related: #125 and #137. The objective is extraction visibility and fidelity; retrieval ranking can be evaluated separately.