Skip to content

feat: support standalone HTML documents - #53

Open
xianjianlf2 wants to merge 3 commits into
firecrawl:mainfrom
xianjianlf2:feat/standalone-html
Open

feat: support standalone HTML documents#53
xianjianlf2 wants to merge 3 commits into
firecrawl:mainfrom
xianjianlf2:feat/standalone-html

Conversation

@xianjianlf2

@xianjianlf2 xianjianlf2 commented Aug 7, 2026

Copy link
Copy Markdown

Summary

Closes #52.

  • Add standalone HTML detection from <!doctype html> or an <html> root, including HTML exports saved with a .doc extension.
  • Reuse the EPUB HTML-to-model walker for headings, formatting, links, tables, lists, CSS visibility, and external images.
  • Treat HTML void elements such as unclosed <meta> and <img> as empty elements in the shared DOM parser.
  • Expose html / .html / .htm through the Rust, Node, Python, and WASM APIs and update docs/tests.
  • MHTML remains out of scope for this change.

Validation

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo test --locked
  • npm test
  • python -m unittest discover -s python/tests
  • node --test wasm/test.mjs

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@xianjianlf2

Copy link
Copy Markdown
Author

Follow-up: synchronized node/package-lock.json so the CI npm ci step now succeeds.

Local validation after the follow-up commit:

  • npm ci --ignore-scripts
  • npm test (13/13)
  • cargo test --locked
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • Python and WASM binding tests

The upstream CI run is currently action_required, which requires a maintainer of firecrawl/anydoc to approve the fork workflow.

@xianjianlf2

Copy link
Copy Markdown
Author

Merged current origin/main into feat/standalone-html and resolved the node/package.json release conflict by keeping version 0.1.8 while preserving the HTML support description.

Validation after merge:

  • npm ci --ignore-scripts
  • npm test (13/13)
  • cargo test --locked (212 unit + robustness + snapshots passed; 1 samples test ignored as expected)
  • cargo clippy --workspace --all-targets --all-features -- -D warnings

@xianjianlf2

Copy link
Copy Markdown
Author

Rechecked current head c8a1b73; no code update was needed.

Local validation:

  • cargo test --locked (unit/robustness/snapshots passed; samples_sweep remains ignored)
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • npm --prefix node ci --ignore-scripts
  • npm --prefix node run build
  • npm --prefix node test (13 passed)
  • git diff --check

Note: npm test fails before npm run build because the local native binding artifact is not present; after the repo's build step it passes. GitHub currently shows cubic success and mergeStateStatus=UNSTABLE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Support for HTML / MHTML files (e.g. Jira exports)

1 participant