BetterDocs is a popular WordPress knowledge base plugin with thousands of users worldwide. It helps create and organize documentation, FAQs, knowledge bases, and glossaries with beautiful layouts, AI-powered chatbots, and instant answers.
This project provides end-to-end frontend automation testing for BetterDocs using Playwright. It covers Gutenberg blocks, Elementor widgets, shortcodes, instant answer, chatbot interactions, permalink routing, API endpoints, and more — all without requiring admin/backend access.
- Node.js version 22 LTS
- npm
- Clone the repo
git clone <repo-url>
- Install NPM packages
npm install
- Create the
.envfile and provide necessary detailscp .env.example .env
- Install Playwright browsers
npx playwright install --with-deps
# Run all tests (headless)
npm test
# Run in headed mode (visible browser)
npx playwright test --headed
# Run by category
npm run test:blocks
npm run test:widgets
npm run test:shortcodes
npm run test:basic
npm run test:chatbot
npm run test:instant-answer
# View HTML report
npm run reportFor more examples, refer to the Playwright Documentation
tests/
├── blocks/ # Gutenberg block aria snapshot tests (39)
├── widgets/ # Elementor widget aria snapshot tests (34)
├── shortcodes/ # Shortcode aria snapshot tests (16)
├── basic/
│ ├── 404-checks/ # Page load / 404 verification across sites (10)
│ │ ├── cbotai-* # Cbotai domain checks (docs, encyclopedia)
│ │ ├── msf-* # MSF domain checks (docs, encyclopedia)
│ │ ├── docs-* # Main site doc/category checks
│ │ ├── cross-domain # Cross-domain parity (cbotai, msf, main)
│ │ ├── cross-domain-smoke # Homepage 200 + non-empty title across all 4 sites (BASE_URL_1-4)
│ │ ├── 404-page-usability # 404 URL still renders theme header/nav/footer (no blank error page)
│ │ └── intentional-404 # Verify 404 renders for invalid URLs
│ ├── permalink-routing/ # URL routing, SEO, structure, feeds & API (62)
│ │ ├── trailing-slash # /docs vs /docs/ consistency
│ │ ├── pagination # /docs/page/1/, out-of-range pages
│ │ ├── search-permalinks # ?s=test&post_type=docs search URLs
│ │ ├── author-archive # /docs/authors/ archive pages
│ │ ├── edge-cases # Uppercase URLs, encoded chars, homepage
│ │ ├── sitemap-robots # sitemap.xml & robots.txt availability
│ │ ├── sitemap-structure # XML root tag valid, ≥5 <loc> entries
│ │ ├── wp-sitemaps # All WP sitemap sub-files (post, page, doc_category, users)
│ │ ├── sitemap-live-parity # Sample URLs from each sub-sitemap actually resolve 200 (stale sitemap guard)
│ │ ├── robots-content # robots.txt User-agent, Disallow, Sitemap directives
│ │ ├── canonical-urls # Canonical URL consistency (strips UTM, multi-doc)
│ │ ├── permalink-structure # Trailing-slash redirect, double-slash, UTM, category & encyclopedia
│ │ ├── legacy-permalinks # ?p= / ?page_id= redirect, /page/2/ multi-page, ?paged=
│ │ ├── feed-api # Docs RSS feed, wp-json docs & doc_category
│ │ ├── feed-permalinks # Main /feed/, atom, rss2, category feeds
│ │ ├── rss-feed-content # Feed bodies contain valid <rss><channel><title>, XML Content-Type, and ≥1 <item> (non-empty feed guard)
│ │ ├── betterdocs-rest-api # /wp-json/betterdocs/v1, search, doc_category, knowledge_base + wp/v2/docs pagination headers (X-WP-Total, X-WP-TotalPages)
│ │ └── multi-category-archive # Regression test for BetterDocs issue #57 (multi-cat URLs return 200)
│ ├── card-based/ # Frontend regression tests (39)
│ │ ├── chatbot-style # Launcher styling, color, hover, click
│ │ ├── deprecated-code # .elementor-widget-container absence
│ │ ├── disable-js # Verify removed scripts aren't loaded
│ │ └── nested-slug # Nested category URL slug verification
│ ├── navigation/ # Interactive navigation tests (51)
│ │ ├── archive-pagination # Pagination controls, page nav, doc links
│ │ ├── category-navigation # Category box visibility, counts, nav
│ │ ├── doc-link-navigation # Single doc clicks, category archives
│ │ ├── encyclopedia-nav # A-Z filter, entry links, reset
│ │ ├── glossary-navigation # Glossary tooltips, links, new tab nav
│ │ ├── faq-interaction # FAQ expand/collapse behavior
│ │ ├── search # Search bar, modal, live results
│ │ ├── search-modal-behaviors # Escape closes, result click navigates, no-result query
│ │ ├── search-filter # Category dropdown, popular tags
│ │ ├── ai-search-suggestion # AI-powered search-to-chatbot connection block (sparkle, label, query echo)
│ │ ├── sidebar-navigation # Sidebar categories, icons, doc counts
│ │ └── language-switcher # WPML language switcher on cbotai — presence, current-lang, locale-prefixed URLs
│ ├── single-doc/ # Single doc & encyclopedia entry features (27)
│ │ ├── single-doc-features # Breadcrumb, TOC, sidebar, prev/next, related, body class, FSE chrome guard
│ │ ├── article-reactions # Thumbs up/down reactions, feedback form
│ │ ├── encyclopedia-single # Entry title, alphabet list, URL match, body class, FSE chrome guard
│ │ ├── doc-summary # AI-powered Doc Summary — header, title, arrow, loading resolves + AI-degradation guard (no provider errors leaked) + content safety (no script/iframe/onerror, balanced tags)
│ │ ├── related-docs # Related Docs list: ≥1 link, first link resolves, no ".undefined" corruption rows
│ │ ├── print-view # Print button present, print media renders doc heading (CSS not hiding content)
│ │ ├── reading-time # "X min read" label renders with a positive number
│ │ └── social-share # Facebook/Twitter/LinkedIn share links use correct intent URLs with encoded doc URL
│ ├── multi-kb/ # Multi Knowledge Base flow on aichatbotliveserver (32)
│ │ ├── mkb-flow # MKB block → KB archive → category grid → single doc end-to-end (Gutenberg) + template/body-class integrity checks
│ │ └── encyclopedia-routing # Encyclopedia archive, single entry, alphabet filter, invalid path 404, FSE template & chrome integrity
│ ├── seo/ # SEO, meta tag & HTML structure tests (15)
│ │ ├── meta-tags # Title, canonical, H1, viewport meta
│ │ ├── html-structure # HTML5 doctype, <html lang>, UTF-8 charset, WP REST bootstrap <link rel="https://api.w.org/">
│ │ └── structured-data # JSON-LD FAQPage schema on faq-b-classic + parseable JSON
│ ├── security/ # Security headers, access control & vuln guards (22)
│ │ ├── security-headers # X-Frame-Options, X-Content-Type-Options, wp-admin redirect, REST API, comments feed, .js.map source maps NOT exposed
│ │ ├── secret-leak-scan # No API keys/tokens (OpenAI, Claude sk-ant, Gemini AIza, Google OAuth, api_key, license_key, AWS) leaked in HTML or inline scripts
│ │ ├── xss-reflection # Search/404-slug injections HTML-escaped, no raw <script> reflected, no DB/PHP internals leaked
│ │ ├── sql-error-leak # WP search + BetterDocs REST resilient to injection markers, no SQL/PHP error signatures
│ │ └── rest-auth # /betterdocs/v1/overview unauth → 401 (no data leak); feedback POST without nonce → 4xx (SM-09, SM-10)
│ ├── accessibility/ # Accessibility & error-leak tests (31)
│ │ ├── console-errors # No JS console errors on 5 key pages
│ │ ├── image-alt-text # All images have alt attributes
│ │ ├── broken-image-scan # HEAD-check every <img src> on key pages, none return 404
│ │ ├── php-error-scan # HTML body of 7 page types contains no "Fatal error", "Warning:", "Notice:", stack traces, etc.
│ │ ├── heading-hierarchy # Single doc has exactly one <h1>, ≥1 <h2>, and <h1> is not empty
│ │ ├── subresource-health # 8 key pages: no pageerror (uncaught JS) + no same-origin subresource ≥500 (broken enqueues, dead REST, failed admin-ajax)
│ │ └── wcag-scan # axe-core WCAG 2 A/AA scan on 5 key pages — allowlist mode, fails only on NEW violation rule IDs (regression guard, not zero-violations)
│ ├── site-chrome/ # Header, footer, responsive, menu (19)
│ │ ├── header-footer-nav # Footer, logo home, main menu, skip link
│ │ ├── mobile-viewport # Mobile viewport rendering (375×812)
│ │ ├── main-menu-links # All top-level menu links resolve (no 4xx/5xx)
│ │ ├── theme-footer # FSE chrome-leak sweep across 5 page types (no "Proudly powered by WordPress")
│ │ ├── http-compression # Key HTML pages ship gzip/brotli/deflate (perf/CDN regression guard)
│ │ └── mobile-menu-toggle # Hamburger visible + dropdown appears after click at mobile viewport (375×812)
│ └── page-snapshots/ # Full-page aria snapshots (2)
│ ├── docs # Docs page full aria snapshot
│ └── encyclopedia # Encyclopedia page full aria snapshot
├── instant-answer/ # Instant answer widget - 4 tab tests (25)
│ ├── home-tab # Header, search input, docs list, tabs
│ ├── chatbot-tab # Welcome message, email/guest login
│ ├── ask-tab # Query form: email, name, subject, upload
│ └── resources-tab # Doc categories, Q&A section
├── chatbot/ # AI chatbot interaction tests (25)
│ ├── chatbot-ui # Panel title, response time, tabs
│ ├── guest-search # Guest login, message, AI response + links
│ ├── email-search # Email login, search, response + links
│ ├── chatbot-input # Empty/whitespace send is no-op, Enter/button send, input clears after send, AI reply never leaks provider-error strings
│ ├── transcript-menu # Kebab menu 3 options: Download Transcript, Send Transcript, Start a New Chat (cbotai)
│ └── wiring # CORS preflight on /betterdocs-pro/v1/query-post + docs page ships betterdocsAIChatbot config (cbotai) (SM-12, SM-13)
└── helpers.js # Shared utilities (safeGoto, sendChatbotMessage, etc.)
Total: 524 tests across 175 files
Each Gutenberg block, Elementor widget, and shortcode page is visited once, and the full aria tree of main#content is captured using Playwright's toMatchAriaSnapshot(). This verifies the complete content structure — headings, links, categories, doc counts, icons, and text — without relying on visual pixel comparisons.
Pages across three different sites (betteromation, betterdocs.msf, cbotai) are checked to ensure they load properly and don't return 404 errors. Additional permalink routing, SEO, and structure tests verify:
- Trailing slash consistency —
/docsvs/docs/both resolve correctly - Pagination URLs —
/docs/page/1/, category pagination, out-of-range pages return 404 - Search permalinks —
?s=test&post_type=docssearch result pages load - Author archives —
/docs/authors/1/loads without errors - Edge cases — uppercase URLs, URL-encoded characters, homepage validation
- Cross-domain parity — same paths work across all three domains
- Intentional 404 validation — non-existent slugs and double slugs return HTTP 404
- API endpoints — RSS feed, REST API docs and category endpoints respond
- Sitemap & robots.txt —
sitemap.xmlreturns XML,robots.txtreturns 200 - Canonical URL consistency — single doc canonical strips UTM/query params, encyclopedia archive canonical matches URL, multi-doc canonical check (Cricket, Fencing, Apple)
- Trailing-slash redirect —
/docs/cricket-the-gentlemens-game(no slash) redirects to/docs/cricket-the-gentlemens-game/ - Query-param resilience — UTM-tagged URLs don't break rendering
- Double-slash normalization —
/docs//cricket-.../normalizes correctly - Category archive coverage — all real category archives (
sports,fruits,team,qa) return 200 - Encyclopedia entry permalinks — 5 entries (
aesthetic,altruism,ball,cat,dog) all resolve - WP core sitemap sub-files —
/wp-sitemap.xml, posts, pages, doc_category, users sitemaps all return 200 with valid XML; doc_category sub-sitemap is referenced from root - robots.txt content — declares
User-agent, disallows/wp-admin/, referenceswp-sitemap.xml - Legacy permalink redirects —
?p=Nand?page_id=Nredirect to friendly URLs - Multi-page docs & encyclopedia pagination —
/docs/{slug}/page/2/and/encyclopedia/page/2/resolve,?paged=Naccepted - Feed permalinks — site
/feed/,/feed/atom/,/feed/rss2/,/docs/feed/, doc-category feed/docs/sports/feed/all return 200; main feed body is valid RSS XML - BetterDocs REST API — plugin namespace
/wp-json/betterdocs,/betterdocs/v1,/betterdocs/v1/search, WP RESTdoc_categoryandknowledge_basetaxonomies, discovery JSON advertises BetterDocs namespace - BetterDocs issue #57 regression — multi-category archive (
/docs/cat-1,cat-2/), multi-cat RSS feed, multi-cat JSON feed, three-category URL, and control cases (single-cat 200, invalid 404). Locks in the expected behavior so the 4.3.8 regression cannot silently come back
- Single doc features — Breadcrumb, home-link navigation, Table of Contents, sidebar, prev/next docs-nav, related articles
- Article reactions & feedback — Reaction buttons and feedback form presence
- Encyclopedia single entry — Entry title, alphabet navigation list, URL match
- Meta tags — Title tag, canonical URL, H1 presence on docs & encyclopedia, viewport meta tag
- Console errors — No JS console errors on 5 key pages (homepage, docs, encyclopedia, single doc, single encyclopedia entry)
- Image alt text — All images have
altattributes on 4 key pages - Header & footer — Footer visibility, logo→home navigation, main menu, skip-to-content link
- Mobile viewport (375×812) — Homepage renders without horizontal scroll, docs/encyclopedia/single doc render correctly on mobile
- Main menu link resolution — All top-level menu links resolve without 4xx/5xx
- HTML5 document validity — Doctype
<!doctype html>,<html lang>attribute, UTF-8 charset on 4 key pages - Security headers —
X-Frame-Options(clickjacking) andX-Content-Type-Options: nosniff(MIME-sniff defense) - Access control —
/wp-admin/redirects unauthenticated users to login - WP REST API —
/wp-json/root endpoint accessible - Comments feed —
/comments/feed/returns 200 - Sitemap structure — XML root is
<urlset>or<sitemapindex>, lists ≥5 URLs/sitemaps
- Category Navigation: Verifies category boxes render correctly with names, doc counts, and icons. Clicks each category and confirms navigation to the correct archive page.
- FAQ Interaction: Tests expand/collapse behavior of FAQ sections, verifies multiple answers can be expanded simultaneously, and checks that answer content becomes visible on click.
- Search: Validates search bar visibility, placeholder text, modal opening on click, and live search results appearing when typing.
- Search Filter: Category dropdown selection, popular search tags visibility, submit button functionality on the block search page.
- Encyclopedia Navigation: A-Z alphabet filter links, clicking a letter filters entries, clicking All resets, entry links navigate to single encyclopedia pages.
- Archive Pagination: Pagination controls visibility, page 2 navigation, next arrow click, and doc list link click-through.
- Doc Link Navigation: Clicks docs from Sports and Fruits category archives, verifies single doc content loads, checks nested category archives.
- Sidebar Navigation: Sidebar category visibility with icons and doc counts across multiple sections.
- Glossary Navigation: Verifies glossary tooltip terms on a single doc page, checks tooltip data, confirms links point to encyclopedia entries, validates new-tab navigation, and ensures all glossary links are not 404.
Adapted from the existing BDS_Automation admin test suite (CardBased folder):
- Chatbot Launcher Styling: Verifies launcher button background color, border-radius, hover color change, and panel opening behavior.
- Deprecated Code Check: Scans 12 pages to confirm
.elementor-widget-containerclass is absent (regression for removed Elementor code). - Removed Script Check: Scans 19 pages to verify
extend-search-modal.jsis no longer loaded. - Nested URL Slugs: Validates that nested category URLs (
/docs/team/qa/,/docs/sports/) resolve correctly.
The instant answer widget has 4 tabs: Home, Chatbot, Query, and Assets. Each tab is tested individually:
- Home: header text, search input, docs list, tab navigation
- Chatbot: welcome message, email login, guest login, field visibility
- Query: contact form fields (email, name, subject, message, file upload, send button)
- Assets: doc categories list, Q&A section with FAQ questions
- Guest Mode: Login as guest, type and send a message, verify AI responds and response links aren't 404.
- Email Mode: Login with
msf@pro.automation, search "Fencing", verify AI responds and response links aren't 404. - UI Elements: Panel title, response time info, description text, welcome message, send button, tab switching.
| Scenario | How It's Handled |
|---|---|
| DB Connection Error | safeGoto() detects "Error Establishing a Database Connection", waits 2 minutes, retries once |
| HTTP Error (502/500) | safeGoto() catches page.goto() failures gracefully, test checks status code instead of page content |
| Chatbot "Thinking" | sendChatbotMessage() polls for up to 1 minute waiting for AI response text to appear |
| Chatbot Technical Error | If AI returns "experiencing" error, page is reloaded and the full chatbot flow is retried once |
| Code Snippet YAML | Pages with raw code content (colons break YAML) use structural-only snapshots, skipping code text |
- Aria snapshots are content-sensitive. If page content changes (new categories added, text modified, FAQ questions updated), the corresponding snapshot test will fail and needs to be regenerated. Run
npx playwright test --update-snapshotsto refresh. - AI chatbot responses vary. The chatbot is powered by an external AI service. Response time, content, and availability can fluctuate. Tests verify that a response was received rather than checking exact response text.
- Code snippet pages contain raw source code with special characters (colons, backticks, curly braces) that are incompatible with YAML-based aria snapshots. These tests intentionally skip the code text and only verify structural elements.
- Server-side errors (502/500) on certain pages (e.g.
/docs/team/) are server infrastructure issues, not test failures. Tests check HTTP status is not 404 rather than requiring 200. - Headed vs headless mode may have minor timing differences. The test suite is optimized for both, but AI chatbot tests may take longer in headed mode due to rendering overhead.
This test suite is designed to evolve alongside BetterDocs:
- Snapshot regeneration will be needed when new features, layouts, or content changes are deployed. The
generate-tests.jsscript in/scriptscan bulk-regenerate snapshots for any category. - New blocks/widgets/shortcodes can be added by appending URLs to the generator script and running it for the relevant category.
- Admin/backend tests could be introduced later with authentication setup (similar to the BDS_Automation project's
auth.setup.jspattern). - Additional chatbot scenarios (multi-turn conversations, different queries, edge cases) can be added using the
sendChatbotMessage()helper. - Cross-browser testing (Firefox, WebKit) can be enabled by adding projects to
playwright.config.js. - CI/CD integration via GitHub Actions is straightforward — a workflow template can be added to
.github/workflows/. - Instant answer tab names and content may change as the plugin is customized. Tests use flexible text matching where possible but may need updates after significant UI changes.
- All Gutenberg block snapshots
- All Elementor widget snapshots
- All shortcode snapshots
- Basic page snapshots (Docs, Encyclopedia)
- 404 checks across multiple sites
- Cross-domain parity checks
- Permalink routing & edge case tests
- API endpoint availability checks
- Intentional 404 validation
- Category navigation tests
- FAQ interaction tests
- Search modal tests
- Instant answer (4 tabs)
- AI chatbot (guest + email login, response link validation)
- Frontend regression tests (deprecated code, removed scripts, URL slugs)
- Encyclopedia A-Z filter navigation
- Archive pagination navigation
- Doc link & category archive navigation
- Search category filter & popular tags
- Sidebar category navigation
- Glossary tooltip & encyclopedia link validation
- Single doc features (breadcrumb, TOC, reactions, related, feedback)
- Encyclopedia single-entry page tests
- SEO meta tag checks (title, canonical, H1, viewport)
- Accessibility: console errors & image alt text
- Header/footer navigation & skip-to-content link
- Mobile viewport responsive rendering
- Sitemap.xml & robots.txt availability
- Canonical URL consistency & query-param stripping
- Trailing-slash redirects, double-slash normalization
- CI/CD pipeline (GitHub Actions)
- Cross-browser support (Firefox, WebKit)
- Admin panel tests (with authentication)
Muammar Shahrear - @Muammar Shahrear - shahrearmuammar@gmail.com
Muammar Shahrear is a software tester and researcher specializing in test automation, AI agents, WordPress plugin testing, and SaaS product quality assurance. He completed his B.Sc. and M.Sc. from the Institute of Information Technology (IIT), Jahangirnagar University (JU), Bangladesh, and also holds an M.Sc. from Technische Hochschule Mittelhessen (THM), Germany.
