Capture bounded evidence
Playwright captures 1440-pixel desktop and Pixel 7 full-page views, plus headings, landmarks, navigation visibility, interactive controls, overflow, console errors, failed requests, and a limited text excerpt.
Playwright / OpenAI Responses API / pytest / axe / Lighthouse
Web QA Playwright Suite
Deterministic browser, API, accessibility, performance, and security gates paired with blind multimodal exploration and evidence-based failure triage. Every result remains inspectable in a human-readable report hub.
What this proves
Deterministic web QA with a bounded exploratory layer.
This is a purpose-built example application and quality suite. The app includes an Express API, authentication, protected task mutations, validation, responsive UI states, rate limiting, security headers, and deterministic reset hooks so automated tests can run repeatably. An optional GPT-5.6 workflow then reviews captured evidence for semantic risks that exact assertions may not anticipate.
- Browser
- 38/38 Playwright tests across Chromium, Firefox, and mobile Chrome
- Contracts
- 7/7 pytest checks for schemas, boundaries, auth, and latency
- AI exploratory review
- Blind desktop/mobile decision with 3 evidence-backed findings and deterministic follow-ups
- Performance
- Lighthouse 96 performance, 100 accessibility, 96 best practices, 100 SEO
- Security
- 0 high/critical npm audit findings plus authz, headers, rate-limit tests
LLM-guided exploratory testing
Model judgment that ends in testable evidence.
The AI layer looks for semantic and cross-viewport risks without replacing Playwright, axe, Lighthouse, pytest, npm audit, or human review.
Run a blind multimodal assessment
The OpenAI Responses API receives screenshots and neutral page evidence, but no seeded defect, expected answer, or application source. The model must distinguish credible bugs from usability risks and preference.
Require structured, reviewable output
A strict JSON schema requires a verdict, strongest evidence, uncertainty, severity, viewport, confidence, impact, recommendation, and a concrete deterministic follow-up for every finding.
Convert hypotheses into ordinary tests
Candidate findings become Playwright, axe, or DOM checks. Failed suite artifacts can also be classified as product, test, environment, timing, or unknown causes before a person chooses the next diagnostic step.
Advisory by design. Artifact paths are allowlisted, common credential patterns are redacted, source and log excerpts are truncated, and failed-test triage is capped. A model result does not silently become a release gate; exact checks remain authoritative unless a team explicitly adopts a reviewed, repeatable threshold.
Recorded evidence
The suite, report hub, and live AI review in motion.
Suite run and QA workflow
Headed Playwright execution across the application and its deterministic quality layers.
Report hub walkthrough
Human-readable browser, API, performance, Python, security, and AI artifacts from one run.
Live AI exploratory review
Evidence capture, blind GPT-5.6 decision, structured findings, and the populated review report.
Generated artifacts
A report hub for engineers and reviewers.
The report hub links the Playwright HTML report, Lighthouse report, self-contained pytest report, npm audit output, populated AI QA review, and optional ZAP slot.
Coverage layers
Six deterministic layers remain the release authority.
Functional E2E
Playwright validates navigation, category filtering, login/logout, authenticated task creation/completion, contact form validation, and desktop/mobile behavior.
API Behavior
Request-level tests assert health contracts, auth tokens, protected profile access, CRUD lifecycle behavior, stable error shapes, and filtered task responses.
Independent Contracts
pytest and jsonschema independently check API schemas, contact boundary cases, authenticated creation, and health endpoint latency.
Accessibility
axe-core scans WCAG A/AA rules while focused tests cover keyboard entry, skip-link behavior, modal focus containment, Escape handling, and focus restoration.
Performance Budgets
Browser tests enforce useful content under 2 seconds, DOMContentLoaded under 1.5 seconds, load under 2 seconds, document transfer below 100 KB, and first-party resources below 350 KB.
Security Gates
Tests verify CSP, nosniff, frame/referrer policies, hidden framework details, authorization requirements, logout invalidation, login rate limits, oversized payload handling, and stack-trace suppression.
CI/CD and portability
Portable automation with optional model calls.
- Node and Python dependencies are installed in CI with pinned browser setup.
- Required quality gates run through a full-suite script that continues through layers so every report is produced.
- Reports, traces, screenshots, and videos upload as artifacts even when a gate fails.
- Failed deterministic runs can trigger advisory AI triage when the repository secret is present.
- A separate manual-dispatch job runs the multimodal review with model findings non-blocking by default.
- Optional OWASP ZAP baseline job runs separately and uploads passive DAST output.