Files
crank/_bmad-output/test-artifacts/test-design-progress-epic-3.md
T

4.9 KiB
Raw Blame History

runScope, runKey, workflowStatus, totalSteps, stepsCompleted, lastStep, nextStep, inputDocuments, lastSaved
runScope runKey workflowStatus totalSteps stepsCompleted lastStep nextStep inputDocuments lastSaved
epic-level epic-3 completed 5
step-01-detect-mode
step-02-load-context
step-03-risk-and-testability
step-04-coverage-plan
step-05-generate-output
step-05-generate-output
_bmad-output/planning-artifacts/epics.md
_bmad-output/test-artifacts/test-design-architecture.md
_bmad-output/test-artifacts/test-design-qa.md
_bmad-output/planning-artifacts/prds/prd-crank-2026-08-03/prd.md
_bmad-output/planning-artifacts/architecture/architecture-crank-2026-08-03/ARCHITECTURE-SPINE.md
_bmad/tea/config.yaml
2026-09-13

Epic 3 test design progress

Output: test-design-epic-3.md. Completed 2026-09-13. Story/requirement/AC-count coverage, source references, risk scores and format validated. Template adapted to existing-project inventory: counts are scenario seeds, estimates are provisional QA deltas, no synthetic code-coverage or execution claim. Native loop ATDD is post-dev; pre-implementation customization is a separate pending approval, not executed by this design run.

Coverage decisions

20 story families, two independently assertable scenario seeds per family (A positive contract; B rejection/fault). Approved decomposition separates call lifecycle from Tool signals, Prompt Draft/publish/list/portability, and completion/logging/Resource-Prompt signals; former 3.14 is now 3.20. All owning-story Given/When/Then groups remain required ATDD input; these are not a claim of exhaustive implemented test cases.

Scope: FR-13–FR-20; system families CP-08..11. Lowest effective level assigned per family. P0 security/data integrity with no workaround; P1 user/core flow. No speculative mandatory P2/P3 additions.

Source reuse: crates/crank-community-mcp/tests/unit/{manifest,session,tool_error}.rs; apps/mcp-server/tests/integration/{transport_protocol,catalog_access,tool_search,execution_stages}.rs; apps/ui/tests/e2e/onboarding.spec.js.

Gaps: Existing tests cover baseline Tools, transport, catalog and runtime only. Resources/Prompts/modern adapter matrix are planned; no feature-level tests were found in main. Do not confuse MCP client-library behavior with implementation evidence. Tasks cells are completed by Story 4.7; 3.20 must mark them pending rather than falsely PASS.

NFR: Discovery p95 ≤150 ms on 2,000 Operations/200 Agents; zero unauthorized metadata/content reads. Pin stable/N-1/N-2 and ≥3 clients at feature freeze. The 2026-07-28 wire shapes named in epics.md are planning inputs, not a newly verified claim about latest MCP; verify official specifications/digests before adapter ATDD, preserving legacy deprecation gate.

Fixtures: Wire fixtures per protocol era; scope-separated Agent catalogs; text/JSON/binary hostile resources; exact-version graphs; capture upstream; clock/cancellation barriers; browser keyboard and RU/EN fixtures.

Execution: PR functional tests, nightly bounded expensive/fault work, weekly calibrated qualifications; scheduled tiers are planned, not configured. Mandatory story ACs still must pass before acceptance, regardless of tier. 72h/30day release windows only in Epic8. QA effort assumption with prerequisite implementation ready: P0 60–120h; P1 32–64h; optional P2 0–12h/P3 0–4h; total 92–200h (~3–6 contributor-weeks). Not a backlog of compulsory new tests; refine against existing-test reuse during ATDD.

Risks and NFR planning

Risk Category Failure P I Score System risk Mitigation / owner / deadline
E3-R1 BUS Advertised capabilities or version behavior disagree 2 3 6 R-009 Revision/client matrix and wire errors; MCP owner, 3.20
E3-R2 SEC Resource/Prompt scope or REST expansion leaks data 3 3 9 R-004 Negative authorization + SSRF + redaction; Security, 3.8/3.11/3.16
E3-R3 DATA Mutable catalog or stale cursor serves wrong published version 2 3 6 R-009 Snapshot/revoke races; Registry/MCP, publication stories

Scores describe inherent risk, not observed failures or a release verdict. Security/reliability/performance thresholds inherit the system design and owning story ACs; missing environment/measurement evidence remains explicit, never inferred PASS.

Context: Rust/PostgreSQL + browser JavaScript fullstack. Existing test source inventory at main 083974c; f8ee7c9 candidate is separate. Browser CLI absent; no live exploration, no fabricated evidence. Playwright/Pact utility flags are intentions only: packages absent; no library migration or Pact scaffolding in this planning scope. Required risk, probability/impact, test levels/priorities and NFR knowledge loaded; concrete source reuse and gaps appear in the epic plan.

Explicit batch request, independent epic scope. Requirements and acceptance criteria: planning-artifacts/epics.md, Epic 3. System designs already updated; no product implementation or test execution authorized by this planning run.