Files
crank/_bmad-output/test-artifacts/test-design-progress-epic-5.md
T

5.3 KiB
Raw Blame History

runScope, runKey, workflowStatus, totalSteps, stepsCompleted, lastStep, nextStep, inputDocuments, lastSaved
runScope runKey workflowStatus totalSteps stepsCompleted lastStep nextStep inputDocuments lastSaved
epic-level epic-5 completed 5
step-01-detect-mode
step-02-load-context
step-03-risk-and-testability
step-04-coverage-plan
step-05-generate-output
step-05-generate-output
_bmad-output/planning-artifacts/epics.md
_bmad-output/test-artifacts/test-design-architecture.md
_bmad-output/test-artifacts/test-design-qa.md
_bmad-output/planning-artifacts/prds/prd-crank-2026-08-03/prd.md
_bmad-output/planning-artifacts/architecture/architecture-crank-2026-08-03/ARCHITECTURE-SPINE.md
_bmad/tea/config.yaml
2026-09-13

Epic 5 test design progress

Output: test-design-epic-5.md. Completed 2026-09-13. Story/requirement/AC-count coverage, source references, risk scores and format validated. Template adapted to existing-project inventory: counts are scenario seeds, estimates are provisional QA deltas, no synthetic code-coverage or execution claim. Native loop ATDD is post-dev; pre-implementation customization is a separate pending approval, not executed by this design run.

Coverage decisions

13 story families, two independently assertable scenario seeds per family (A positive contract; B rejection/fault). Former 5.9 splits into Prometheus (5.9) and notification gateway/Apprise (5.10); Grafana becomes 5.11; former 5.11 splits into SLO (5.12) and alert delivery/end-of-epic evidence (5.13). All owning-story Given/When/Then groups remain required ATDD input; these are not a claim of exhaustive implemented test cases.

Scope: FR-27–FR-35; system families CP-16..21. Lowest effective level assigned per family. P0 security/data integrity with no workaround; P1 user/core flow. No speculative mandatory P2/P3 additions.

Source reuse: crates/crank-observability/tests/{request_correlation,propagation,json_logging,redaction,critical_errors,otlp,prometheus,http_metrics,metrics_signals}.rs; apps/mcp-server/tests/integration/execution_stages.rs; apps/admin-api/tests/integration/logs_usage.rs; crates/crank-metrics/tests/.

Gaps: IDs, local metric registry, redaction and captured OTLP are source-present foundations. Real GlitchTip receiver delivery, durable encrypted spool/loss reconciliation, complete ProductEvent rollups, packaged Prometheus/Grafana and SLO qualification are not proven by them. Actual Load Run data acceptance remains in 6.6, not Grafana story 5.11. No services are provisioned in this task.

NFR: 100% support IDs independent of sampling; healthy delivery ≥99.99% within60s; outage≤24h with100% saved-envelope delivery within25h, sized30h spool. Observability overhead≤5% p95 latency and≤10% CPU; drain≤30s. Availability30d≥99.95%, control plane≥99.9%. Exact series budgets/buckets and burn windows must use versioned typed/rule contracts; unspecified values remain UNKNOWN until owning story freezes them.

Fixtures: Queryable local trace/error receiver and real GlitchTip qualification environment; independent generated/accepted/ACK/lost envelope ledger; bounded disk partitions and fault controls; golden UTC events; Prometheus synthetic series; provisioned Grafana fixture without committed secrets.

Execution: PR functional tests, nightly bounded expensive/fault work, weekly calibrated qualifications; scheduled tiers are planned, not configured. Mandatory story ACs still must pass before acceptance, regardless of tier. 72h/30day release windows only in Epic8. QA effort assumption with prerequisite implementation ready: P0 60–120h; P1 24–48h; optional P2 0–8h/P3 0–4h; total 84–180h (~3–5 contributor-weeks). Not a backlog of compulsory new tests; refine against existing-test reuse during ATDD.

Risks and NFR planning

Risk Category Failure P I Score System risk Mitigation / owner / deadline
E5-R1 OPS Spool/replay loss yields false trace completeness 2 3 6 R-005 Independent envelope accounting and crash/outage; Observability, 5.6
E5-R2 SEC Browser/export/link leaks secrets or executes unsafe URL 3 3 9 R-004 Canaries at every sink and hostile links; Security/UI, 5.2–5.4
E5-R3 DATA Rollups/cardinality/SLI denominators lie 2 3 6 R-011 Idempotency + UTC + rules oracles; Metrics, 5.8/5.12
E5-R4 PERF Telemetry pressure blocks product requests 3 3 9 R-006 Quota isolation + enabled/disabled comparison; SRE, 5.6

Scores describe inherent risk, not observed failures or a release verdict. Security/reliability/performance thresholds inherit the system design and owning story ACs; missing environment/measurement evidence remains explicit, never inferred PASS.

Context: Rust/PostgreSQL + browser JavaScript fullstack. Existing test source inventory at main 083974c; f8ee7c9 candidate is separate. Browser CLI absent; no live exploration, no fabricated evidence. Playwright/Pact utility flags are intentions only: packages absent; no library migration or Pact scaffolding in this planning scope. Required risk, probability/impact, test levels/priorities and NFR knowledge loaded; concrete source reuse and gaps appear in the epic plan.

Explicit batch request, independent epic scope. Requirements and acceptance criteria: planning-artifacts/epics.md, Epic 5. System designs already updated; no product implementation or test execution authorized by this planning run.