21 KiB
story_id, story_key, requirement, status, baseline_commit, created
| story_id | story_key | requirement | status | baseline_commit | created |
|---|---|---|---|---|---|
| 1.15 | 1-15-invocation-history-usage | FR-6 | done | 767428436d |
2026-08-23T01:47:15+03:00 |
Story 1.15: Предоставить безопасные Invocation History и Usage
Status: done
Story
As an оператор Crank, I want видеть фильтруемую историю и показатели выполнения, so that могу диагностировать ошибки и оценивать качество работы Agents и Operations без доступа к чувствительным данным.
Operational Brief
Story 1.15 закрывает FR-6 и завершает hardened core перед onboarding Story 1.16.
Предыдущие stories уже дали основу:
- 1.5 сохраняет Request ID и Trace ID независимо от telemetry sampling;
- 1.7 и 1.13 гарантируют immutable Operation/Agent published references;
- 1.8 и 1.9 ввели typed execution taxonomy и honest side-effect uncertainty;
- 1.14 гарантирует atomic approvals и не заменяет product result при history-loss.
Теперь нужно сделать Invocation History и Usage полноценным операторским инструментом:
- history record должен быть typed, scoped и безопасно редактированным;
- Logs API/UI должны фильтровать по bounded period, Agent, Operation и outcome;
- details должны показывать stage, safe previews, duration, Agent, Operation Version, Request ID и Trace ID;
- Usage должен считать counts, outcomes и latency percentiles без смешивания upstream/client/schema failures с Crank-caused failures;
- CSV export обязан наследовать filters/auth/redaction/bounds и защищать spreadsheet consumers;
- retention должен быть typed/observable и не повреждать audit/evidence;
- production-size query path не должен грузить 1 000 000 records в память приложения или браузера.
Requirement / Architecture Trace
- Direct requirement: FR-6.
- Primary architecture: AD-11 correlation identity; AD-13 typed metrics/local product analytics; AD-24 retention.
- Supporting architecture: AD-1, AD-2, AD-4, AD-12, AD-16, AD-19, AD-20.
- Applicable NFR: NFR-P3, NFR-P5, NFR-R2, NFR-S1, NFR-S2, NFR-S4, NFR-S6, NFR-S8, NFR-O1, NFR-O4, NFR-O5, NFR-C1, NFR-C4.
Scope
In scope:
- Invocation History record contract and persistence validation.
- Safe preview redaction/limits/truncation evidence.
- Logs list/detail Admin API filters, pagination and scoped access.
- Usage aggregation by workspace, Agent and Operation with outcome groups and percentiles.
- UTC half-open period boundary semantics.
- CSV export for filtered history/usage dataset with redaction and formula-injection protection.
- Retention cleanup policy and observable outcome.
- Logs/Usage UI loading, empty, error, retry, export and stale-response guards in RU/EN.
- Registry/Admin/UI/Playwright tests, docs, capability inventory and baseline evidence for FR-6.
Out of scope:
- GlitchTip trace deep links/search UI beyond showing/copying local Request ID/Trace ID.
- Product analytics DAU/adoption rollups (Epic 5 / FR-35).
- Prometheus/Grafana packaged dashboards (Epic 5).
- Background Tasks history (Epic 4), Load Runs (Epic 6), Resources/Prompts usage (Epic 3).
- Changing Operation/Agent/approval semantics from Stories 1.7/1.13/1.14.
Acceptance Criteria
- Typed history record. Given завершённая или отклонённая Execution attempt, when формируется Invocation History record, then сохраняются workspace, Agent, Operation/version, execution ID, outcome taxonomy, timestamps, duration, Request ID и Trace ID, and запись не содержит raw credential, arbitrary headers/body или unbounded error text.
- Safe previews. Given execution input и output, when создаются previews, then централизованная redaction применяет byte/field/depth limits, сохраняет валидный UTF-8/JSON and oversized content получает явный truncation marker.
- History-loss honesty. Given failure записи Invocation History после фактического outcome, when persistence недоступен или record отклонён, then product result не заменяется ложным execution failure, and typed
crank-metricsувеличивает low-cardinality history-loss counter. - Logs filters and pagination. Given авторизованный Admin UI caller, when он открывает Logs, then history фильтруется по bounded period, Agent, Operation и outcome с deterministic opaque pagination, and foreign-scope query не раскрывает counts, names или records.
- Log details. Given конкретная Invocation History record, when администратор открывает details, then UI показывает stage, safe previews, duration, Agent, точную Operation Version, Request ID и Trace ID, and raw internal cause остаётся server-side.
- Usage aggregation. Given Usage query за период, when данные агрегируются, then UI получает counts, success/failure outcomes и latency percentiles по Agent и Operation, and upstream/client/schema outcomes отображаются отдельно от Crank-caused failures.
- UTC period boundaries. Given period boundary, when агрегируются records на границе дня или timezone UI, then storage/query contract использует UTC instants and one record cannot be counted twice by inclusive overlap.
- CSV export. Given CSV export, when администратор экспортирует отфильтрованный dataset, then export applies same authorization, filters, redaction and bounded row/size policy as UI, and spreadsheet formula injection values are escaped safely.
- Retention. Given configured Invocation History retention, when record reaches retention boundary, then archive/delete uses typed observable policy/outcome, and deletion does not damage Published Version audit, aggregate Usage or immutable release evidence.
- Production-size query plan. Given 1 000 000 history records, when Logs/Usage screens query data, then query path is bounded and does not load full dataset into app/browser memory; local evidence records plan/row-count/proxy timing.
- Logs/Usage UI RU/EN. Given Logs/Usage UI на RU и EN, when данные загружаются, отсутствуют, фильтруются, экспортируются или завершаются ошибкой, then loading/empty/error/retry/stale-response states exist, and workspace/request generation guard prevents stale response overwrite.
- Verification. Given automated verification, when history and usage suites run, then redaction, bounded previews, persistence loss, period boundaries, percentiles, CSV injection, retention, cross-scope access and production-size query plans are covered, and registry/Admin API/Playwright tests plus
just verifypass.
Tasks / Subtasks
-
Task 1 — Freeze history/usage oracle and RED tests (AC: 1–12)
- Characterize current
invocation_logs, Admin Logs/Usage API and UI behavior. - Add RED tests for filters, pagination, details, period boundaries, CSV injection, retention and production-size plan.
- Add RED tests for redaction, bounded previews, cross-scope denial and history-loss product-result preservation.
- Characterize current
-
Task 2 — Invocation History persistence contract (AC: 1–3, 5, 9)
- Validate all new history records include operation version, execution taxonomy, request_id and trace_id.
- Ensure safe previews are produced by one central sanitizer and stored bounded.
- Add retention API/policy with typed outcome and metrics; do not delete Published Version audit/evidence.
-
Task 3 — Logs list/detail API (AC: 4–5, 7–8, 12)
- Add bounded query DTO fields for period start/end, Agent, Operation, outcome and opaque cursor.
- Implement deterministic pagination without leaking foreign counts or names.
- Add safe details projection including stage, previews, duration, Agent, Operation Version, Request ID and Trace ID.
- Add CSV export endpoint using same filters/auth/redaction/bounds and formula escaping.
-
Task 4 — Usage aggregation API (AC: 6–7, 10, 12)
- Implement UTC half-open window semantics and prevent boundary double-counting.
- Aggregate counts, success/failure outcomes and p50/p95/p99 by Agent and Operation.
- Separate upstream/client/schema outcomes from Crank-caused failures.
- Add production-size query-plan/index evidence and bounded memory proof.
-
Task 5 — Logs/Usage UI and Playwright (AC: 4–8, 11–12)
- Add Agent/Operation/outcome filters, pagination, retry/export states and Request ID/Trace ID details.
- Add stale-response guards for workspace/filter/request generation.
- Add RU/EN strings together and avoid raw HTML sinks.
- Extend Playwright
logs-usage.spec.jsfor filters, details, CSV export, empty/error and stale-response behavior.
-
Task 6 — Docs, inventory, gates and handoff (AC: 12)
- Update docs/admin-api.md, docs/data-model.md, docs/manual-regression-checklist.md and capability inventory for FR-6 only after evidence passes.
- Update baseline manifest/results without raw browser reports, secrets, payloads or host paths.
- Run registry/Admin/UI targeted suites, capability gates, SQLx checks and final
just verify. - Update Dev Agent Record, File List, Change Log and story/sprint status to
reviewonly after gates pass.
Review Findings
- [Review][Patch][High] Safe preview sanitizer does not enforce field/depth limits [crates/crank-core/src/observability.rs:12]
- [Review][Patch][High] Logs detail UI omits exact Operation Version [apps/ui/js/logs.js:126]
- [Review][Patch][High] Usage CSV export remains client-side snapshot instead of server-authoritative bounded export [apps/ui/js/usage.js:556]
- [Review][Patch][High] Retention remains raw timestamp delete without typed observable policy/outcome and preserved aggregate usage semantics [crates/crank-registry/src/postgres/observability.rs:19]
- [Review][Patch][Medium] Usage stale-response guard mutates state before validating workspace/request generation [apps/ui/js/usage.js:527]
- [Review][Patch][Medium] Logs filter changes do not invalidate in-flight responses immediately [apps/ui/js/logs.js:727]
- [Review][Patch][Medium] Logs CSV export inherits page limit 100 instead of export dataset bound [apps/ui/js/logs.js:443]
- [Review][Patch][Medium] Logs outcome filter supports only ok/error status, not execution outcome groups [apps/admin-api/src/dto.rs:307]
- [Review][Patch][Medium] Usage outcome grouping defaults uncovered typed failures to
crank[crates/crank-registry/src/postgres/observability.rs:633] - [Review][Patch][Medium] Admin Logs/Usage API lacks explicit start/end UTC window filters claimed by the story [apps/admin-api/src/dto.rs:307]
Dev Notes
Current State / Repository Findings
InvocationLogalready contains workspace, optional Agent, Operation, optional operation version, source, level, status, request_id, trace_id, status_code, duration, execution stage/error/retryability/certainty and request/response previews.- Registry
create_invocation_logalready validates request_id, trace_id, operation_version and execution taxonomy for new records and sanitizes previews before insert. - Current
list_invocation_logssupports workspace, level, source, operation_id, agent_id, created_after, search and limit, but does not provide cursor pagination, explicit start/end period, outcome filter or CSV export. - Current
get_invocation_logscopes by workspace and ID, but details projection must explicitly stay redacted and expose trace_id in UI. - Current usage aggregation supports workspace/source/created_after and p50/p95/p99, but lacks explicit Agent/Operation endpoint evidence, outcome group separation and half-open start/end boundary contract.
- Current UI Logs supports level/search/period/live mode, but does not include Agent/Operation/outcome filters, opaque pagination, CSV export, trace_id in details or robust logs stale-response guard.
- Current UI Usage renders summary/timeline/operation breakdown and client-side CSV snapshot, but export is not server-authoritative and does not prove same filters/auth/redaction/bounds as UI.
- Existing history-loss contract exists in Admin/MCP DC08 tests and
crank-metricshistory-loss metric; Story 1.15 must keep product result unchanged and add visible evidence, not invent a new failure path. - Existing retention method
delete_invocation_logs_before(cutoff)is raw and timestamp-only; Story 1.15 must wrap it in typed policy/outcome and prove it does not damage immutable Operation/Agent evidence.
Architecture and Safety Constraints
- PostgreSQL remains source of truth. Admin API/MCP startup must remain read-only; schema changes, if needed, go through MigrationAuthority append-only version.
- History records are diagnostic evidence, not user payload storage. Do not store raw credentials, arbitrary headers/body or unbounded error text.
- Request ID and Trace ID are support identifiers and must never become metric labels.
- Logs/Usage filters must include workspace scope in every query and must not disclose foreign object existence through counts, names or different error shapes.
- Pagination cursor must be opaque, deterministic and bounded; no offset pagination over production-size dataset if it requires scanning skipped rows at scale.
- CSV export is an exfiltration surface: same auth/filter/redaction as UI, bounded rows/bytes, formula injection escaping, no raw host paths/secrets.
- Usage/telemetry must not leave the installation; CSV remains local Admin response only.
- UI updates must use
textContent/DOM APIs and discard stale responses when workspace/filter/request generation changes. - No dependency/toolchain/MSRV changes; no broad frontend framework migration.
Likely File Map
Likely UPDATE:
crates/crank-core/src/observability.rscrates/crank-registry/src/model.rscrates/crank-registry/src/postgres/observability.rscrates/crank-registry/src/postgres/mod.rscrates/crank-registry/tests/integration/observability.rscrates/crank-registry/tests/integration/common.rsapps/admin-api/src/dto.rsapps/admin-api/src/app.rsapps/admin-api/src/routes/observability.rsapps/admin-api/src/service/observability.rsapps/admin-api/src/service.rsapps/admin-api/tests/integration/community_access_usage.rsapps/admin-api/tests/dc08.rsapps/mcp-server/tests/dc08.rsapps/ui/js/api.jsapps/ui/js/logs.jsapps/ui/js/usage.jsapps/ui/js/i18n.jsapps/ui/html/logs.htmlapps/ui/html/usage.htmlapps/ui/tests/e2e/logs-usage.spec.jsdocs/admin-api.mddocs/data-model.mddocs/manual-regression-checklist.mddocs/capability-inventory.jsondocs/capability-baseline/results.jsondocs/capability-baseline/manifest.json
Likely NEW:
crates/crank-registry/tests/integration/observability_scale.rsonly if keeping ignored production-size evidence separate.- Admin API CSV/export tests if existing
community_access_usage.rswould exceed maintainability limits.
Required Test Matrix
- New history record rejects missing/invalid request_id, trace_id, operation_version and unknown taxonomy.
- Secret canaries in request/response/error material are absent from stored previews, API details, UI DOM and CSV.
- Oversized nested previews preserve valid UTF-8/JSON and include explicit truncation markers.
- History-loss after product success/error keeps product response and emits low-cardinality metric.
- Logs filters: period, Agent, Operation, outcome, source/search and cursor; foreign objects return empty/safe error without counts.
- Details show stage, duration, Agent, Operation Version, Request ID and Trace ID; raw internal cause absent.
- Usage: counts, success/failure, p50/p95/p99 by Agent/Operation; upstream/client/schema groups separate from Crank-caused failures.
- Period boundary: records exactly at start included, exactly at end excluded; adjacent windows do not double-count.
- CSV: same filters as list, bounded rows/bytes, redaction, formula escaping for
=,+,-,@, tab and CR/LF-leading values. - Retention: deletes only eligible records, reports typed outcome, preserves published versions, Agent snapshots and release evidence.
- Scale: 1 000 000-row seed/plan evidence or ignored heavy test with deterministic local proxy budget.
- UI: loading, empty, error, retry, filters, details, export, stale workspace/filter response, RU/EN.
Verification Commands
cargo test -p crank-core observability -- --test-threads=1
cargo test -p crank-registry --test integration observability -- --test-threads=1
cargo test -p crank-registry --test integration observability -- --test-threads=1 --ignored
cargo test -p admin-api --test dc08 -- --test-threads=1
cargo test -p mcp-server --test dc08 -- --test-threads=1
cargo test -p admin-api --test integration community_access_usage -- --test-threads=1
cargo test -p crank-observability --test incidents -- --test-threads=1
just ui-build
cd apps/ui && npx playwright test tests/e2e/logs-usage.spec.js --project=chromium
just capability-inventory-check
just sqlx-check
python3 scripts/check-community-scope.py --root . --files <all NEW/UPDATE paths>
git diff --check
just verify
Dev Agent Record
Agent Model Used
GPT-5 Codex
Debug Log References
- 2026-08-23T01:47:15+03:00 — Created Story 1.15 from Epic 1 FR-6 after Story 1.14 moved to done.
- 2026-08-23T02:50:00+03:00 — Implemented Logs/Usage API, UI, CSV export, outcome aggregation, cursor pagination, 1M-row bounded plan evidence, docs/inventory updates and final gates.
- 2026-08-23T10:51:37+03:00 — Applied all code-review patch findings: bounded preview depth/field/array sanitization, operation version in log detail, server-authoritative Usage CSV export, typed retention outcome, immediate stale-response invalidation, execution outcome groups and explicit UTC windows. Re-ran targeted suites, Playwright Logs/Usage, capability/scope gates and full
just verify.
Completion Notes List
- Implemented bounded Invocation History list/detail API with status, Agent, Operation, period/search filters and opaque cursor pagination.
- Added server-side CSV export using the same filter path with row/size bounds and spreadsheet formula escaping.
- Added Usage outcome breakdown groups plus half-open UTC period aggregation for workspace/Agent/Operation usage queries.
- Added Logs UI status/Agent/Operation filters, load-more pagination, CSV export, Request ID/Trace ID details and stale-response guards; Usage UI renders outcome groups and safer CSV cells.
- Added registry/admin/UI tests including real test-run evidence, retention/boundary assertions and an ignored heavy 1,000,000-row query-plan test that passed locally.
- Updated FR-6 docs, manual checklist, capability inventory and baseline manifest checksum.
- Closed 10/10 code-review patch findings and moved Story 1.15 from
reviewtodoneafterjust verifypassed.
File List
_bmad-output/implementation-artifacts/1-15-invocation-history-usage.md_bmad-output/implementation-artifacts/sprint-status.yamlapps/admin-api/src/app.rsapps/admin-api/src/dto.rsapps/admin-api/src/main.rsapps/admin-api/src/routes/observability.rsapps/admin-api/src/service.rsapps/admin-api/src/service/agents.rsapps/admin-api/src/service/demo.rsapps/admin-api/src/service/observability.rsapps/admin-api/tests/integration.rsapps/admin-api/tests/integration/community_access_usage.rsapps/admin-api/tests/integration/logs_usage.rsapps/ui/css/logs.cssapps/ui/html/logs.htmlapps/ui/html/usage.htmlapps/ui/js/api.jsapps/ui/js/i18n.jsapps/ui/js/logs.jsapps/ui/js/usage.jsapps/ui/tests/e2e/logs-usage.spec.jscrates/crank-core/src/observability.rscrates/crank-metrics/src/schema.rscrates/crank-metrics/tests/schema_v1.rscrates/crank-registry/src/lib.rscrates/crank-registry/src/model.rscrates/crank-registry/src/postgres/mod.rscrates/crank-registry/src/postgres/observability.rscrates/crank-registry/tests/integration/agents_usage.rscrates/crank-registry/tests/integration/common.rscrates/crank-registry/tests/integration/observability.rsdocs/admin-api.mddocs/capability-baseline/manifest.jsondocs/capability-inventory.jsondocs/data-model.mddocs/manual-regression-checklist.mddocs/observability.mddocs/schemas/metrics-registry-v1.json
Change Log
- 2026-08-23 — Created Story 1.15 with FR-6 history/usage implementation context.
- 2026-08-23 — Implemented FR-6 Invocation History/Usage hardening and moved Story 1.15 to review.
- 2026-08-23 — Closed code-review patch findings, re-ran full verification and moved Story 1.15 to done.