diff --git a/README.md b/README.md index 585f993..b2afd43 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,10 @@ Crank - платформа для публикации внешних API в в - `docs/public-smoke-targets.md` - готовые публичные upstream-сервисы и payload-ы для smoke-проверки MCP. - `docs/secrets-auth-plan.md` - целевая модель upstream secrets, auth profiles и пошаговый план реализации. - `docs/streaming-mcp-plan.md` - целевая модель MCP transport streaming, upstream streaming и поэтапный план реализации. +- `docs/streaming-admin-api.md` - точные HTTP-контракты и DTO для streaming configuration, sessions и jobs. +- `docs/streaming-runtime-design.md` - функция-за-функцией разложенная streaming runtime architecture. +- `docs/streaming-ui-contract.md` - точный UI-контракт для streaming configuration и test flows. +- `docs/protocol-capability-matrix.md` - capability matrix по всем protocol families и execution modes. - `docs/rust-design.md` - правила распределения поведения в Rust. - `docs/development-rules.md` - правила разработки и workflow. - `docs/rust-code-rules.md` - Rust-specific coding rules. diff --git a/TASKS.md b/TASKS.md index 4685bf3..2123577 100644 --- a/TASKS.md +++ b/TASKS.md @@ -2,14 +2,14 @@ ## Current -### `feat/full-protocol-platform-architecture` +### `feat/streaming-detailed-specs` Status: completed DoD: -- Product scope covers REST, GraphQL, gRPC, WebSocket and SOAP -- Execution model is documented independently from protocol families -- Core docs and protocol docs are synchronized around full protocol platform scope +- Streaming API, runtime, UI and capability docs exist as separate detailed specs +- Function-level responsibilities are documented +- Field-level UI contract and validation rules are documented ## Next @@ -30,6 +30,9 @@ DoD: - `feat/soap-adapter-foundation` - `feat/streaming-ui-config` - `feat/streaming-e2e` +- `feat/websocket-upstream-adapter` +- `feat/soap-architecture-and-core-model` +- `feat/soap-adapter-foundation` - `feat/auth-profile-secret-resolution` - `feat/runtime-upstream-auth` - `feat/secrets-ui` diff --git a/docs/admin-api.md b/docs/admin-api.md index 63ea0fc..b994db8 100644 --- a/docs/admin-api.md +++ b/docs/admin-api.md @@ -4,6 +4,10 @@ Этот документ фиксирует целевые HTTP-контракты административного API, через которое UI управляет workspace, operations, agents, platform access и observability. +Для потоковой модели детальные HTTP DTO вынесены отдельно в: + +- `docs/streaming-admin-api.md` + ## 2. Общие правила API - все payload по умолчанию в `JSON`; @@ -205,6 +209,10 @@ - gRPC descriptor upload и discovery. - upstream auth selector; - quick-create secret / auth profile modal. +- execution mode selector; +- streaming config blocks; +- stream test-runs; +- tool family preview. Детальные DTO и response shapes для экранов `Operations` и `Wizard` зафиксированы отдельно в: diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 9568897..dcbc2e6 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -175,3 +175,15 @@ DoD: - runtime умеет строить SOAP envelopes и нормализовать SOAP Faults; - operator может выбрать service, port и operation; - test-run, publish и observability работают так же, как для остальных протоколов. + +## 15. Этап 14. Detailed streaming specs + +Цель: + +- довести streaming-docs до function-level и field-level design спецификации. + +DoD: + +- есть отдельные docs для streaming admin-api, runtime, UI и capability matrix; +- protocol docs не противоречат общей execution model; +- roadmap и `TASKS.md` синхронизированы с full-detail architecture. diff --git a/docs/protocol-capability-matrix.md b/docs/protocol-capability-matrix.md new file mode 100644 index 0000000..90d6568 --- /dev/null +++ b/docs/protocol-capability-matrix.md @@ -0,0 +1,123 @@ +# Protocol Capability Matrix + +## 1. Назначение документа + +Этот документ фиксирует capability matrix по всем protocol families и execution modes. + +Он нужен как source of truth для: + +- UI availability rules; +- admin-api validation; +- runtime dispatch; +- roadmap planning. + +## 2. Matrix by protocol and execution mode + +| Protocol | Unary | Window | Session | Async Job | Notes | +| --- | --- | --- | --- | --- | --- | +| REST | Yes | Yes | Yes | Yes | window/session используют SSE или bounded polling | +| GraphQL | Yes | No | No | Limited | async job только как явная status/result workflow model | +| gRPC | Yes | Yes | Yes | Yes | только unary и server-streaming | +| WebSocket | No | Yes | Yes | Yes | upstream-only adapter | +| SOAP | Yes | Limited | No | Yes | request-response first, async через separate workflow contract | + +## 3. Matrix by protocol and transport behavior + +| Protocol | Request/Response | Server Stream | Stateful Session | +| --- | --- | --- | --- | +| REST | Yes | Yes | Limited | +| GraphQL | Yes | No | No | +| gRPC | Yes | Yes | Yes | +| WebSocket | No | Yes | Yes | +| SOAP | Yes | Limited | No | + +## 4. Matrix by protocol and artifact requirements + +| Protocol | Sample JSON | Descriptor | WSDL/XSD | Query Template | Subscribe Template | +| --- | --- | --- | --- | --- | --- | +| REST | Optional | No | No | No | Optional | +| GraphQL | Optional | No | No | Yes | No | +| gRPC | Optional | Yes | No | No | No | +| WebSocket | Optional | No | No | No | Yes | +| SOAP | Optional | No | Yes | No | No | + +## 5. Matrix by protocol and auth support + +| Protocol | None | Bearer | Basic | API Key Header | API Key Query | WS-Security | +| --- | --- | --- | --- | --- | --- | --- | +| REST | Yes | Yes | Yes | Yes | Yes | No | +| GraphQL | Yes | Yes | Yes | Yes | Limited | No | +| gRPC | Yes | Yes | Yes | Limited | No | No | +| WebSocket | Yes | Yes | Yes | Yes | Limited | No | +| SOAP | Yes | Yes | Yes | Limited | Limited | Future | + +## 6. Matrix by protocol and UI blocks + +| UI Block | REST | GraphQL | gRPC | WebSocket | SOAP | +| --- | --- | --- | --- | --- | --- | +| Endpoint URL | Yes | Yes | No | Yes | Optional | +| Method selector | Yes | No | No | No | No | +| Query template | No | Yes | No | No | No | +| Descriptor upload | No | No | Yes | No | No | +| WSDL upload | No | No | No | No | Yes | +| Subscribe template | Optional | No | No | Yes | No | +| Subprotocol list | No | No | No | Yes | No | +| SOAP headers | No | No | No | No | Yes | +| Stream kind selector | Optional | No | Yes | Yes | Limited | + +## 7. Validation rules matrix + +## 7.1. Unsupported combinations + +- `GraphQL + Window` +- `GraphQL + Session` +- `WebSocket + Unary` +- `SOAP + Session` +- `gRPC + Session` для non-server-streaming method +- `REST + Server Stream` без stream-capable target model + +## 7.2. Conditionally supported combinations + +- `GraphQL + Async Job` + Требует отдельного status/result workflow contract. + +- `SOAP + Window` + Допускается только для polling-style workflow, а не для native stream. + +- `SOAP + Async Job` + Требует отдельной operation family для status/result. + +## 8. Product-wave grouping + +### Wave 1 + +- REST +- GraphQL query/mutation +- gRPC unary +- Streamable HTTP downstream +- Window/session/job core model + +### Wave 2 + +- REST SSE +- gRPC server-streaming +- session/job tool families +- streaming UI + +### Wave 3 + +- WebSocket upstream adapter +- richer streaming observability + +### Wave 4 + +- SOAP foundation +- WSDL/XSD tooling +- SOAP UI flow + +### Wave 5 + +- GraphQL subscriptions +- gRPC client-streaming +- gRPC bidi +- advanced enterprise security stacks diff --git a/docs/streaming-admin-api.md b/docs/streaming-admin-api.md new file mode 100644 index 0000000..45a9496 --- /dev/null +++ b/docs/streaming-admin-api.md @@ -0,0 +1,607 @@ +# Streaming Admin API + +## 1. Назначение документа + +Этот документ фиксирует точные HTTP-контракты для потоковой модели Crank. + +Он дополняет: + +- [admin-api.md](/home/a.tolmachev/code/rust/mcpaas/docs/admin-api.md) +- [streaming-mcp-plan.md](/home/a.tolmachev/code/rust/mcpaas/docs/streaming-mcp-plan.md) +- [streaming-ui-contract.md](/home/a.tolmachev/code/rust/mcpaas/docs/streaming-ui-contract.md) +- [streaming-runtime-design.md](/home/a.tolmachev/code/rust/mcpaas/docs/streaming-runtime-design.md) + +Цель документа: + +- зафиксировать DTO; +- зафиксировать route groups; +- зафиксировать валидацию; +- зафиксировать expected error model; +- зафиксировать page-to-endpoint contract для streaming configuration и test-runs. + +## 2. Общие принципы + +- все ресурсы являются `workspace-scoped`; +- streaming configuration является частью `operation version`; +- `session` и `async_job` state не редактируются напрямую из UI; +- test-runs могут создавать временные sessions и jobs, но не публикуют их как runtime resources; +- transport errors и validation errors разделяются; +- лимиты и safety-параметры валидируются на сервере, а не только в UI. + +Базовый префикс: + +```text +/api/admin/workspaces/{workspace_id} +``` + +## 3. Основные ресурсы + +- `streaming-presets` +- `streaming-validation` +- `stream-test-runs` +- `stream-sessions` +- `async-jobs` +- `protocol-capabilities` + +## 4. Общие DTO + +### 4.1. `ExecutionMode` + +```json +{ + "mode": "unary" +} +``` + +Допустимые значения: + +- `unary` +- `window` +- `session` +- `async_job` + +### 4.2. `StreamingConfig` + +```json +{ + "mode": "window", + "transport_behavior": "server_stream", + "window_duration_ms": 5000, + "poll_interval_ms": 2000, + "upstream_timeout_ms": 10000, + "idle_timeout_ms": 30000, + "max_session_lifetime_ms": 300000, + "max_items": 200, + "max_bytes": 131072, + "aggregation_mode": "summary_plus_samples", + "summary_path": "$.summary", + "items_path": "$.items", + "cursor_path": "$.cursor", + "status_path": "$.status", + "done_path": "$.done", + "redacted_paths": [ + "$.items[*].token", + "$.summary.secret" + ], + "truncate_item_fields": true, + "max_field_length": 512, + "drop_duplicates": true, + "sampling_rate": 1.0, + "tool_family": { + "start_tool_name": "cluster_events_start", + "poll_tool_name": "cluster_events_poll", + "stop_tool_name": "cluster_events_stop", + "status_tool_name": "deploy_status", + "result_tool_name": "deploy_result", + "cancel_tool_name": "deploy_cancel" + } +} +``` + +### 4.3. `StreamingValidationError` + +```json +{ + "code": "streaming_validation_error", + "message": "Streaming configuration is invalid", + "details": [ + { + "field": "window_duration_ms", + "reason": "must_be_positive" + }, + { + "field": "max_items", + "reason": "must_not_exceed_workspace_limit" + } + ] +} +``` + +### 4.4. `ProtocolCapability` + +```json +{ + "protocol": "grpc", + "supports_execution_modes": [ + "unary", + "window", + "session", + "async_job" + ], + "supports_transport_behaviors": [ + "request_response", + "server_stream" + ], + "supports_auth_kinds": [ + "none", + "bearer", + "basic", + "api_key_header", + "api_key_query" + ], + "supports_upload_artifacts": [ + "proto", + "descriptor_set" + ], + "supports_cursor_path": true, + "supports_done_path": true, + "supports_aggregation_mode": [ + "raw_items", + "summary_only", + "summary_plus_samples", + "stats", + "latest_state" + ] +} +``` + +## 5. Capabilities endpoints + +### `GET /api/admin/workspaces/{workspace_id}/protocol-capabilities` + +Назначение: + +- отдать UI полную capability matrix; +- убрать protocol-specific hardcode из frontend. + +Ответ: + +```json +{ + "items": [ + { + "protocol": "rest", + "supports_execution_modes": ["unary", "window", "session", "async_job"], + "supports_transport_behaviors": ["request_response", "server_stream"], + "supports_auth_kinds": ["none", "bearer", "basic", "api_key_header", "api_key_query"], + "supports_upload_artifacts": [], + "supports_cursor_path": true, + "supports_done_path": true, + "supports_aggregation_mode": ["raw_items", "summary_only", "summary_plus_samples", "stats", "latest_state"] + } + ] +} +``` + +## 6. Streaming validation endpoints + +### `POST /api/admin/workspaces/{workspace_id}/streaming/validate` + +Назначение: + +- проверить streaming config до сохранения operation; +- вернуть protocol-aware ошибки. + +Тело: + +```json +{ + "protocol": "websocket", + "target": {}, + "execution_config": { + "streaming": {} + } +} +``` + +Успех: + +```json +{ + "valid": true, + "warnings": [ + { + "field": "max_bytes", + "code": "may_truncate_large_event_payloads", + "message": "Large event payloads may be truncated" + } + ] +} +``` + +Ошибка: + +```json +{ + "valid": false, + "errors": [ + { + "field": "transport_behavior", + "code": "unsupported_transport_behavior" + } + ] +} +``` + +## 7. Streaming presets endpoints + +### `GET /api/admin/workspaces/{workspace_id}/streaming-presets` + +Назначение: + +- отдать рекомендованные UI presets. + +Ответ: + +```json +{ + "items": [ + { + "preset_id": "logs_window_5s", + "display_name": "Logs Window 5s", + "protocols": ["rest", "grpc", "websocket"], + "streaming": { + "mode": "window", + "window_duration_ms": 5000, + "max_items": 100, + "max_bytes": 65536, + "aggregation_mode": "summary_plus_samples" + } + } + ] +} +``` + +## 8. Streaming operation test-runs + +### `POST /api/admin/workspaces/{workspace_id}/operations/{operation_id}/stream-test-runs` + +Назначение: + +- выполнить `window`, `session` или `async_job` test-run для draft version; +- показать оператору runtime behavior до publish. + +Тело: + +```json +{ + "version": 4, + "input": { + "service": "billing", + "level": "error" + }, + "test_mode": "window", + "overrides": { + "streaming": { + "window_duration_ms": 3000, + "max_items": 50 + } + } +} +``` + +Успех для `window`: + +```json +{ + "run_id": "strun_01j0test", + "mode": "window", + "status": "completed", + "window_complete": true, + "truncated": false, + "has_more": false, + "summary": {}, + "items": [], + "cursor": null, + "duration_ms": 2871 +} +``` + +Успех для `session`: + +```json +{ + "run_id": "strun_01j0test", + "mode": "session", + "status": "running", + "session": { + "session_id": "sess_01j0stream", + "expires_at": "2026-04-06T12:05:00Z", + "poll_after_ms": 2000 + }, + "preview": { + "summary": {}, + "items": [] + } +} +``` + +Успех для `async_job`: + +```json +{ + "run_id": "strun_01j0test", + "mode": "async_job", + "status": "running", + "job": { + "job_id": "job_01j0deploy", + "status": "running", + "progress": { + "pct": 12 + } + } +} +``` + +### `POST /api/admin/workspaces/{workspace_id}/operations/{operation_id}/stream-test-runs/{run_id}/poll` + +Назначение: + +- продолжить session-oriented test-run. + +Тело: + +```json +{ + "session_id": "sess_01j0stream" +} +``` + +Ответ: + +```json +{ + "run_id": "strun_01j0test", + "mode": "session", + "status": "running", + "window_complete": false, + "truncated": false, + "has_more": true, + "summary": {}, + "items": [], + "cursor": "next_cursor" +} +``` + +### `POST /api/admin/workspaces/{workspace_id}/operations/{operation_id}/stream-test-runs/{run_id}/stop` + +Назначение: + +- остановить test session. + +### `GET /api/admin/workspaces/{workspace_id}/operations/{operation_id}/stream-test-runs/{run_id}/result` + +Назначение: + +- получить финальный результат async-job test-run. + +## 9. Runtime session resources + +### `GET /api/admin/workspaces/{workspace_id}/stream-sessions` + +Назначение: + +- список активных и недавних session resources для observability/debug UI. + +Query params: + +- `operation_id` +- `agent_id` +- `status` +- `page` +- `page_size` + +Ответ: + +```json +{ + "items": [ + { + "id": "sess_01j0stream", + "operation_id": "op_01j0", + "agent_id": "agent_01j0", + "mode": "session", + "status": "running", + "expires_at": "2026-04-06T12:05:00Z", + "last_poll_at": "2026-04-06T12:00:10Z", + "created_at": "2026-04-06T12:00:00Z" + } + ], + "page": 1, + "page_size": 20, + "total": 1 +} +``` + +### `GET /api/admin/workspaces/{workspace_id}/stream-sessions/{session_id}` + +Назначение: + +- detail session metadata; +- без возврата полного raw state. + +### `POST /api/admin/workspaces/{workspace_id}/stream-sessions/{session_id}/stop` + +Назначение: + +- административно остановить runtime session. + +### `DELETE /api/admin/workspaces/{workspace_id}/stream-sessions/{session_id}` + +Назначение: + +- hard cleanup завершенной session; +- доступен только для `stopped`, `failed`, `expired`. + +## 10. Async job resources + +### `GET /api/admin/workspaces/{workspace_id}/async-jobs` + +Назначение: + +- список активных и недавних job resources. + +### `GET /api/admin/workspaces/{workspace_id}/async-jobs/{job_id}` + +Назначение: + +- status/progress/result metadata. + +### `POST /api/admin/workspaces/{workspace_id}/async-jobs/{job_id}/cancel` + +Назначение: + +- административная отмена long-running job. + +### `GET /api/admin/workspaces/{workspace_id}/async-jobs/{job_id}/result` + +Назначение: + +- получить финальный нормализованный результат. + +## 11. Integration into operation contracts + +`streaming` не является отдельным top-level resource для опубликованной операции. Он живет внутри: + +- `OperationVersionDocument.execution_config.streaming` + +и входит в: + +- `POST /operations` +- `PATCH /operations/{operation_id}` +- `POST /operations/{operation_id}/versions` +- `GET /operations/{operation_id}/versions/{version}` + +### 11.1. `OperationVersionDocument.execution_config.streaming` + +```json +{ + "execution_config": { + "timeout_ms": 10000, + "retries": 0, + "auth_profile_ref": "auth_profile_01j0", + "streaming": { + "mode": "window", + "transport_behavior": "server_stream", + "window_duration_ms": 5000, + "upstream_timeout_ms": 10000, + "max_items": 100, + "max_bytes": 65536, + "aggregation_mode": "summary_plus_samples", + "summary_path": "$.summary", + "items_path": "$.items", + "cursor_path": "$.cursor", + "status_path": "$.status", + "done_path": "$.done", + "redacted_paths": [], + "truncate_item_fields": true, + "max_field_length": 512, + "drop_duplicates": true, + "sampling_rate": 1.0, + "tool_family": {} + } + } +} +``` + +## 12. Validation rules + +### 12.1. Общие + +- `mode=unary` запрещает `tool_family`; +- `max_items` > 0; +- `max_bytes` > 0; +- `window_duration_ms` > 0 для `window`; +- `idle_timeout_ms` обязателен для `session`; +- `max_session_lifetime_ms` обязателен для `session`; +- `tool_family.start_tool_name/poll_tool_name/stop_tool_name` обязательны для `session`; +- `tool_family.start_tool_name/status_tool_name/result_tool_name/cancel_tool_name` обязательны для `async_job`. + +### 12.2. Protocol-aware + +REST: + +- `transport_behavior=server_stream` допустим только при streaming-capable target. + +GraphQL: + +- `window`, `session`, `async_job` пока запрещены; +- `subscription` невалиден как target type. + +gRPC: + +- `transport_behavior=server_stream` допустим только для server-streaming method. + +WebSocket: + +- `mode=unary` запрещен; +- требуется `subscribe_message_template` для `session` и `window`. + +SOAP: + +- `mode=session` запрещен в первой волне; +- `mode=window` допустим только при наличии polling-style status operation family; +- `mode=async_job` требует status/result contract. + +## 13. Error model + +Коды: + +- `streaming_validation_error` +- `unsupported_execution_mode` +- `unsupported_transport_behavior` +- `stream_window_timeout` +- `stream_window_truncated` +- `stream_session_expired` +- `stream_session_not_found` +- `async_job_not_found` +- `async_job_not_ready` +- `async_job_cancelled` +- `protocol_capability_mismatch` + +### 13.1. Ошибка несовместимого protocol mode + +```json +{ + "code": "protocol_capability_mismatch", + "message": "websocket target does not support unary execution mode", + "details": [ + { + "field": "execution_config.streaming.mode", + "reason": "unsupported_for_protocol" + } + ] +} +``` + +## 14. Route-to-service mapping + +Ожидаемые service handlers в `apps/admin-api`: + +- `list_protocol_capabilities` +- `list_streaming_presets` +- `validate_streaming_config` +- `start_stream_test_run` +- `poll_stream_test_run` +- `stop_stream_test_run` +- `get_stream_test_result` +- `list_stream_sessions` +- `get_stream_session` +- `stop_stream_session` +- `delete_stream_session` +- `list_async_jobs` +- `get_async_job` +- `cancel_async_job` +- `get_async_job_result` + +Это не route names, а service-level функции orchestration. diff --git a/docs/streaming-runtime-design.md b/docs/streaming-runtime-design.md new file mode 100644 index 0000000..57d0877 --- /dev/null +++ b/docs/streaming-runtime-design.md @@ -0,0 +1,580 @@ +# Streaming Runtime Design + +## 1. Назначение документа + +Этот документ фиксирует, как потоковая модель должна быть разложена по crates, traits, services и функциям. + +Цель: + +- избежать god-services; +- зафиксировать boundaries между `core`, `registry`, `runtime`, protocol adapters и `mcp-server`; +- расписать ожидаемые функции и их ответственность до уровня инженерной спецификации. + +## 2. Общая схема слоев + +### 2.1. `crank-core` + +Отвечает за: + +- типы `ExecutionMode`, `StreamingConfig`, `AggregationMode`; +- типы `StreamSession`, `AsyncJobHandle`, `StreamStatus`, `JobStatus`; +- validation helpers без инфраструктуры. + +### 2.2. `crank-registry` + +Отвечает за: + +- persistence session/job state; +- optimistic transitions; +- cleanup expired state. + +### 2.3. `crank-runtime` + +Отвечает за: + +- orchestration execution modes; +- dispatch в protocol adapter; +- bounded collection; +- aggregation; +- state transitions; +- invocation logging. + +### 2.4. Protocol adapters + +Отвечают за: + +- protocol-specific upstream behavior; +- connect/request/collect/close; +- преобразование upstream payload в нормализованный JSON. + +### 2.5. `apps/mcp-server` + +Отвечает за: + +- downstream MCP transport; +- tool-family generation; +- JSON-RPC lifecycle; +- transport/session routing. + +### 2.6. `apps/admin-api` + +Отвечает за: + +- config CRUD; +- test-runs; +- validation; +- UI-oriented metadata and previews. + +## 3. Core domain types + +## 3.1. `ExecutionMode` + +```rust +pub enum ExecutionMode { + Unary, + Window, + Session, + AsyncJob, +} +``` + +Методы: + +- `fn is_stateful(&self) -> bool` +- `fn requires_tool_family(&self) -> bool` + +## 3.2. `AggregationMode` + +```rust +pub enum AggregationMode { + RawItems, + SummaryOnly, + SummaryPlusSamples, + Stats, + LatestState, +} +``` + +Методы: + +- `fn needs_items(&self) -> bool` +- `fn needs_summary(&self) -> bool` + +## 3.3. `StreamingConfig` + +```rust +pub struct StreamingConfig { + pub mode: ExecutionMode, + pub transport_behavior: TransportBehavior, + pub window_duration_ms: Option, + pub poll_interval_ms: Option, + pub upstream_timeout_ms: Option, + pub idle_timeout_ms: Option, + pub max_session_lifetime_ms: Option, + pub max_items: Option, + pub max_bytes: Option, + pub aggregation_mode: AggregationMode, + pub summary_path: Option, + pub items_path: Option, + pub cursor_path: Option, + pub status_path: Option, + pub done_path: Option, + pub redacted_paths: Vec, + pub truncate_item_fields: bool, + pub max_field_length: Option, + pub drop_duplicates: bool, + pub sampling_rate: Option, + pub tool_family: ToolFamilyConfig, +} +``` + +Методы: + +- `fn validate_common(&self) -> Result<(), StreamingConfigError>` +- `fn validate_for_protocol(&self, protocol: Protocol) -> Result<(), StreamingConfigError>` +- `fn effective_max_items(&self, limits: &StreamingLimits) -> u32` +- `fn effective_max_bytes(&self, limits: &StreamingLimits) -> u32` +- `fn effective_timeout(&self, defaults: &StreamingDefaults) -> Duration` + +## 3.4. `StreamSession` + +```rust +pub struct StreamSession { + pub id: StreamSessionId, + pub workspace_id: WorkspaceId, + pub agent_id: Option, + pub operation_id: OperationId, + pub protocol: Protocol, + pub mode: ExecutionMode, + pub status: StreamStatus, + pub cursor: Option, + pub state: Value, + pub expires_at: DateTime, + pub last_poll_at: Option>, + pub created_at: DateTime, + pub closed_at: Option>, +} +``` + +Методы: + +- `fn is_expired(&self, now: DateTime) -> bool` +- `fn can_poll(&self, now: DateTime) -> bool` +- `fn mark_polled(&mut self, now: DateTime)` +- `fn mark_closed(&mut self, now: DateTime)` + +## 3.5. `AsyncJobHandle` + +```rust +pub struct AsyncJobHandle { + pub id: AsyncJobId, + pub workspace_id: WorkspaceId, + pub agent_id: Option, + pub operation_id: OperationId, + pub status: JobStatus, + pub progress: Value, + pub result: Option, + pub error: Option, + pub expires_at: Option>, + pub created_at: DateTime, + pub updated_at: DateTime, + pub finished_at: Option>, +} +``` + +Методы: + +- `fn is_finished(&self) -> bool` +- `fn can_cancel(&self) -> bool` +- `fn mark_finished(&mut self, now: DateTime, result: Value)` +- `fn mark_failed(&mut self, now: DateTime, error: Value)` + +## 4. Registry contracts + +## 4.1. `StreamSessionRepository` + +```rust +#[async_trait] +pub trait StreamSessionRepository { + async fn create_stream_session(&self, session: NewStreamSession) -> Result; + async fn get_stream_session(&self, id: &StreamSessionId) -> Result, RegistryError>; + async fn update_stream_session_state(&self, update: StreamSessionStateUpdate) -> Result; + async fn close_stream_session(&self, id: &StreamSessionId, now: DateTime) -> Result<(), RegistryError>; + async fn list_stream_sessions(&self, filter: StreamSessionFilter) -> Result, RegistryError>; + async fn delete_expired_stream_sessions(&self, now: DateTime) -> Result; +} +``` + +## 4.2. `AsyncJobRepository` + +```rust +#[async_trait] +pub trait AsyncJobRepository { + async fn create_async_job(&self, job: NewAsyncJob) -> Result; + async fn get_async_job(&self, id: &AsyncJobId) -> Result, RegistryError>; + async fn update_async_job_status(&self, update: AsyncJobStatusUpdate) -> Result; + async fn cancel_async_job(&self, id: &AsyncJobId, now: DateTime) -> Result<(), RegistryError>; + async fn list_async_jobs(&self, filter: AsyncJobFilter) -> Result, RegistryError>; + async fn delete_expired_async_jobs(&self, now: DateTime) -> Result; +} +``` + +## 4.3. Storage rules + +- update methods должны использовать optimistic concurrency, если state transitions конфликтуют; +- `poll` не должен silently reopen closed session; +- cleanup job должен быть отдельным service; +- session/job state не должен хранить необрезанные raw payloads без лимитов. + +## 5. Runtime contracts + +## 5.1. `ProtocolAdapter` + +Базовый trait не должен пытаться описать все streaming cases в одном методе. + +```rust +#[async_trait] +pub trait ProtocolAdapter { + async fn execute_unary(&self, ctx: UnaryExecutionContext) -> Result; + async fn execute_window(&self, ctx: WindowExecutionContext) -> Result; + async fn start_session(&self, ctx: SessionStartContext) -> Result; + async fn poll_session(&self, ctx: SessionPollContext) -> Result; + async fn stop_session(&self, ctx: SessionStopContext) -> Result<(), AdapterError>; + async fn start_async_job(&self, ctx: AsyncJobStartContext) -> Result; + async fn get_async_job_status(&self, ctx: AsyncJobStatusContext) -> Result; + async fn get_async_job_result(&self, ctx: AsyncJobResultContext) -> Result; + async fn cancel_async_job(&self, ctx: AsyncJobCancelContext) -> Result<(), AdapterError>; +} +``` + +Не каждый adapter обязан поддерживать все методы. Capability mismatch должен проверяться выше. + +## 5.2. `OperationExecutor` + +Главная orchestration точка в `crank-runtime`. + +Ожидаемые функции: + +- `execute_unary_operation` +- `execute_window_operation` +- `start_stream_session` +- `poll_stream_session` +- `stop_stream_session` +- `start_async_job` +- `get_async_job_status` +- `get_async_job_result` +- `cancel_async_job` + +### `execute_unary_operation` + +Отвечает за: + +- schema validation; +- auth resolution; +- adapter dispatch; +- output mapping; +- observability write. + +### `execute_window_operation` + +Отвечает за: + +- schema validation; +- auth resolution; +- adapter dispatch в `execute_window`; +- aggregation; +- truncation flags; +- observability write. + +### `start_stream_session` + +Отвечает за: + +- schema validation; +- auth resolution; +- adapter `start_session`; +- registry `create_stream_session`; +- initial preview result; +- observability write. + +### `poll_stream_session` + +Отвечает за: + +- load session from registry; +- expiration check; +- adapter `poll_session`; +- registry `update_stream_session_state`; +- output mapping; +- observability write. + +### `stop_stream_session` + +Отвечает за: + +- load session; +- adapter `stop_session`; +- registry close; +- observability write. + +### `start_async_job` + +Отвечает за: + +- schema validation; +- adapter `start_async_job`; +- registry `create_async_job`; +- observability write. + +### `get_async_job_status` + +Отвечает за: + +- registry load; +- adapter `get_async_job_status`, если upstream status is live; +- registry update; +- normalized status result. + +### `get_async_job_result` + +Отвечает за: + +- job readiness check; +- adapter `get_async_job_result`, если result lazy-loaded; +- output mapping; +- observability write. + +### `cancel_async_job` + +Отвечает за: + +- adapter cancel; +- registry cancel transition; +- observability write. + +## 5.3. Aggregation services + +Нужно отделить aggregation от adapters. + +Отдельные services: + +- `WindowAggregator` +- `SummaryBuilder` +- `CursorTracker` +- `PayloadLimiter` +- `RedactionService` + +### `WindowAggregator` + +Функции: + +- `collect_items` +- `apply_item_limit` +- `apply_byte_limit` +- `mark_truncated` + +### `SummaryBuilder` + +Функции: + +- `build_summary` +- `build_stats_summary` +- `build_latest_state_summary` +- `build_summary_plus_samples` + +### `PayloadLimiter` + +Функции: + +- `truncate_item_fields` +- `truncate_bytes` +- `enforce_max_items` + +### `RedactionService` + +Функции: + +- `redact_paths` +- `redact_object` +- `redact_item` + +## 6. Adapter responsibilities + +## 6.1. REST adapter + +Функции: + +- `execute_rest_unary` +- `execute_rest_window` +- `start_rest_session` +- `poll_rest_session` +- `stop_rest_session` +- `start_rest_async_job` +- `get_rest_async_job_status` +- `get_rest_async_job_result` +- `cancel_rest_async_job` + +Дополнительные helpers: + +- `open_sse_stream` +- `collect_sse_window` +- `parse_sse_event` +- `close_sse_stream` + +## 6.2. GraphQL adapter + +Функции: + +- `execute_graphql_unary` + +Отдельно отложено: + +- `execute_graphql_subscription_window` +- `start_graphql_subscription_session` + +## 6.3. gRPC adapter + +Функции: + +- `execute_grpc_unary` +- `execute_grpc_window` +- `start_grpc_session` +- `poll_grpc_session` +- `stop_grpc_session` +- `start_grpc_async_job` +- `get_grpc_async_job_status` +- `get_grpc_async_job_result` +- `cancel_grpc_async_job` + +Helpers: + +- `invoke_unary_method` +- `open_server_stream` +- `collect_server_stream_window` +- `decode_stream_item` + +## 6.4. WebSocket adapter + +Функции: + +- `execute_websocket_window` +- `start_websocket_session` +- `poll_websocket_session` +- `stop_websocket_session` +- `start_websocket_async_job` +- `get_websocket_async_job_status` +- `cancel_websocket_async_job` + +Helpers: + +- `connect_websocket` +- `send_subscribe_message` +- `send_unsubscribe_message` +- `read_next_frame` +- `decode_text_frame` +- `heartbeat_tick` +- `reconnect_if_needed` + +## 6.5. SOAP adapter + +Функции: + +- `execute_soap_unary` +- `start_soap_async_job` +- `get_soap_async_job_status` +- `get_soap_async_job_result` +- `cancel_soap_async_job` + +Helpers: + +- `render_soap_envelope` +- `render_soap_headers` +- `parse_soap_envelope` +- `parse_soap_fault` +- `normalize_xml_value` + +## 7. MCP server design + +## 7.1. Route handlers + +Ожидаемые handlers: + +- `handle_initialize` +- `handle_notifications_initialized` +- `handle_ping` +- `handle_tools_list` +- `handle_tools_call` +- `handle_get_sse_stream` +- `handle_delete_session` + +## 7.2. Tool-family generation + +Функции: + +- `build_unary_tool_definition` +- `build_window_tool_definition` +- `build_session_tool_family` +- `build_async_job_tool_family` + +Для `session`: + +- `build_session_start_tool` +- `build_session_poll_tool` +- `build_session_stop_tool` + +Для `async_job`: + +- `build_async_job_start_tool` +- `build_async_job_status_tool` +- `build_async_job_result_tool` +- `build_async_job_cancel_tool` + +## 7.3. Transport session management + +Функции: + +- `ensure_protocol_version` +- `resolve_transport_mode` +- `ensure_mcp_session` +- `attach_session_headers` +- `stream_sse_event` +- `close_transport_session` + +## 8. Admin API service design + +Ожидаемые service функции: + +- `validate_streaming_config` +- `list_protocol_capabilities` +- `list_streaming_presets` +- `start_stream_test_run` +- `poll_stream_test_run` +- `stop_stream_test_run` +- `get_stream_test_result` +- `list_stream_sessions` +- `get_stream_session` +- `stop_stream_session` +- `delete_stream_session` +- `list_async_jobs` +- `get_async_job` +- `cancel_async_job` +- `get_async_job_result` + +## 9. Cleanup jobs + +Отдельные jobs: + +- `expire_stream_sessions` +- `expire_async_jobs` +- `reap_orphaned_transport_sessions` +- `compact_stream_payloads`, если будет нужен storage optimization layer + +## 10. Design rules + +- adapters не агрегируют product-level summaries; +- runtime не знает деталей transport framing; +- `mcp-server` не знает про upstream protocols; +- `admin-api` не управляет live protocol connections напрямую; +- session/job lifecycle должен быть явным в names и contracts; +- cancellation не должна зависеть от TCP disconnect; +- bounded limits должны применяться раньше, чем payload попадет в final result. diff --git a/docs/streaming-ui-contract.md b/docs/streaming-ui-contract.md new file mode 100644 index 0000000..4c6e9aa --- /dev/null +++ b/docs/streaming-ui-contract.md @@ -0,0 +1,434 @@ +# Streaming UI Contract + +## 1. Назначение документа + +Этот документ фиксирует точный UI-контракт для настройки потоковых операций. + +Цель: + +- определить экраны и блоки wizard; +- перечислить все поля; +- перечислить валидации; +- определить protocol-specific visibility rules; +- зафиксировать UX states, warnings и system messages. + +## 2. Основные экраны + +Потоковая конфигурация живет в: + +- `Operations Wizard` +- `Operation Detail` +- `Test Run` +- `Agent Tool Preview` +- `Logs` / `Usage` observability surfaces + +Дополнительно админские страницы: + +- `Stream Sessions` +- `Async Jobs` + +## 3. Wizard information architecture + +## 3.1. Shared wizard structure + +Шаги: + +1. `Protocol` +2. `Upstream` +3. `Request / Subscription` +4. `Execution Mode` +5. `Input Schema` +6. `Output / Aggregation` +7. `Safety and Limits` +8. `Tool Family` +9. `Test Run` +10. `Publish` + +## 3.2. Step `Execution Mode` + +Поля: + +- `Mode` + - `Unary` + - `Window` + - `Session` + - `Async Job` +- `Transport behavior` + - `Request / response` + - `Server stream` + +Подсказки: + +- `Unary` для обычных request-response integrations. +- `Window` для bounded snapshots из stream или feed. +- `Session` для follow-style tools с `start/poll/stop`. +- `Async Job` для long-running operations с `status/result/cancel`. + +Валидации: + +- `Transport behavior=server_stream` не может использоваться с protocol, который его не поддерживает. +- `WebSocket` не может быть `Unary`. +- `SOAP` в первой волне не может быть `Session`. + +## 3.3. Step `Input Schema` + +Поля: + +- `Input schema source` + - manual + - sample-derived + - descriptor/wsdl-derived +- `Input fields` +- `Required fields` +- `Defaults` +- `Mapping preview` + +## 3.4. Step `Output / Aggregation` + +Поля: + +- `Aggregation mode` + - `Raw items` + - `Summary only` + - `Summary + samples` + - `Stats` + - `Latest state` +- `Items path` +- `Summary path` +- `Cursor path` +- `Status path` +- `Done path` +- `Output mapping` + +Валидации: + +- `Items path` обязателен для `Raw items` и `Summary + samples`. +- `Summary path` обязателен для `Summary only`, `Summary + samples`, `Stats`, `Latest state`. +- `Done path` обязателен для `Async Job`, если upstream status не выражается отдельным field set. + +## 3.5. Step `Safety and Limits` + +Поля: + +- `Window duration` +- `Poll interval` +- `Upstream timeout` +- `Idle timeout` +- `Session lifetime` +- `Max items` +- `Max bytes` +- `Max field length` +- `Drop duplicates` +- `Sampling rate` +- `Redacted paths` + +Валидации: + +- `Window duration` обязателен для `Window`. +- `Poll interval` обязателен для `Session`. +- `Idle timeout` обязателен для `Session`. +- `Session lifetime` обязателен для `Session`. +- `Max items` > 0. +- `Max bytes` > 0. +- `Sampling rate` > 0 and <= 1. + +## 3.6. Step `Tool Family` + +Показывается только для: + +- `Session` +- `Async Job` + +Для `Session`: + +- `Start tool name` +- `Poll tool name` +- `Stop tool name` + +Для `Async Job`: + +- `Start tool name` +- `Status tool name` +- `Result tool name` +- `Cancel tool name` + +Валидации: + +- имена обязательны; +- имена должны быть уникальны в пределах agent; +- имена не должны конфликтовать с already bound tools. + +## 4. Protocol-specific UI + +## 4.1. REST + +Поля: + +- `Base URL` +- `HTTP method` +- `Path template` +- `Headers` +- `Query mapping` +- `Body mapping` +- `SSE enabled` +- `SSE event filter` + +Visibility: + +- `SSE enabled` показывается только если method/endpoint допускают stream use case; +- `Window` и `Session` доступны, если оператор включает stream behavior. + +## 4.2. GraphQL + +Поля: + +- `Endpoint` +- `Operation type` +- `Operation name` +- `Query template` +- `Variables schema` +- `Response path` + +Visibility: + +- `Window`, `Session`, `Async Job` скрыты в текущей продуктовой волне; +- `Subscription` UI не показывается. + +## 4.3. gRPC + +Поля: + +- `Server address` +- `Package` +- `Service` +- `Method` +- `Descriptor source` +- `Stream kind` + - `Unary` + - `Server streaming` + +Visibility: + +- `Window`, `Session`, `Async Job` доступны только для `Server streaming`; +- `Client streaming` и `Bidirectional` не показываются вообще. + +## 4.4. WebSocket + +Поля: + +- `WebSocket URL` +- `Subprotocols` +- `Connect timeout` +- `Heartbeat interval` +- `Subscribe message template` +- `Unsubscribe message template` +- `Message envelope path` + +Visibility: + +- `Unary` не показывается; +- `Window`, `Session`, `Async Job` доступны всегда; +- `Subscribe message template` обязательно для `Window` и `Session`. + +## 4.5. SOAP + +Поля: + +- `WSDL source` +- `Service` +- `Port` +- `Operation` +- `SOAP version` +- `SOAPAction` +- `Endpoint override` +- `Header config` + +Visibility: + +- `Session` скрыт; +- `Window` скрыт по умолчанию и включается только для polling-style enterprise workflows; +- `Unary` и `Async Job` доступны. + +## 5. Test Run UX + +## 5.1. Window + +UI должен показывать: + +- `Status` +- `Duration` +- `Items count` +- `Bytes count` +- `Window complete` +- `Truncated` +- `Has more` +- `Summary` +- `Items preview` +- `Cursor` + +Кнопки: + +- `Run window test` +- `Repeat` +- `Save config` + +## 5.2. Session + +UI должен показывать: + +- `Session id` +- `Status` +- `Expires at` +- `Poll after` +- `Summary preview` +- `Items preview` + +Кнопки: + +- `Start session` +- `Poll next chunk` +- `Stop session` + +## 5.3. Async Job + +UI должен показывать: + +- `Job id` +- `Status` +- `Progress` +- `Started at` +- `Finished at` +- `Result preview` + +Кнопки: + +- `Start job` +- `Refresh status` +- `Get result` +- `Cancel job` + +## 6. Page-level states + +Каждый streaming-aware экран обязан поддерживать: + +- `idle` +- `validating` +- `saving` +- `testing` +- `running` +- `completed` +- `failed` +- `stopped` +- `expired` + +## 7. Warnings and messages + +## 7.1. Validation warnings + +- `This mode will truncate responses above the configured byte limit.` +- `Current protocol does not support the selected execution mode.` +- `Session lifetime is shorter than idle timeout.` +- `Aggregation mode summary_only hides raw items from the final tool output.` +- `WebSocket reconnect may duplicate events if upstream does not provide cursor semantics.` +- `SOAP async mode requires a separate status/result contract.` + +## 7.2. Confirmations + +- `Stop current session?` +- `Cancel running job?` +- `Switching execution mode will reset protocol-specific fields.` + +## 7.3. Errors + +- `Streaming configuration is invalid.` +- `Test session expired.` +- `Upstream did not return any messages within the configured window.` +- `The collected stream payload was truncated by size limits.` +- `This protocol does not support the selected execution mode.` + +## 8. Stream Sessions page + +Колонки: + +- `Session ID` +- `Operation` +- `Agent` +- `Mode` +- `Status` +- `Created` +- `Last poll` +- `Expires` + +Actions: + +- `Open` +- `Stop` +- `Delete` + +Detail view: + +- `Session metadata` +- `Cursor preview` +- `State summary` +- `Recent events summary` + +## 9. Async Jobs page + +Колонки: + +- `Job ID` +- `Operation` +- `Agent` +- `Status` +- `Progress` +- `Created` +- `Updated` +- `Finished` + +Actions: + +- `Open` +- `Cancel` +- `Get result` + +Detail view: + +- `Job metadata` +- `Progress payload` +- `Result preview` +- `Error preview` + +## 10. Exact frontend adapters + +Ожидаемые frontend modules: + +- `streaming-form.js` +- `streaming-validation.js` +- `stream-test-run.js` +- `stream-sessions.js` +- `async-jobs.js` + +Ожидаемые frontend functions: + +- `loadProtocolCapabilities()` +- `applyStreamingPreset()` +- `validateStreamingConfig()` +- `serializeStreamingConfig()` +- `deserializeStreamingConfig()` +- `startWindowTest()` +- `startSessionTest()` +- `pollSessionTest()` +- `stopSessionTest()` +- `startAsyncJobTest()` +- `refreshAsyncJobStatus()` +- `loadAsyncJobResult()` + +## 11. Shared UI rules + +- UI никогда не должен предлагать unsupported mode; +- UI должен строить availability по `protocol-capabilities`, а не по hardcoded if-else; +- UI должен явно объяснять, почему поле скрыто или disabled; +- UI должен всегда показывать итоговую tool topology: + - `1 tool` + - `3 tools start/poll/stop` + - `4 tools start/status/result/cancel`