diff --git a/AGENTS.md b/AGENTS.md index 7f084f3..c3cacb5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,6 +18,10 @@ Use the documents in this order when there is ambiguity: 8. `docs/development-rules.md` 9. `docs/rust-code-rules.md` 10. `docs/implementation-plan.md` +11. `docs/product-editions.md` +12. `docs/commercial-boundaries.md` +13. `docs/frontend-roadmap.md` +14. `docs/refactoring-roadmap.md` If code and docs diverge, update docs first or together with code. diff --git a/TASKS.md b/TASKS.md index b45e765..d569fe6 100644 --- a/TASKS.md +++ b/TASKS.md @@ -2,30 +2,327 @@ ## Current -### `feat/agent-scoped-machine-auth` +### `feat/community-agent-key-cutover` + +Status: in_progress + +Goal: +- довести открытую редакцию до продуктово завершенного машинного доступа через ключ AI-агента. + +Main code areas: +- `crates/crank-core/src/agent.rs` +- `crates/crank-core/src/auth.rs` +- `crates/crank-registry/src/postgres/agent.rs` +- `crates/crank-registry/src/postgres/api_key.rs` +- `apps/admin-api/src/service.rs` +- `apps/admin-api/src/app.rs` +- `apps/mcp-server/src/app.rs` +- `apps/ui/js/agents.js` +- `apps/ui/js/api-keys.js` +- `apps/ui/html/agents.html` +- `apps/ui/html/api-keys.html` +- `apps/ui/js/i18n.js` + +Implementation slices: +1. Закрыть remaining workspace/platform-key assumptions в `admin-api` и `mcp-server`. +2. Нормализовать list/create/revoke/delete flow для agent keys. +3. Привести UI страницы `Agents` и `API Keys` к одной модели agent-scoped machine access. +4. Явно ограничить `Community` на `security_level = standard`. + +Progress: +- done: `mcp-server` принимает ключ только по `(workspace_slug, agent_slug, secret_hash)`; +- done: `admin-api` уже отдает и принимает agent-scoped key routes; +- done: UI `API Keys` переведен на agent-scoped machine access; +- done: removed legacy workspace-key routes, demo/bootstrap assumptions, and per-agent key count bug. + +DoD: +- ключ создается и принадлежит конкретному AI-агенту; +- UI объясняет назначение ключа без ссылок на старую workspace-key модель; +- `mcp-server` принимает Community machine access только по agent key; +- Community flow не требует short-lived token service. + +Verification: +- backend unit and integration tests for key lifecycle; +- MCP tool-call tests through agent endpoint; +- e2e smoke for agent key create/reveal/revoke. + +## Next + +### `feat/edition-capability-model` Status: ready -DoD: -- AI agent can receive its own long-lived agent key from the admin UI -- operation has an explicit security level: `standard`, `elevated`, or `strict` -- machine calls to MCP can use short-lived agent tokens for elevated operations -- one-time token mode is defined for strict operations -- transition path from workspace-wide keys is explicit in docs and implementation slices +Goal: +- ввести capability model по редакциям и перестать смешивать Community и premium surface. -## Next +Main code areas: +- `crates/crank-core/src/*` +- `apps/admin-api/src/service.rs` +- `apps/admin-api/src/app.rs` +- `apps/ui/js/*` +- `apps/ui/html/*` +- `docs/product-editions.md` +- `docs/frontend-roadmap.md` + +Implementation slices: +1. Ввести серверную модель capabilities: + - protocols + - auth modes + - security levels + - workspace/user/agent limits +2. Отдавать capability flags в `admin-api`. +3. Привести UI к capability gating вместо скрытых продуктовых допущений. +4. Зафиксировать Community defaults в docs и runtime config. + +DoD: +- UI знает, какие функции входят в текущую редакцию; +- Community не показывает рабочие controls для premium-only features; +- capability checks существуют не только в UI, но и на сервере. + +Verification: +- API tests for capability payloads; +- frontend smoke for hidden/locked states; +- manual verification on Community build. + +### `feat/private-token-service-seam` + +Status: ready + +Goal: +- подготовить public codebase к private реализации короткоживущих и одноразовых токенов. + +Main code areas: +- `crates/crank-core/src/auth.rs` +- `crates/crank-core/src/agent.rs` +- `apps/admin-api/src/app.rs` +- `apps/admin-api/src/service.rs` +- `apps/mcp-server/src/app.rs` +- `docs/agent-auth-model.md` +- `docs/admin-api.md` +- `docs/mcp-interface.md` + +Implementation slices: +1. Определить stable contracts для: + - `POST /mcp-auth/v1/token` + - `POST /mcp-auth/v1/token/one-time` +2. Ввести abstraction для token verification в `mcp-server`. +3. Подготовить capability-gated UI/API surface без private implementation в Community. +4. Отделить Community `standard` flow от commercial `elevated/strict`. + +DoD: +- public contracts зафиксированы и тестируемы; +- Community продолжает работать без private token service; +- premium auth paths не размазываются по Community logic. + +Verification: +- contract tests for auth payload shapes; +- unit tests for credential-kind enforcement; +- docs sync check. + +### `feat/community-surface-trim` + +Status: ready + +Goal: +- привести открытую редакцию к честному product surface и убрать расхождение между текущим кодом и edition policy. + +Main code areas: +- `apps/ui/js/wizard.js` +- `apps/ui/js/wizard-model.js` +- `apps/ui/js/agents.js` +- `apps/ui/js/settings.js` +- `apps/ui/js/i18n.js` +- `docs/product-editions.md` +- `docs/architecture.md` +- `docs/mcp-interface.md` + +Implementation slices: +1. Скрыть или честно заблокировать: + - `WebSocket` + - `SOAP` + - `gRPC streaming` + - streaming execution modes + - `elevated/strict` security levels + в Community. +2. Привести copy и feature affordances к open-core границе. +3. Добавить edition-aware explanation в wizard и agents UI. + +DoD: +- Community UX не создает ложного ожидания наличия premium functionality; +- protocol set и security levels совпадают в docs, UI и runtime contracts. + +Verification: +- e2e checks for wizard protocol list; +- manual Community smoke; +- localized copy review. + +## Planned + +### `feat/frontend-commercial-polish` + +Status: ready + +Goal: +- закрыть оставшиеся UI/UX дефекты, мешающие коммерческой демонстрации и продажной версии продукта. + +Main code areas: +- `apps/ui/js/api-keys.js` +- `apps/ui/js/agents.js` +- `apps/ui/js/secrets.js` +- `apps/ui/js/usage.js` +- `apps/ui/js/settings.js` +- `apps/ui/js/workspace-setup.js` +- `apps/ui/js/auth.js` +- `apps/ui/css/*` +- `docs/frontend-roadmap.md` + +Implementation slices: +1. Перевести mobile tables в card layouts для: + - `API Keys` + - `Secrets` + - `Usage` +2. Доделать `Agents` page и modal: + - объяснение отдельных agent endpoints + - empty states + - slug hints +3. Добить `Settings` and workspace flows: + - terminology cleanup + - navigation spacing + - notifications honesty + - mobile sticky bug +4. Довести agent-key UX на `API Keys` page. + +DoD: +- mobile UI не требует обязательного горизонтального скролла для primary actions; +- terminology и product copy согласованы; +- коммерческая демонстрация проходит без UI contradictions. + +Verification: +- targeted Playwright coverage; +- manual mobile checks at 375px width; +- EN/RU copy audit. + +### `feat/open-core-repo-boundary` + +Status: ready + +Goal: +- закрепить техническую границу между public Community repo и private commercial delivery. + +Main code areas: +- `docs/commercial-boundaries.md` +- `docs/module-decomposition.md` +- `docs/development-rules.md` +- `docs/implementation-plan.md` +- release scripts / Docker manifests / future packaging files + +Implementation slices: +1. Определить extension seams в public code. +2. Подготовить отдельные delivery manifests для Community. +3. Убрать из public repo assumptions о размещении private code рядом с Community logic. +4. Подготовить naming и packaging strategy для private repositories. + +DoD: +- можно объяснить, что именно публикуется как OSS, а что уходит в private delivery; +- Community release path отделен от commercial release path; +- документация не ссылается на удаленные review files. + +Verification: +- docs consistency pass; +- release checklist review. + +### `feat/enterprise-access-governance` + +Status: ready + +Goal: +- реализовать enterprise access layer вне Community scope. + +Main code areas: +- private `enterprise` services and crates +- public contracts in: + - `docs/admin-api.md` + - `docs/agent-auth-model.md` + - `docs/product-editions.md` + +Implementation slices: +1. `SSO` +2. `2FA` +3. extended `RBAC` +4. `audit log` +5. enterprise admin flows in UI through capability gating + +DoD: +- governance features не живут как полумеры в Community; +- enterprise access model согласован с public contracts. + +### `feat/cloud-metering-control-plane` + +Status: ready + +Goal: +- подготовить hosted-редакцию как отдельный продуктовый контур. + +Main code areas: +- private cloud services +- usage/metering integration points +- `docs/product-editions.md` +- `docs/commercial-boundaries.md` + +Implementation slices: +1. usage metering by workspace / agent / token; +2. billing integration; +3. hosted tenant controls; +4. cloud operational tooling. + +DoD: +- Cloud edition не зависит от ручного учета usage; +- hosted product surface согласован с feature matrix. + +### `feat/release-protection-distribution` + +Status: ready + +Goal: +- подготовить безопасную поставку Community и коммерческих редакций. + +Main code areas: +- CI workflows +- Dockerfiles +- deployment manifests +- release docs + +Implementation slices: +1. public GitHub release flow for Community; +2. private registry flow for Enterprise; +3. signed artifacts and provenance; +4. release checklist for edition-specific packaging. + +DoD: +- Community публикуется из public repo; +- Enterprise и Cloud используют private delivery path; +- коммерческий код не требуется публиковать в open source ради поставки. ### `feat/live-authenticated-staging-entry` Status: ready +Goal: +- держать реальный staging/demo контур в состоянии, пригодном для демонстрации и регрессии. + +Main code areas: +- `docs/demo-runbook.md` +- `docs/deploy-and-staging-smoke.md` +- `docs/authenticated-staging-pass.md` +- deployment workflows + +Implementation slices: +1. актуализировать demo user flow; +2. прогнать authenticated staging pass; +3. зафиксировать regressions и manual checks; +4. синхронизировать deploy docs с реальным окружением. + DoD: -- authenticated staging flow runs against the real deployed environment -- operator docs match the current deploy/runtime model -- staging validation is captured without relying on local fixtures - -## Completed - -- `feat/websocket-test-run-polish` -- `feat/frontend-wizard-modularization` -- `feat/distributed-mcp-session-store` +- staging validation не зависит от локальных фикстур; +- documentation matches deployed behavior; +- demo flow reproducible on real environment. diff --git a/__REFACTORING.md b/__REFACTORING.md deleted file mode 100644 index 66c7739..0000000 --- a/__REFACTORING.md +++ /dev/null @@ -1,1333 +0,0 @@ -# Refactoring Plan - -## Purpose - -This document defines the refactoring track required to improve maintainability, safety, and production-readiness in the current codebase. It is intentionally implementation-oriented: each area is split into concrete slices, target modules, function ownership, migration order, and acceptance criteria. - -The refactoring scope covers: - -1. splitting `crates/crank-registry/src/postgres.rs` into domain modules; -2. migrating static SQL to `sqlx::query!` / `sqlx::query_as!` where possible; -3. replacing SHA-256 key derivation in `SecretCrypto` with HKDF; -4. replacing string timestamps in domain models with typed timestamps; -5. adding `Display` support to `define_id!`; -6. introducing explicit PostgreSQL pool configuration; -7. removing duplicated SQL timestamp formatting; -8. making runtime and API error payloads more structured; -9. introducing correlation IDs across transport, runtime, and adapters; -10. adding rate limiting and backpressure controls; -11. replacing in-memory MCP session state with a multi-instance-safe store. - -## Guiding Rules - -- Keep domain logic separate from SQL storage concerns. -- Preserve current behavior while shrinking file size and coupling. -- Move in vertical slices: storage module split first, then SQL typing, then timestamp migration. -- Every slice must be independently reviewable and mergeable. -- Prefer compatibility shims during migration instead of breaking multiple crates at once. -- Add or update tests in the same slice as the refactoring. - -## Refactoring Tracks - -### Track A: Registry Modularization - -#### Problem - -`crates/crank-registry/src/postgres.rs` currently concentrates almost all database logic: - -- bootstrap and auth; -- workspaces and memberships; -- invitations; -- operations and versions; -- agents and bindings; -- samples and descriptors; -- API keys; -- secrets; -- stream sessions and async jobs; -- invocation logs and usage; -- YAML import jobs. - -This creates: - -- a permanent git conflict hotspot; -- slow code navigation; -- oversized review diffs; -- accidental coupling between unrelated domains. - -#### Target Layout - -`crates/crank-registry/src/` - -```text -error.rs -lib.rs -migrations.rs -model.rs -postgres/ - mod.rs - connect.rs - auth.rs - workspace.rs - invitation.rs - secret.rs - stream.rs - api_key.rs - operation.rs - agent.rs - observability.rs - yaml_import.rs - shared.rs -``` - -#### Ownership by Module - -##### `postgres/mod.rs` - -Responsibilities: - -- define `PostgresRegistry`; -- wire submodules; -- expose the public impl surface; -- keep only high-level constructor entry points. - -Functions staying here: - -- `new` -- `connect` -- `connect_with_options` -- `migrate` - -Functions moving out: - -- everything domain-specific. - -##### `postgres/connect.rs` - -Responsibilities: - -- connection setup; -- pool initialization; -- migration invocation; -- pool option translation from config to `PgPoolOptions`. - -Functions: - -- `connect` -- `connect_with_options` -- `migrate` -- internal helpers: - - `build_pool_options` - - `build_connect_options` - -##### `postgres/auth.rs` - -Responsibilities: - -- auth users; -- user sessions; -- membership access checks; -- auth profiles. - -Functions: - -- `upsert_bootstrap_user` -- `get_auth_user_by_email` -- `get_auth_user_by_id` -- `update_user_profile` -- `update_user_password` -- `create_user_session` -- `get_user_session` -- `touch_user_session` -- `revoke_user_session` -- `set_user_session_current_workspace` -- `user_has_workspace_access` -- `save_auth_profile` -- `get_auth_profile` -- `list_auth_profiles` -- `list_auth_profiles_referencing_secret` - -Internal helpers to extract: - -- `map_auth_user_row` -- `map_session_row` -- `map_auth_profile_row` - -##### `postgres/workspace.rs` - -Responsibilities: - -- workspaces; -- memberships; -- workspace CRUD. - -Functions: - -- `list_workspaces` -- `list_workspaces_for_user` -- `ensure_membership` -- `list_memberships` -- `update_membership_role` -- `delete_membership` -- `get_workspace` -- `create_workspace` -- `update_workspace` -- `delete_workspace` - -Internal helpers: - -- `workspace_exists` -- `map_workspace_row` -- `map_membership_row` - -##### `postgres/invitation.rs` - -Responsibilities: - -- invitations lifecycle. - -Functions: - -- `list_invitations` -- `create_invitation` -- `delete_invitation` - -##### `postgres/secret.rs` - -Responsibilities: - -- secrets; -- secret versions; -- secret touch/rotation. - -Functions: - -- `list_secrets` -- `get_secret` -- `get_current_secret_version` -- `create_secret` -- `rotate_secret` -- `delete_secret` -- `touch_secret` - -##### `postgres/stream.rs` - -Responsibilities: - -- stream sessions; -- async jobs; -- state transitions; -- cleanup. - -Functions: - -- `create_stream_session` -- `get_stream_session` -- `update_stream_session_state` -- `close_stream_session` -- `list_stream_sessions` -- `delete_expired_stream_sessions` -- `create_async_job` -- `get_async_job` -- `update_async_job_status` -- `cancel_async_job` -- `list_async_jobs` -- `delete_expired_async_jobs` - -Internal helpers: - -- `assert_stream_session_transition` -- `assert_async_job_transition` -- `map_stream_session_row` -- `map_async_job_row` - -##### `postgres/api_key.rs` - -Responsibilities: - -- platform API key lifecycle. - -Functions: - -- `list_platform_api_keys` -- `get_platform_api_key_by_secret_for_workspace_slug` -- `create_platform_api_key` -- `revoke_platform_api_key` -- `delete_platform_api_key` -- `touch_platform_api_key` - -##### `postgres/operation.rs` - -Responsibilities: - -- operations; -- versions; -- published operations; -- samples; -- descriptors. - -Functions: - -- `create_operation` -- `create_version` -- `update_operation_draft` -- `archive_operation` -- `delete_operation` -- `has_published_agent_bindings_for_operation` -- `list_operations` -- `list_operation_usage_summaries` -- `list_operation_agent_refs` -- `get_operation_summary` -- `get_operation_version` -- `list_operation_versions` -- `publish_operation` -- `get_published_operation` -- `list_published_operations` -- `save_sample_metadata` -- `list_sample_metadata` -- `save_descriptor_metadata` -- `list_descriptor_metadata` -- `insert_version_row` - -Internal helpers: - -- `map_operation_summary_row` -- `map_operation_version_row` -- `map_published_operation_row` -- `serialize_optional_json` - -##### `postgres/agent.rs` - -Responsibilities: - -- agents; -- agent versions; -- bindings; -- publish lifecycle. - -Functions: - -- `list_agents` -- `get_agent_summary` -- `create_agent` -- `get_agent_version` -- `save_agent_bindings` -- `update_agent_summary` -- `delete_agent` -- `publish_agent` -- `unpublish_agent` -- `archive_agent` -- `get_published_agent_tools_by_slug` -- `list_agent_bindings` -- `insert_agent_version_row` -- `replace_agent_bindings_rows` - -##### `postgres/observability.rs` - -Responsibilities: - -- invocation logs; -- usage rollups and breakdowns. - -Functions: - -- `create_invocation_log` -- `list_invocation_logs` -- `get_invocation_log` -- `summarize_usage` -- `list_usage_timeline` -- `list_usage_by_operation` -- `get_usage_for_operation` -- `list_usage_by_agent` -- `get_usage_for_agent` - -##### `postgres/yaml_import.rs` - -Responsibilities: - -- YAML import jobs. - -Functions: - -- `create_yaml_import_job` -- `finish_yaml_import_job` -- `get_yaml_import_job` - -##### `postgres/shared.rs` - -Responsibilities: - -- SQL snippets reused by several modules; -- common timestamp select fragments during the migration period; -- small row-mapping helpers; -- pagination helpers. - -Planned content: - -- `Page` helpers if they remain storage-local; -- constants for table-independent fragments; -- transitional helpers until typed timestamps remove string formatting entirely. - -#### Slice Plan for Track A - -##### Slice A1: File layout and constructors - -- create `postgres/` directory; -- move `connect`, `connect_with_options`, `migrate`, and struct wiring; -- keep all public signatures unchanged. - -##### Slice A2: Workspace/Auth/API-key split - -- move `auth.rs`, `workspace.rs`, `invitation.rs`, `api_key.rs`; -- leave behavior unchanged; -- add targeted module tests if not present. - -##### Slice A3: Operation/Agent split - -- move `operation.rs`, `agent.rs`, `yaml_import.rs`; -- keep SQL unchanged for this slice; -- ensure published-operation and published-agent behavior is unchanged. - -##### Slice A4: Secret/Stream/Observability split - -- move `secret.rs`, `stream.rs`, `observability.rs`; -- keep transition logic and cleanup behavior unchanged. - -##### Slice A5: Remove legacy monolith - -- remove remaining methods from root `postgres.rs`; -- rename module structure so `postgres/mod.rs` is the only entry point. - -#### Done Criteria for Track A - -- no `postgres.rs` monolith remains; -- no single registry source file exceeds roughly 800–1200 lines; -- all existing registry tests pass unchanged; -- `lib.rs` still re-exports `PostgresRegistry` without downstream breakage. - -## Track B: Compile-Time SQL Verification - -### Problem - -Most static SQL currently uses `sqlx::query(...)` / `query_as::<_, T>(...)` with runtime validation only. That leaves: - -- column name typos; -- mismatched types; -- bad nullability assumptions; -- schema drift - -to runtime instead of compile time. - -### Goal - -Use `sqlx::query!` and `sqlx::query_as!` for every static query with stable shape. - -Keep runtime builders only where the query shape is truly dynamic. - -### Query Classification - -#### Must migrate to `query!` / `query_as!` - -Use macros for: - -- single-row fetches by ID; -- list queries with static columns; -- insert/update/delete with static statement shape; -- existence checks; -- count queries; -- cleanup queries with fixed predicates. - -Expected modules: - -- `auth.rs` -- `workspace.rs` -- `invitation.rs` -- `secret.rs` -- `stream.rs` -- `api_key.rs` -- most of `agent.rs` -- most of `yaml_import.rs` - -#### May stay runtime-based - -Keep dynamic APIs only when necessary: - -- optional filters that materially change SQL shape; -- optional sort columns chosen from a whitelist; -- bulk dynamic `IN` lists where macro usage becomes unreadable; -- `QueryBuilder`-based multi-row inserts/updates. - -Expected modules: - -- `operation.rs` list queries with optional filters; -- `observability.rs` usage aggregations and breakdowns; -- some search/filter endpoints. - -### Required Infrastructure - -#### Add SQLx offline support - -Repository-level requirements: - -- use `sqlx prepare` to generate checked metadata; -- keep checked metadata in version control if the team wants deterministic CI; -- wire verification into CI or `just verify`. - -Concrete additions: - -- `sqlx-data.json` or the current SQLx metadata equivalent for the chosen version; -- `DATABASE_URL` or test DB wiring for `cargo sqlx prepare`. - -#### Introduce row structs where needed - -For `query_as!`, add storage-local row structs for cases where domain structs should not be coupled directly to SQL column shape. - -Examples: - -- `WorkspaceRow` -- `AuthUserRow` -- `OperationSummaryRow` -- `InvocationLogRow` -- `UsageTimelineRow` - -Rule: - -- domain structs remain domain-oriented; -- row structs remain SQL-oriented. - -### Function-Level Migration Rules - -#### For `query_as!` - -Use `query_as!` when: - -- the query result maps cleanly to a typed row struct; -- all returned columns are explicit; -- nullability is known. - -#### For `query!` - -Use `query!` when: - -- no reusable row struct is needed; -- the query is a simple `insert`, `update`, `delete`, or `exists` fetch. - -#### For dynamic list queries - -Use: - -- `sqlx::QueryBuilder` with explicit allowed sort/filter sets; -- storage-local mapper functions from row to domain object. - -### Migration Order - -##### Slice B1: `auth`, `workspace`, `invitation` - -Low-risk, fixed-shape queries. - -##### Slice B2: `secret`, `stream`, `api_key`, `yaml_import` - -Still mostly fixed-shape. - -##### Slice B3: `operation`, `agent` - -Introduce mixed strategy: macros for fixed queries, builder for dynamic lists. - -##### Slice B4: `observability` - -Likely the most dynamic area. Convert only static pieces first, leave complex aggregations to `QueryBuilder`. - -### Done Criteria for Track B - -- all fixed-shape SQL uses compile-time checked macros; -- dynamic SQL is explicitly documented and isolated; -- CI fails on SQL drift before runtime tests; -- no new static query is accepted through `sqlx::query(...)` without justification. - -## Track C: SecretCrypto KDF Correction - -### Problem - -Current code in `crates/crank-runtime/src/secret_crypto.rs` derives the AES key by hashing the master key with SHA-256: - -- no KDF semantics; -- no labeled derivation context; -- no future rotation/versioning path. - -### Goal - -Replace direct hashing with HKDF-SHA256. - -### Target Design - -`SecretCrypto::new(master_key: &str)` will: - -1. validate non-empty master key; -2. derive a 32-byte AES key using `Hkdf`; -3. use explicit `info` context, for example: - - `b"crank.secret-envelope.v1"` -4. initialize `Aes256Gcm` from the derived bytes. - -### Module Changes - -File: - -- `crates/crank-runtime/src/secret_crypto.rs` - -Changes: - -- replace `Sha256::digest(trimmed.as_bytes())` with HKDF expansion; -- preserve the public constructor signature; -- keep ciphertext format unchanged if possible; -- if ciphertext format must change later, introduce envelope `version` before doing so. - -### Required Tests - -- `encrypt_decrypt_roundtrip_still_works` -- `empty_master_key_is_rejected` -- `same_master_key_derives_stable_key` -- `different_master_keys_produce_different_ciphertexts` - -### Done Criteria for Track C - -- no direct SHA-256 KDF remains; -- `SecretCrypto` uses HKDF-SHA256 with explicit context; -- all existing secret encrypt/decrypt flows remain compatible. - -## Track D: Typed Timestamps - -### Problem - -Domain and storage structs use `String` timestamps. SQL converts timestamps via repeated `to_char(...)`. - -That causes: - -- repeated formatting logic in SQL; -- weak type guarantees; -- inability to do time arithmetic without reparsing; -- broad surface for formatting drift. - -### Goal - -Use `time::OffsetDateTime` in domain and storage models, with RFC 3339 serialization at the transport boundary. - -### Target Type Strategy - -#### Domain layer - -Use: - -```rust -#[serde(with = "time::serde::rfc3339")] -pub created_at: OffsetDateTime -``` - -and the same for `updated_at`, `expires_at`, `used_at`, and similar fields. - -#### Storage row layer - -Also use `OffsetDateTime`. - -Do not format timestamps in SQL except for transitional compatibility shims. - -#### Transport layer - -JSON still emits RFC 3339 strings through Serde. - -### Files Affected - -#### Domain crates - -- `crates/crank-core/src/workspace.rs` -- `crates/crank-core/src/access.rs` -- `crates/crank-core/src/auth.rs` -- `crates/crank-core/src/agent.rs` -- `crates/crank-core/src/operation.rs` -- `crates/crank-core/src/observability.rs` -- `crates/crank-core/src/secret.rs` -- `crates/crank-core/src/stream_session.rs` - -#### Registry models - -- `crates/crank-registry/src/model.rs` - -#### Admin API transport/service DTOs - -- `apps/admin-api/src/service.rs` -- any route DTO files that mirror time fields directly - -### Migration Strategy - -##### Slice D1: storage row types - -- change storage row structs to `OffsetDateTime`; -- remove `to_char(...)` from fixed-shape SQL; -- keep service DTOs as strings temporarily if needed. - -##### Slice D2: domain models - -- convert `crank-core` models to typed timestamps with Serde helpers; -- update constructors and tests. - -##### Slice D3: API/service transport - -- remove string timestamp shims from service DTOs where possible; -- ensure JSON remains stable. - -##### Slice D4: cleanup - -- remove transitional formatting helpers; -- remove `now_string()` helpers in favor of typed timestamps. - -### Acceptance Criteria - -- no `created_at: String` or `updated_at: String` remains in domain/storage models; -- no `to_char(created_at...)` / `to_char(updated_at...)` remains in registry SQL except perhaps in legacy compatibility code slated for removal in the same slice; -- JSON contract stays RFC 3339. - -## Track E: `Display` for ID Newtypes - -### Problem - -`define_id!` currently requires `.as_str()` everywhere for formatting and messages. - -### Goal - -Add `std::fmt::Display` support to every generated ID newtype. - -### File - -- `crates/crank-core/src/ids.rs` - -### Change - -Add inside `define_id!`: - -```rust -impl std::fmt::Display for $name { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!(f, "{}", self.0) - } -} -``` - -### Follow-Up Cleanup - -After macro support lands: - -- replace formatting sites like `workspace_id.as_str()` inside messages; -- keep `.as_str()` only where an actual `&str` API requires it. - -### Acceptance Criteria - -- all ID newtypes implement `Display`; -- formatting code no longer uses `.as_str()` inside `format!`, `println!`, or error messages unless `&str` is actually required. - -## Track F: PostgreSQL Pool Configuration - -### Problem - -The pool is currently created via default `PgPoolOptions`, which means the runtime behavior is implicit and under-configured. - -### Goal - -Make pool behavior explicit and configurable from environment. - -### Target Config Model - -Add a typed config struct, for example: - -```rust -pub struct PostgresPoolConfig { - pub max_connections: u32, - pub min_connections: u32, - pub acquire_timeout_ms: u64, - pub idle_timeout_ms: u64, - pub max_lifetime_ms: u64, -} -``` - -### Runtime Environment - -Add env variables: - -- `POSTGRES_MAX_CONNECTIONS` -- `POSTGRES_MIN_CONNECTIONS` -- `POSTGRES_ACQUIRE_TIMEOUT_MS` -- `POSTGRES_IDLE_TIMEOUT_MS` -- `POSTGRES_MAX_LIFETIME_MS` - -### Files - -- `crates/crank-registry/src/postgres/connect.rs` -- `apps/admin-api/src/main.rs` -- `apps/mcp-server/src/main.rs` -- config docs and examples - -### Behavior - -`connect_with_options` should accept either: - -- direct `PgConnectOptions` plus explicit pool config; -- or an internal default config built from env. - -Defaults should be conservative and explicit, for example: - -- `max_connections = 20` -- `min_connections = 2` -- `acquire_timeout = 5s` -- `idle_timeout = 10m` -- `max_lifetime = 30m` - -Exact values can be tuned later, but they must no longer be hidden defaults. - -### Acceptance Criteria - -- pool limits and timeouts are explicit; -- docs reflect the new variables; -- startup logs or config inspection make active pool settings visible. - -## Track G: Timestamp Formatting Deduplication - -### Problem - -The same `to_char(... at time zone 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS"Z"')` fragment is copied across many queries. - -### Goal - -Eliminate duplicated formatting logic. - -### Strategy - -This track is mostly resolved by Track D, but there is a transition window. - -During transition: - -- centralize any remaining formatting in `postgres/shared.rs`; -- define named SQL fragments only once; -- remove inline duplicated snippets. - -After Track D: - -- remove formatting fragments entirely. - -### Acceptance Criteria - -- duplicated inline timestamp formatting is no longer scattered across query strings; -- final state has typed timestamps instead of SQL formatting. - -## Track H: Error Structure Normalization - -### Problem - -Some runtime and API error variants carry only a `details: String`, which makes programmatic handling and consistent observability harder. - -### Goal - -Introduce structured error payloads where the current model is too string-heavy. - -### Scope - -Priority types: - -- `RuntimeError` -- registry storage errors where context is stringly typed; -- API error mapping around secret/runtime/streaming faults. - -### Design Rules - -- keep `thiserror`; -- use structured fields in variants whenever stable context exists; -- reserve free-form `details` only for opaque third-party messages. - -Examples: - -Instead of: - -```rust -RuntimeError::SecretCrypto { details: String } -``` - -prefer: - -```rust -RuntimeError::SecretCrypto { - operation: &'static str, - details: String, -} -``` - -or dedicated variants such as: - -- `InvalidSecretEnvelope` -- `SecretDecryptFailed` -- `SecretSerializeFailed` - -### Slice Plan - -##### Slice H1 - -- normalize `SecretCrypto` and streaming/session/job errors. - -##### Slice H2 - -- normalize adapter errors for REST/gRPC/WebSocket/SOAP where stable categories exist. - -##### Slice H3 - -- normalize API mapping and error codes. - -### Acceptance Criteria - -- core runtime error variants expose structured context; -- API can map key failure classes deterministically; -- logs and telemetry can group errors by category instead of by ad hoc message text. - -## Track I: Correlation IDs and Cross-Layer Tracing - -### Problem - -Today, a request can enter through: - -- `admin-api` HTTP routes; -- `mcp-server` JSON-RPC over Streamable HTTP; -- internal runtime execution; -- protocol adapters (`REST`, `gRPC`, `WebSocket`, `SOAP`) - -without a single stable correlation key that survives the entire path. - -There are already partial pieces in the codebase: - -- `request_id` exists in observability models and storage; -- MCP mappings can inject `$.mcp.correlation_id`; -- some protocol fixtures mention `correlation_id`. - -But there is no guaranteed end-to-end propagation model. When an adapter fails, logs cannot always be tied back to: - -- the incoming HTTP request; -- the MCP session; -- the runtime operation execution; -- the invocation log row. - -### Goal - -Introduce a first-class correlation model that is generated once at ingress and propagated across: - -- transport; -- service/runtime orchestration; -- adapter calls; -- persistence and logs. - -### Target Model - -Introduce two related identifiers: - -- `request_id` - - unique per incoming HTTP or MCP request; - - used for ingress/egress tracing and log correlation. -- `correlation_id` - - stable business-level identifier propagated into upstream payloads when configured; - - may equal `request_id` by default, but remains distinct conceptually. - -### Flow Rules - -#### `admin-api` - -At HTTP ingress: - -- read `X-Request-Id` if supplied and valid; -- otherwise generate a UUID v7 request ID; -- store it in request extensions; -- include it in structured logs; -- pass it into runtime execution context; -- persist it into `invocation_logs.request_id` when a test run or runtime execution creates a log row. - -#### `mcp-server` - -At JSON-RPC ingress: - -- derive a request-scoped `request_id`; -- bind it to the current transport session and JSON-RPC request; -- propagate it into runtime execution and adapter context; -- return it in logs and diagnostics. - -#### Runtime - -Add an execution context struct, for example: - -```rust -pub struct ExecutionContext { - pub request_id: String, - pub correlation_id: String, - pub workspace_id: WorkspaceId, - pub agent_id: Option, - pub operation_id: OperationId, - pub mcp_session_id: Option, -} -``` - -This context must be passed through: - -- unary execution; -- window execution; -- session tool start/poll/stop; -- async job start/status/result/cancel. - -#### Adapters - -Each adapter must receive execution context and apply it consistently: - -- `REST` - - add `X-Request-Id` - - add `X-Correlation-Id` - - allow explicit mapping into headers/query/body where configured -- `gRPC` - - attach metadata keys such as `x-request-id`, `x-correlation-id` -- `WebSocket` - - inject correlation metadata into subscribe payloads when configured -- `SOAP` - - inject correlation value into configured header/body mappings - -### Files and Functions - -#### New or expanded files - -- `apps/admin-api/src/request_context.rs` -- `apps/mcp-server/src/request_context.rs` -- `crates/crank-runtime/src/context.rs` -- adapter request builders in each protocol crate - -#### Functions to add or refactor - -`admin-api` - -- `extract_or_generate_request_id(...)` -- `request_context_middleware(...)` -- `runtime_request_context(...)` - -`mcp-server` - -- `extract_mcp_request_context(...)` -- `runtime_request_context_from_message(...)` - -`crank-runtime` - -- `ExecutionContext` -- `PreparedRequest` / `RuntimeOperation` updates to carry context -- `execute_*` functions accept `&ExecutionContext` - -### Acceptance Criteria - -- every runtime invocation has a request ID; -- every adapter call can log the request ID; -- invocation logs persist the same request ID; -- operator can trace a failing adapter call back to the original HTTP or MCP request. - -## Track J: Rate Limiting and Backpressure - -### Problem - -There is currently no explicit protection layer for: - -- excessive API request rates; -- excessive tool invocation rates; -- excessive stream/session polling; -- unbounded adapter output pressure. - -This is a production-readiness gap, especially now that the platform supports: - -- unary execution; -- windowed streaming; -- session tools; -- async jobs; -- WebSocket and SSE upstream adapters. - -### Goal - -Introduce explicit controls for request rate, concurrency, and bounded output pressure. - -### Scope - -#### API-level controls - -Apply to: - -- `admin-api` routes; -- `mcp-server` transport endpoints. - -Dimensions: - -- per-IP -- per-session -- per-workspace -- per-platform-key - -#### Runtime-level controls - -Apply to: - -- concurrent unary executions; -- concurrent window executions; -- concurrent live stream sessions; -- concurrent async jobs; -- polling frequency for session/job tools. - -#### Adapter-level backpressure - -Apply to: - -- REST SSE -- gRPC server-streaming -- WebSocket windows/sessions - -Controls: - -- max buffered items -- max buffered bytes -- max window duration -- max session polls -- max per-message bytes - -### Target Design - -#### `admin-api` - -Add middleware with a keyed limiter, for example: - -- by authenticated user ID; -- by workspace ID; -- by remote IP for unauthenticated endpoints. - -#### `mcp-server` - -Add a limiter keyed by: - -- platform key ID; -- workspace slug; -- MCP session ID; -- optionally agent slug. - -#### `crank-runtime` - -Add a runtime guard layer, for example: - -```rust -pub struct RuntimeLimits { - pub max_concurrent_unary: usize, - pub max_concurrent_window: usize, - pub max_concurrent_sessions: usize, - pub max_concurrent_jobs: usize, - pub max_polls_per_session: u32, -} -``` - -And an execution gate: - -- acquire permit before execution; -- release permit on completion; -- reject with stable error code if limits are exceeded. - -### Implementation Slices - -##### Slice J1: API request throttling - -- add middleware and config for `admin-api`; -- add middleware and config for `mcp-server`. - -##### Slice J2: Runtime concurrency limits - -- add semaphores/permits around execution paths. - -##### Slice J3: Streaming/session pressure limits - -- cap poll frequency and active session count; -- cap pending buffer sizes in adapters. - -### Acceptance Criteria - -- platform can reject overload deterministically instead of degrading unpredictably; -- stream/session polling is bounded; -- slow clients cannot force unbounded buffering in streaming adapters. - -## Track K: Distributed MCP Session Store - -### Problem - -`apps/mcp-server/src/session.rs` currently keeps transport/session state in process memory through `Arc>>`. - -This is acceptable only for a single instance. It breaks when: - -- multiple `mcp-server` instances run behind a load balancer; -- a container restarts and loses in-memory session state; -- GET/POST/DELETE transport requests land on different instances. - -### Goal - -Replace the in-memory session store with a multi-instance-safe shared store. - -### Target Design - -Introduce a `SessionStore` trait: - -```rust -#[async_trait] -pub trait SessionStore { - async fn create_transport_session(&self, session: TransportSessionRecord) -> Result<...>; - async fn get_transport_session(&self, session_id: &str) -> Result<...>; - async fn touch_transport_session(&self, session_id: &str, now: OffsetDateTime) -> Result<...>; - async fn delete_transport_session(&self, session_id: &str) -> Result<...>; - async fn append_pending_message(&self, session_id: &str, message: Value) -> Result<...>; - async fn drain_pending_messages(&self, session_id: &str) -> Result, ...>; - async fn cleanup_expired_transport_sessions(&self, now: OffsetDateTime) -> Result; -} -``` - -Provide two implementations: - -- `InMemorySessionStore` - - test-only and local dev fallback -- `PostgresSessionStore` - - production default - -### Storage Model - -Add tables such as: - -- `mcp_transport_sessions` -- `mcp_transport_outbox` - -Suggested fields: - -`mcp_transport_sessions` - -- `id` -- `workspace_id` -- `agent_id` -- `platform_api_key_id` -- `transport_kind` -- `status` -- `created_at` -- `updated_at` -- `expires_at` - -`mcp_transport_outbox` - -- `id` -- `session_id` -- `sequence_no` -- `payload_json` -- `created_at` - -### Semantics - -#### POST initialize - -- create transport session record in shared store. - -#### GET SSE stream - -- attach to existing session record; -- stream messages from outbox or polling bridge; -- support reconnect without requiring process-local state. - -#### POST JSON-RPC - -- resolve session from shared store; -- append outbound messages to outbox. - -#### DELETE - -- mark session closed in shared store; -- best-effort cleanup of outbox rows. - -### Files and Modules - -Add: - -- `apps/mcp-server/src/session/mod.rs` -- `apps/mcp-server/src/session/store.rs` -- `apps/mcp-server/src/session/in_memory.rs` -- `apps/mcp-server/src/session/postgres.rs` - -Potential shared data types: - -- `crates/crank-core/src/transport_session.rs` - -### Migration Order - -##### Slice K1: trait and in-memory implementation - -- preserve current behavior but hide it behind a trait. - -##### Slice K2: persistent schema and Postgres store - -- add migrations and registry/storage support. - -##### Slice K3: transport handlers on shared store - -- wire GET/POST/DELETE through the trait-backed store. - -##### Slice K4: cleanup and failover semantics - -- cleanup job for expired transport sessions; -- reconnect tests across store-backed state. - -### Acceptance Criteria - -- transport session state survives process restart if store is persistent; -- GET/POST/DELETE requests can hit different instances safely; -- in-memory session state is no longer the production bottleneck. - -## Delivery Order - -Recommended order: - -1. `feat/refactoring-plan` -2. `feat/id-display-support` -3. `feat/postgres-pool-config` -4. `feat/postgres-registry-modularization` -5. `feat/sqlx-compile-time-verification` -6. `feat/secret-crypto-hkdf` -7. `feat/typed-timestamps` -8. `feat/error-structure-normalization` -9. `feat/correlation-id-propagation` -10. `feat/runtime-rate-limiting-and-backpressure` -11. `feat/distributed-mcp-session-store` -12. `feat/post-refactor-cleanup` - -Reasoning: - -- `Display` is tiny and immediately reduces noise. -- pool config is operational and isolated. -- module split should happen before large query migrations, otherwise the macro migration lands into the monolith and becomes harder to review. -- HKDF is isolated and low-risk. -- typed timestamps should land after storage modules are split and query migration is underway. - -## Per-Slice Verification - -Every slice must run: - -- `just fmt-check` -- `just check` -- `just clippy` -- targeted tests for the touched area - -Additionally: - -- SQL macro migration slices must run SQLx prepare/verification; -- timestamp slices must run serialization regression tests; -- secret crypto slice must run compatibility and roundtrip tests. - -## Success Definition - -The refactoring track is complete when: - -- registry storage code is split into domain modules; -- static SQL is compile-time validated; -- `SecretCrypto` uses HKDF instead of SHA-256 as a KDF; -- domain/storage timestamps are typed; -- ID newtypes implement `Display`; -- PostgreSQL pool settings are explicit and configurable; -- timestamp formatting duplication is removed; -- error variants carry structured context where it matters. - -At that point, maintainability improves materially without changing the product model or feature scope. diff --git a/__REVIEW_FRONT.md b/__REVIEW_FRONT.md deleted file mode 100644 index f762308..0000000 --- a/__REVIEW_FRONT.md +++ /dev/null @@ -1,832 +0,0 @@ -# Frontend Refactoring Plan - -## Purpose - -This document defines the frontend refactoring track for the Alpine.js UI. It is based on the current implementation, the observed code smells, and the explicit review findings. - -The goal is not to replace Alpine.js. The stack choice is valid for the current product size. The goal is to make the UI: - -- safe against obvious XSS classes; -- internally consistent; -- easier to maintain; -- easier to test; -- easier to ship and profile. - -This document is implementation-oriented and split into concrete tracks, slices, file ownership, and acceptance criteria. - -## Scope - -The frontend refactoring scope covers: - -1. XSS hardening and safe DOM rendering; -2. shell/nav/auth unification; -3. wizard modularization; -4. module-pattern unification; -5. removal of inline event handler patterns; -6. lightweight build pipeline introduction; -7. CSS class cleanup instead of imperative inline styles; -8. low-risk performance improvements; -9. frontend observability and testability improvements. - -## Non-Goals - -This plan does not assume: - -- migration to React/Vue/Svelte; -- a full SPA rewrite; -- a design-system rewrite; -- replacing Alpine.js as the core UI model. - -The target is a cleaner Alpine-compatible codebase, not a framework migration. - -## Current State Summary - -### Valid architectural choices - -- Alpine.js is an appropriate choice for this admin UI size. -- `api.js` is already a strong shared transport layer. -- `ui-feedback.js` is simple and correct. -- the visual language and token system are already above average for an admin panel. -- EN/RU i18n support is already broad and worth preserving. - -### Primary issues - -1. unsafe `innerHTML` and string-built DOM with API data; -2. mixed UI architecture: - - Alpine on some pages; - - imperative DOM on others; -3. `wizard.js` is a monolith; -4. duplicated shell logic between `auth.js` and `nav.js`; -5. inconsistent module patterns; -6. inline `onclick="..."` string injection in some renderers; -7. no bundling/build pipeline; -8. direct `element.style` mutation where CSS classes should be used; -9. avoidable repeated filtering/computation in Alpine render paths. - -## Guiding Rules - -- Keep Alpine.js. -- Centralize common frontend primitives instead of re-implementing them page by page. -- Prefer DOM node construction and `textContent` over string-built HTML. -- Restrict `innerHTML` to trusted, static, translation-reviewed content only. -- Keep shared shell state in one place. -- Break the wizard by concern, not by arbitrary line count alone. -- Introduce the lightest build pipeline that solves the real problems. - -## Track A: XSS Hardening - -### Problem - -Several files still render data from API responses through `innerHTML` and string concatenation without guaranteed escaping. - -Known examples from review: - -- `apps/ui/js/secrets.js` -- `apps/ui/js/workspace.js` -- error rendering paths where backend-provided strings are interpolated into HTML - -This is a real frontend safety issue even in an authenticated admin panel. - -### Goal - -No untrusted API data is inserted through raw `innerHTML`. - -### Target Design - -Introduce a small shared DOM/safety utility module: - -```text -apps/ui/js/utils/ - dom.js - escape.js -``` - -### `utils/escape.js` - -Responsibilities: - -- `escapeHtml(value)` -- `escapeAttr(value)` if attribute-safe escaping is still needed during transition - -Rule: - -- this file becomes the only source of escape helpers; -- page scripts stop defining local copies such as the current `escapeHtml()` inside `wizard.js`. - -### `utils/dom.js` - -Responsibilities: - -- helper builders for repetitive patterns: - - `textNode(value)` - - `el(tag, className?, text?)` - - `clear(node)` - - `appendChildren(node, children)` -- helper renderers for empty/error banners when they currently rely on string HTML - -### Required Migrations - -#### Slice A1: shared escape helpers - -Files: - -- `apps/ui/js/wizard.js` -- `apps/ui/js/secrets.js` -- `apps/ui/js/workspace.js` -- any other file using local escape helpers or raw string interpolation into HTML - -Actions: - -- move escape helpers into shared `utils/escape.js`; -- remove duplicate local definitions. - -#### Slice A2: high-risk string renderers - -Priority files: - -- `apps/ui/js/secrets.js` -- `apps/ui/js/workspace.js` -- `apps/ui/js/api-keys.js` if any dynamic HTML remains -- `apps/ui/js/logs.js` / `usage.js` review for dynamic interpolation patterns - -Actions: - -- replace `innerHTML` row rendering with node creation; -- use `textContent` for names, labels, and API-provided error strings; -- use explicit element creation for buttons and actions. - -#### Slice A3: error rendering contract - -Actions: - -- never assign backend error text to `innerHTML`; -- render API errors through: - - `textContent` - - safe toast body - - pre-defined empty/error state templates - -Explicit case to fix: - -- `apps/ui/js/secrets.js` - - current pattern builds `profilesList.innerHTML` using `state.error` - - `state.error` is populated from caught API/fetch error text - - that makes server-provided message text an unsafe HTML source - - this renderer must be rewritten to build DOM nodes and place the message through `textContent` - -### Acceptance Criteria - -- no API-provided field is rendered via raw `innerHTML`; -- `escapeHtml()` exists once, in a shared utility file; -- XSS-sensitive renderers use `textContent` or safe DOM builders. - -## Track B: Shell/Auth/Nav Unification - -### Problem - -`auth.js` and `nav.js` both: - -- read `localStorage.getItem('crank_user')`; -- derive role/workspace labels; -- update `.nav-avatar`, `.user-dropdown-name`, `.user-dropdown-role`. - -This creates duplicate state and drift. - -### Goal - -One module owns shell identity state and shell rendering. - -### Target Design - -Introduce: - -```text -apps/ui/js/shell/ - identity.js - nav.js - routes.js -``` - -### Ownership - -#### `shell/identity.js` - -Responsibilities: - -- session cache; -- local mirror persistence; -- deriving initials; -- deriving role label; -- deriving current workspace label; -- rendering shell identity DOM; -- broadcasting session change events. - -Functions to absorb from current files: - -- from `auth.js`: - - `roleLabel` - - `currentWorkspaceLabel` - - `clearUserMirror` - - `primaryMembership` - - `initials` - - `persistUserMirror` - - `mirroredUser` - - `renderShellIdentity` - - `replaceSession` -- from `nav.js`: - - `currentUser` - - `fillUserInfo` - -#### `shell/nav.js` - -Responsibilities: - -- dropdown open/close behavior; -- mobile nav behavior; -- binding click handlers for: - - logout - - profile - - settings - -It must not own identity derivation anymore. - -#### `auth.js` - -Responsibilities after refactor: - -- login/logout/session fetch; -- route guarding; -- delegating shell identity updates to `shell/identity.js`. - -### Slice Plan - -##### Slice B1: move identity rendering - -- create `shell/identity.js`; -- move identity ownership there; -- update `auth.js` and `nav.js` to consume it. - -##### Slice B2: remove duplicate storage constants - -- `STORAGE_KEY` exists once; -- `roleLabel()` exists once. - -##### Slice B3: event-driven shell refresh - -- shell reacts to `crank:sessionchange`; -- individual page scripts stop updating shell identity directly. - -### Acceptance Criteria - -- only one module owns mirrored user state; -- `auth.js` and `nav.js` no longer duplicate shell rendering logic; -- shell identity remains correct after login/logout/workspace switch. - -## Track C: Wizard Modularization - -### Problem - -`apps/ui/js/wizard.js` is a monolith of roughly 3k lines, holding: - -- global state; -- step navigation; -- 5 protocol flows; -- file uploads; -- mapping; -- test-run handling; -- import/export; -- UI state transitions. - -This is the frontend equivalent of the `postgres.rs` monolith. - -### Goal - -Split wizard code by stable responsibility while keeping the current HTML-driven workflow. - -### Target Layout - -```text -apps/ui/js/wizard/ - index.js - state.js - navigation.js - protocol-capabilities.js - upstream.js - auth-selector.js - mapping.js - test-run.js - protocol-rest.js - protocol-graphql.js - protocol-grpc.js - protocol-websocket.js - protocol-soap.js - shared.js -``` - -### Ownership by Module - -#### `wizard/state.js` - -Responsibilities: - -- encapsulate all mutable wizard state; -- replace top-level globals with a single state object and accessors. - -State currently to move: - -- `currentStep` -- `wizardProtocol` -- `wizardMode` -- `wizardEditId` -- `wizardWorkspaceId` -- `wizardCurrentOperation` -- `wizardCurrentVersion` -- upload buffers -- protocol capability cache -- test-run preview state - -Target API: - -- `createWizardState()` -- `getWizardState()` -- `resetWizardState()` - -#### `wizard/navigation.js` - -Responsibilities: - -- dynamic step loading; -- `goToStep` -- `loadStep` -- `step3PanelId` -- sidebar/progress rendering; -- continue/back button states. - -Important cleanup: - -- replace inline style mutation for disabled buttons with CSS classes. - -#### `wizard/protocol-capabilities.js` - -Responsibilities: - -- defaults and fetched capability model; -- `defaultProtocolCapabilities()` -- `currentProtocolCapabilities()` -- `loadProtocolCapabilities()` - -#### `wizard/upstream.js` - -Responsibilities: - -- upstream dropdown rendering; -- upstream auth badge rendering; -- upstream create/edit interactions; -- upstream selection state. - -#### `wizard/auth-selector.js` - -Responsibilities: - -- secret/auth profile selection; -- quick-create secret/profile flows; -- auth form validation and serialization. - -#### `wizard/mapping.js` - -Responsibilities: - -- input/output mapping; -- preview helpers; -- field inference helpers; -- path defaulting. - -#### `wizard/test-run.js` - -Responsibilities: - -- test-run request build; -- test-run submission; -- mode-aware result rendering for: - - unary - - window - - session - - async job - -#### `wizard/protocol-*.js` - -Responsibilities: - -- only protocol-specific behavior. - -Examples: - -`protocol-graphql.js` - -- selected GraphQL type; -- query/mutation defaults; -- operation name extraction; -- top-level field extraction. - -`protocol-grpc.js` - -- descriptor upload; -- service/method rendering; -- reflection-related flow. - -`protocol-websocket.js` - -- subscribe/unsubscribe templates; -- heartbeat/reconnect form state. - -`protocol-soap.js` - -- WSDL/XSD upload; -- service catalog inspection; -- binding apply. - -### Migration Order - -##### Slice C1: state + navigation split - -- extract state and navigation first; -- preserve current behavior. - -##### Slice C2: protocol capability + shared helpers - -- move generic helper logic out of main file. - -##### Slice C3: GraphQL/gRPC/REST protocol extraction - -- highest-traffic protocol flows first. - -##### Slice C4: WebSocket/SOAP protocol extraction - -- move less common but newer protocol flows. - -##### Slice C5: auth selector + test-run split - -- finalize with secret/auth and result handling modules. - -##### Slice C6: thin entrypoint - -- `wizard/index.js` becomes the only page bootstrap file. - -### Acceptance Criteria - -- no frontend file comparable in maintenance risk to current `wizard.js`; -- wizard state is centralized, not scattered across globals; -- protocol-specific logic is isolated by file. - -## Track D: Module Pattern Unification - -### Problem - -Current JS modules use mixed patterns: - -- IIFE modules with encapsulation; -- top-level globals in page scripts; -- direct `window` leakage. - -### Goal - -Move all first-party UI code to one consistent browser-friendly module pattern. - -### Recommended Target - -Use build-generated ES module bundles, while keeping runtime API globals explicit where needed. - -During transition: - -- keep page entrypoints small; -- expose only intentionally public page bootstrap functions on `window`; -- no free-floating utility functions in global scope. - -### Rules - -- utility modules must never write to `window` unless they are explicitly public; -- page entrypoints may attach one namespace if needed: - - `window.CrankWizard` - - `window.CrankShell` -- generic helpers must stay module-local. - -### Acceptance Criteria - -- no stray utility functions like `escapeHtml()` living in global scope; -- global exposure is explicit and minimal. - -## Track E: Event Handling Cleanup - -### Problem - -Some renderers still inject handlers through HTML strings such as: - -- `onclick="switchWorkspace('...')"` - -This is brittle and unsafe if values contain quotes. - -### Goal - -All dynamic UI actions use bound event listeners, not injected inline JS. - -### Required Migrations - -Priority files: - -- `apps/ui/js/workspace.js` -- any renderer that currently injects action handlers via HTML strings - -Approach: - -- build nodes; -- attach `addEventListener`; -- carry identifiers through closures or `data-*` attributes. - -### Acceptance Criteria - -- no dynamic `onclick="..."` string generation remains in first-party code. - -## Track F: Build Pipeline - -### Problem - -The current UI ships many discrete JS files directly, without bundling or minification. Vendor files are committed in-repo and loaded directly. - -This is acceptable for early stages but no longer ideal for: - -- cache invalidation; -- dependency graph hygiene; -- payload count; -- module splitting; -- future lint/test integration. - -### Goal - -Introduce a lightweight build pipeline without turning the UI into a heavyweight SPA toolchain. - -### Recommended Tooling - -Use `esbuild`. - -Reasoning: - -- minimal configuration; -- fast; -- good enough for bundling this codebase; -- easy Docker integration. - -### Target Layout - -```text -apps/ui/ - src/ - js/ - ... - public/ - ... - dist/ -``` - -This does not require a full source tree rewrite immediately. A transitional layout is acceptable: - -- keep current HTML layout; -- bundle JS entrypoints into `dist/js/*.js`; -- keep CSS mostly as-is initially. - -### Required Outputs - -- one bundle per page entrypoint where needed: - - `catalog` - - `agents` - - `api-keys` - - `logs` - - `usage` - - `settings` - - `workspace-setup` - - `wizard` - - `secrets` -- shared chunk for common utilities - -### Docker Changes - -UI Docker build should: - -1. install npm deps; -2. run build; -3. serve bundled assets via nginx. - -### Vendor Policy - -Move from committed vendor runtime files toward npm-managed dependencies where possible: - -- `alpinejs` -- `js-yaml` - -If some vendored asset remains, it must be intentional and documented. - -### Acceptance Criteria - -- first-party JS is bundled and minified; -- cache invalidation is deterministic; -- page HTML references built bundles, not a long list of raw scripts. - -## Track G: CSS State Cleanup - -### Problem - -Some UI states are expressed by directly mutating inline styles in JS. - -Example class of issue: - -- button disabled visuals set by `element.style.opacity`, `style.cursor`, etc. - -### Goal - -Move stateful visuals into CSS classes. - -### Target Approach - -Introduce semantic classes: - -- `.is-disabled` -- `.is-hidden` -- `.is-loading` -- `.is-active` -- `.is-selected` - -JS only toggles classes and attributes. - -### Priority File - -- `apps/ui/js/wizard.js` - -### Acceptance Criteria - -- no UI state requires imperative style strings when a reusable class can express it. - -## Track H: Performance Polish - -### Problem - -Some Alpine getters recompute filters repeatedly. - -Known example: - -- operations tab counts in `catalog.js` recompute filtered lists multiple times per render. - -### Goal - -Reduce avoidable repeated work without introducing premature complexity. - -### Strategy - -#### `catalog.js` - -Replace repeated tab filtering with a cached derived snapshot, for example: - -- one getter that computes filtered operations once; -- one derived count map per tab; -- `tabCount(tabId)` reads from that cached map. - -#### General Rule - -Only optimize: - -- repeated O(n) list filters inside render loops; -- repeated expensive formatting inside x-for; -- repeated DOM queries in hot paths. - -Do not over-engineer with memoization everywhere. - -### Acceptance Criteria - -- obvious repeated list filtering in Alpine render paths is removed; -- rendered behavior remains identical. - -## Track I: Template and i18n Safety Cleanup - -### Problem - -There are small but risky patterns around naming collisions and template ergonomics. - -Known example: - -- `x-for="t in tabDefs"` where `t` also exists globally as the translation function. - -### Goal - -Avoid template variable names that collide with global utility names. - -### Required Rule - -Within Alpine templates: - -- do not use `t`, `tf`, or other globally meaningful helper names as loop variables. - -Use: - -- `tab` -- `item` -- `entry` -- `profile` -- `workspace` - -instead. - -### Immediate Fixes - -- rename `x-for="t in tabDefs"` to `x-for="tab in tabDefs"` in operations UI; -- review similar short variable names across templates. - -### Acceptance Criteria - -- no Alpine template variable shadows translation helpers or globally meaningful names. - -## Track J: Frontend Observability and Testability - -### Problem - -As the UI grows, debugging and regression confidence depend on: - -- shared state boundaries; -- consistent render helpers; -- stable e2e selectors; -- predictable logs and request tracing. - -### Goal - -Make the UI easier to verify and debug. - -### Work Items - -#### Shared test selectors - -Add stable `data-testid` or equivalent hooks in critical areas: - -- wizard protocol cards -- secret create/rotate flows -- auth selector -- shell identity - -#### Console discipline - -- no stray debug logging in production code; -- user-facing failures flow through toasts or error states only. - -#### Request ID surfacing - -When backend tracing lands, expose request IDs in developer-facing error surfaces where appropriate: - -- test-run result panel; -- logs detail drawer; -- adapter failure toasts where safe. - -### Acceptance Criteria - -- critical UI flows have stable automation hooks; -- frontend failures are easier to trace to backend request IDs once that backend work lands. - -## Delivery Order - -Recommended order: - -1. `feat/frontend-refactoring-plan` -2. `feat/frontend-xss-hardening` -3. `feat/frontend-shell-unification` -4. `feat/frontend-template-safety-cleanup` -5. `feat/frontend-wizard-modularization` -6. `feat/frontend-build-pipeline` -7. `feat/frontend-css-state-cleanup` -8. `feat/frontend-performance-polish` -9. `feat/frontend-observability-and-testability` - -## Per-Slice Verification - -Each slice must run the smallest realistic verification set for the affected area. - -Baseline: - -- `node --check` for touched JS files -- UI Docker build -- relevant Playwright specs where available - -Additional rules: - -- XSS hardening slices must include targeted regression checks for renderer behavior; -- shell unification slices must include login/logout/workspace switch smoke; -- wizard modularization slices must preserve existing wizard e2e coverage; -- build pipeline slices must validate production asset serving in Docker. - -## Success Definition - -The frontend refactoring track is complete when: - -- untrusted API data is no longer rendered through unsafe HTML paths; -- shell identity logic has one owner; -- the wizard no longer lives in a single giant file; -- page scripts follow a consistent module pattern; -- inline event handler injection is removed; -- the UI ships through a lightweight build pipeline; -- common UI states are driven by CSS classes instead of imperative style strings; -- obvious repeated render-time computations are cleaned up. - -At that point, the frontend remains Alpine-based but becomes significantly safer and easier to evolve. diff --git a/__REVIEW_UX.md b/__REVIEW_UX.md deleted file mode 100644 index ea71c7a..0000000 --- a/__REVIEW_UX.md +++ /dev/null @@ -1,736 +0,0 @@ -# UX and Copy Remediation Plan - -## Purpose - -This document defines the UX, copywriting, and localization cleanup track for the current UI. It is separate from `__REVIEW_FRONT.md`: - -- `__REVIEW_FRONT.md` focuses on frontend architecture, safety, module structure, and delivery mechanics; -- `__REVIEW_UX.md` focuses on user-facing clarity, consistency, language quality, and product honesty. - -The goal is to remove developer-facing language, broken localization patterns, misleading placeholders, and half-functional UI signals from the product surface. - -## Core Principle - -The UI must speak to operators, not to developers. - -That means: - -- no implementation jargon such as "slice", "wired", "resolved at runtime" unless the user truly needs it; -- no debug-style booleans in user-visible success messages; -- no fake functionality presented as nearly available unless the user can act on it; -- no mixed EN/RU phrasing where the product can speak normally in one language. - -## Scope - -This plan covers: - -1. English copy cleanup; -2. Russian copy cleanup; -3. pluralization support; -4. login UX cleanup; -5. settings honesty cleanup; -6. command palette decision and follow-through; -7. wizard message polish; -8. terminology consistency rules. - -## Track A: English Copy Quality - -### Problem Types - -#### A1. Literal formatting markers leaking into UI - -Examples: - -- `apikeys.callout.body` -- `wizard.step2.auth_profile_hint` - -Current issue: - -- backticks are rendered literally as characters; -- they do not become semantic formatting in HTML; -- the user sees developer-style markdown notation. - -### Decision - -Do not use markdown-style backticks in translation strings rendered through normal text nodes. - -Allowed alternatives: - -- plain text without emphasis; -- explicit `` only when rendered through a trusted HTML path that is intended for markup; -- a dedicated visual class if code-like styling is actually needed. - -### Required fixes - -- `apikeys.callout.body` - - replace `` `Last used` `` with plain text or a proper code-like span in a trusted template -- `wizard.step2.auth_profile_hint` - - remove `` `execution_config.auth_profile_ref` `` from user-facing copy - -### Preferred rewrites - -Instead of: - -- `Last used updates after successful MCP authentication.` - -Use: - -- `Last used updates after a successful MCP call.` - -Instead of: - -- `Selected profile will be resolved at runtime and attached through execution_config.auth_profile_ref.` - -Use: - -- `The selected auth profile is applied automatically when this tool runs.` - -#### A2. Internal planning jargon visible to users - -Examples: - -- `secrets.profiles.empty_body` - -Current issue: - -- `next slice` is a delivery-planning term, not a product term. - -### Decision - -All planning jargon must be removed from UI strings. - -Forbidden user-facing phrases: - -- `next slice` -- `this build does not pretend` -- `wired yet` -- `not wired` -- `backed by` -- `resolved at runtime` unless the user truly configures runtime behavior - -### Preferred rewrite - -Instead of: - -- `Create auth profiles in the next slice to bind secrets to upstream authentication.` - -Use: - -- `Create auth profiles in the Wizard when configuring an operation.` - -#### A3. Unsafe or awkward symbols in plain text - -Examples: - -- `agents.drawer.recommendation` - -Current issue: - -- `<15` appears directly in text and should not rely on plain text rendering safety. - -### Decision - -Prefer natural language over symbolic shorthand in user guidance. - -Preferred rewrite: - -- `LLMs usually work best when an agent has fewer than 15 tools.` - -#### A4. Debug-style status messages - -Examples: - -- `wizard.test.window_completed_body` -- `wizard.test.session_started_body` - -Current issue: - -- booleans like `true/false` are shown directly to users; -- milliseconds are exposed when approximate guidance is enough. - -### Decision - -User-visible status text must describe outcome, not internal flags. - -### Required rewrites - -`wizard.test.window_completed_body` - -Current: - -- `Window complete: {window_complete}. Truncated: {truncated}. Has more: {has_more}.` - -Target behavior: - -- message should describe: - - how many items were collected; - - whether data was truncated; - - whether more data is available. - -Preferred shape: - -- `Collected {count} items.` -- `Collected {count} items. Results were truncated.` -- `Collected {count} items. More data is available if you extend the window or continue polling.` - -`wizard.test.session_started_body` - -Current: - -- `Poll after {poll_after_ms} ms from the stream sessions view.` - -Preferred: - -- `Session started. Check Stream sessions in a few seconds.` - -#### A5. Awkward English phrasing - -Examples: - -- `logs.range.1h: Last 1 hour` - -Preferred: - -- `Last hour` - -### Acceptance Criteria - -- no markdown backticks in plain-text translations; -- no delivery jargon in user-facing EN strings; -- no debug booleans in wizard result messaging; -- obvious awkward phrases are normalized to natural English. - -## Track B: Russian Copy Quality - -### Problem Types - -#### B1. Missing keys - -Known example: - -- `wizard.step2.auth_basic` exists in EN and must exist in RU with product-quality copy. -- `secrets.profiles.subtitle_count` in the RU section is effectively still English and must be treated as a translation-parity defect, not just as weak wording. - -### Decision - -Translation parity is mandatory for both locales. - -Every newly added EN key must either: - -- be translated in RU in the same change; -- or fail verification. - -### Required future safeguard - -Add a translation parity checker to detect: - -- missing RU keys; -- missing EN keys; -- exact duplicate placeholder keys if unintended. - -#### B2. Russian pluralization is currently structurally incomplete - -Known examples: - -- `workspace_setup.member_count` -- `workspace_setup.members.days_ago` -- `secrets.used_by_count` - -Current problem: - -- templates like `{count} участников` fail for `1`; -- templates like `{count} дня назад` fail for `5`. - -### Decision - -Introduce plural helpers based on `Intl.PluralRules`. - -### Target API - -In `i18n.js`, add a helper such as: - -```js -function tPlural(count, forms) { - // locale-aware choice -} -``` - -Where: - -- English uses `one / other` -- Russian uses `one / few / many` - -### Target translation style - -Instead of one flat string: - -- `workspace_setup.member_count: '{count} участников'` - -Use grouped forms, for example: - -- `workspace_setup.member_count.one` -- `workspace_setup.member_count.few` -- `workspace_setup.member_count.many` - -Or an equivalent plural storage format, as long as the API is stable. - -### First keys to migrate - -- `workspace_setup.member_count` -- `workspace_setup.members.days_ago` -- `secrets.used_by_count` -- `secrets.profiles.subtitle_count` -- `wizard.step3.grpc.streaming_hidden` -- `wizard.grpc.services_found` - -#### B3. Developer calques and transliterated jargon - -Examples: - -- `резолвиться` -- `в следующем срезе` -- `notification settings` -- `lifecycle` -- `usage references` -- `advanced future integrations` - -### Decision - -Russian copy should be plain product Russian, while keeping real technical terms intact. - -Keep in English only true technical terms such as: - -- `gRPC` -- `WSDL` -- `JSON` -- `Bearer` -- `Query` -- `Mutation` -- `WebSocket` - -Rewrite non-terms into Russian. - -### Required rewrites - -`wizard.step2.auth_profile_hint` - -Instead of: - -- `Выбранный профиль будет резолвиться в runtime...` - -Use: - -- `Выбранный профиль применяется автоматически при каждом вызове инструмента.` - -`secrets.callout.body` - -Remove mixed language and rewrite as: - -- `После создания или ротации API возвращает только метаданные, поэтому на этой странице доступны управление секретами и просмотр их использования.` - -`secrets.hint.generic` - -Avoid: - -- `advanced future integrations` - -Use: - -- `Подходит для произвольного JSON, если секрет нужно использовать в нестандартной интеграции.` - -`workspace_setup.create.footer` - -Replace: - -- `Owner` - -With: - -- `Владелец` - -`secrets.profiles.subtitle_count` - -Current RU text is still English-like and must be translated fully. - -### Acceptance Criteria - -- no missing translation keys between EN/RU; -- Russian plural forms are grammatically correct; -- no developer calques remain in user-facing RU strings; -- role names are translated consistently. - -## Track C: Login Experience Cleanup - -### Problem - -The login screen currently highlights unavailable functionality: - -- Forgot password -- Google SSO -- Request access - -all with `Planned` badges. - -This creates the impression of unfinished product state on the very first screen. - -There is also a developer-style note explaining what is not wired. - -### Decision - -The login page must present only working entry points. - -### Recommended product behavior - -#### Production/default mode - -Show: - -- email -- password -- sign-in button - -Optionally show a single unobtrusive note: - -- `Password reset and SSO will be added later.` - -But only if product really needs that promise visible. - -Preferred option: - -- hide non-working actions completely. - -#### Development-only mode - -If the team wants visibility for future auth work, gate it behind a development flag. - -Example: - -- `window.CrankFeatures.showPlannedAuth` - -### Required changes - -#### Remove or gate: - -- disabled `Forgot password?` -- disabled `Google SSO` -- disabled `Request access` -- `login.password_only` developer note from production HTML - -### Acceptance Criteria - -- login page does not lead with unavailable features; -- no developer-facing implementation note is shown in normal product mode. - -## Track D: Settings Honesty Pass - -### Problem - -The settings page still contains a full `Notifications` section and other planned capability language that reads like UI cargo cult: - -- detailed controls that do nothing; -- developer-style subtitle text; -- explanation written from the implementation perspective. - -### Decision - -If a settings area is not functional, it should not be represented as a near-complete interactive configuration panel. - -### Product rule - -There are only two acceptable states: - -1. fully functional configuration; -2. compact informational placeholder. - -Avoid: - -- large fake forms; -- multiple toggles with `Planned` badges; -- controls that imply persistence when none exists. - -### Required changes - -#### `settings.page.subtitle` - -Current EN subtitle speaks about capabilities being visible but not wired. - -Replace with a user-centered subtitle such as: - -- `Manage your profile, password, and workspace settings.` - -#### Notifications section - -Replace the current detailed pseudo-settings with one compact informational card: - -- title -- short explanation -- possibly one sentence: `Notification routing and per-user preferences will be added later.` - -No fake toggles. - -#### Planned security capabilities - -Keep only if: - -- they are clearly informational; -- they do not look like interactive controls. - -If they look interactive, collapse them into one planned card as well. - -### Acceptance Criteria - -- settings page describes what users can do now; -- non-functional areas are compact and clearly informational; -- no fake detailed settings remain. - -## Track E: Command Palette Decision - -### Problem - -`⌘K` is visible in the navigation, but there is no working command palette attached to it. - -This is worse than not showing it at all. - -### Decision Matrix - -#### Option 1: Remove it - -Choose this if: - -- there is no actual command palette implementation; -- there is no near-term product need for it; -- keeping the hint only creates false expectation. - -#### Option 2: Implement it - -Choose this only if the product has a real command list worth exposing. - -Minimum useful scope: - -- go to Operations -- go to Agents -- go to API Keys -- go to Secrets -- go to Logs -- go to Usage -- go to Settings -- switch workspace -- create new operation -- create new agent - -### Current recommendation - -Based on current code surface, remove it first. - -Reason: - -- there is visible affordance; -- there is no working interaction; -- product already has enough active work without adding a new shell feature mid-stream. - -If later implemented, it should land as its own feature: - -- `feat/frontend-command-palette` - -### Acceptance Criteria - -- either `⌘K` is removed everywhere; -- or a real command palette exists and is keyboard-accessible. - -No middle state. - -## Track F: Wizard Messaging and Step Clarity - -### Problem - -Wizard copy still includes: - -- technical internal language; -- poor plural handling; -- insufficiently clear streaming result messages; -- protocol-step context that changes without enough explicit signaling. - -### Required changes - -#### F1. Auth selector copy - -Remove implementation references such as: - -- `execution_config.auth_profile_ref` - -Replace with operator language: - -- `The selected auth profile is applied automatically when the tool runs.` - -#### F2. Streaming result messages - -Replace technical booleans and milliseconds with outcome-focused language. - -For: - -- `window` -- `session` -- `async_job` - -#### F3. gRPC plural and summary strings - -Fix: - -- `streaming method(s)` -- `1 services` - -Use locale-aware pluralization helpers. - -#### F4. Step-3 protocol clarity - -The sidebar label changing per protocol is useful, but not always obvious enough after protocol switching. - -Recommended UI change: - -- add a small protocol badge near the step title on step 3: - - `REST config` - - `GraphQL operation` - - `gRPC method` - - `WebSocket stream` - - `SOAP binding` - -This should exist both: - -- in the main pane heading; -- optionally in the sidebar if compact enough. - -### Acceptance Criteria - -- wizard status/result text reads like product UX, not runtime telemetry; -- plural forms are correct; -- step 3 protocol-specific context is explicit. - -## Track G: Terminology Rules - -### Problem - -The UI currently mixes three categories without a clear rule: - -1. real technical terms; -2. product concepts; -3. implementation jargon. - -### Terminology Policy - -#### Keep as technical terms - -- `Query` -- `Mutation` -- `gRPC` -- `REST` -- `WebSocket` -- `SOAP` -- `WSDL` -- `JSON` -- `Bearer token` - -#### Translate to product language - -- `workflow` -- `settings` -- `usage` -- `notifications` -- `workspace` -- `owner` - -#### Never expose as user-facing jargon - -- `slice` -- `wired` -- `resolved at runtime` -- `metadata only` if better phrasing exists -- `future integrations` -- `usage references` -- `backed by` - -### Acceptance Criteria - -- technical terms remain technical; -- implementation jargon disappears from UI. - -## Track H: Verification and Guardrails - -### Problem - -Without guardrails, translation and UX regressions will return. - -### Required safeguards - -#### H1. Translation parity check - -Add a script/test that verifies: - -- EN and RU contain the same key set; -- missing keys fail CI. - -#### H2. Dangerous copy lint - -Add a lightweight scan for forbidden phrases in user-facing translations: - -- `next slice` -- `wired` -- `resolved at runtime` -- backticks in plain text translations -- `(s)` plural suffix - -This can be a simple repo script at first. - -#### H3. UX regression checklist - -Extend manual regression docs with: - -- login first impression check; -- no dead nav affordances; -- no fake settings sections; -- pluralization smoke in RU. - -### Acceptance Criteria - -- parity regressions are caught automatically; -- obvious UX copy anti-patterns are detectable before release. - -## Delivery Order - -Recommended order: - -1. `feat/frontend-ux-review-plan` -2. `feat/frontend-i18n-remediation` -3. `feat/frontend-login-simplification` -4. `feat/frontend-settings-honesty-pass` -5. `feat/frontend-command-palette-decision` -6. `feat/frontend-wizard-copy-polish` -7. `feat/frontend-plural-rules` -8. `feat/frontend-ux-guardrails` - -## Per-Slice Verification - -Each slice should include: - -- targeted `node --check` for touched JS -- targeted Playwright smoke if the affected page already has coverage -- manual locale switch smoke for EN and RU - -Additional rules: - -- login cleanup must be visually checked on first load; -- plural-rule slice must include sample assertions for RU forms; -- command-palette decision slice must verify no dead `⌘K` affordance remains. - -## Success Definition - -This UX cleanup track is complete when: - -- the product no longer exposes developer planning language; -- EN and RU feel intentional rather than partially translated; -- plural forms are correct in both locales; -- login shows only meaningful actions; -- settings no longer presents fake detailed controls; -- the command palette affordance is either real or removed; -- wizard status and streaming messages are understandable without internal knowledge. - -At that point, the UI stops feeling like a well-built internal tool and starts reading like a finished product surface. diff --git a/docs/agent-auth-model.md b/docs/agent-auth-model.md index 234340d..e5a3b1f 100644 --- a/docs/agent-auth-model.md +++ b/docs/agent-auth-model.md @@ -143,6 +143,8 @@ AI-агент не должен: - статический ключ AI-агента как режим совместимости; - короткоживущие токены по OAuth 2.0 как рекомендуемый режим. +Реализация этого режима должна рассматриваться как коммерческий контур и не должна требовать размещения private token-issuer logic в public repository. + Соответственно, платформа может обслуживать операции уровня: - `standard`; @@ -156,6 +158,8 @@ AI-агент не должен: - короткоживущий токен; - одноразовый токен. +Enterprise-реализация должна поставляться через private delivery и опираться на capability-gated integrations в public codebase. + Соответственно, допускаются операции уровня: - `standard`; diff --git a/docs/architecture.md b/docs/architecture.md index b5a4234..6aff8c5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -218,6 +218,28 @@ Crank - платформа для публикации внешних API в в - request headers - operation template - variables mapping + +## 8. Open-core граница + +Crank развивается как open-core продукт. + +Это означает: + +- `Community` должна полностью собираться из этого репозитория; +- коммерческие возможности не должны храниться здесь как рабочий private code; +- различия между редакциями должны отражаться в capability model, API, UI и delivery pipeline. + +Ключевые продуктовые различия зафиксированы в `docs/product-editions.md`. + +## 9. Коммерческий контур + +Коммерческий контур не должен защищаться "скрытием" уже опубликованного исходного кода. Правильная архитектурная граница выглядит так: + +- в public repo остается Community runtime и extension seams; +- коммерческие реализации поставляются из private repositories или private artifacts; +- критичные правила лицензирования, metering и token issuance исполняются на серверной стороне. + +Практические правила зафиксированы в `docs/commercial-boundaries.md`. - извлечение результата из `data` GraphQL в MCP публикуется как фиксированная операция с предсказуемой структурой ответа. diff --git a/docs/commercial-boundaries.md b/docs/commercial-boundaries.md new file mode 100644 index 0000000..95bc4a9 --- /dev/null +++ b/docs/commercial-boundaries.md @@ -0,0 +1,196 @@ +# Границы open-core и защита коммерческого кода + +## 1. Назначение документа + +Этот документ определяет: + +- как разделять открытый и коммерческий функционал; +- какие части должны оставаться в публичном репозитории; +- какие части должны выноситься в private delivery; +- как защищать коммерческий код без ложной ставки на обфускацию и "антидекомпиляцию". + +## 2. Базовый принцип + +Коммерческий код нужно защищать не попытками спрятать уже опубликованный исходный код, а правильной границей поставки. + +Принцип: + +- открытый код остается действительно открытым; +- коммерческий код не попадает в public repository; +- коммерческие сервисы и модули поставляются из приватного контура; +- критичные правила лицензирования, metering и security policy исполняются на серверной стороне. + +## 3. Что считается открытым контуром + +В публичном репозитории должны оставаться: + +- доменная модель Community; +- `admin-api`, `mcp-server` и `ui`, необходимые для Community; +- `REST`, `GraphQL` и `gRPC unary` в открытой редакции; +- секреты, auth profiles, agent publishing, logs и usage; +- статический ключ AI-агента; +- контейнерное развертывание Community; +- документация, тесты и демо-сценарии Community. + +## 4. Что считается коммерческим контуром + +В приватный контур должны выноситься: + +- short-lived token service; +- one-time token service; +- `SSO`, `2FA`, расширенная `RBAC`, `audit log`; +- `WebSocket`, `SOAP`, `gRPC streaming`, если они не включаются в Community; +- advanced streaming execution modes; +- metering и billing; +- cloud control plane; +- enterprise licensing and entitlement checks; +- private operational tooling and support tooling. + +## 5. Модель разделения репозиториев + +### 5.1. Public repository + +Этот репозиторий должен содержать только: + +- Community runtime; +- extension points; +- capability model; +- честные product contracts для открытой редакции. + +### 5.2. Private repositories + +Коммерческие возможности должны жить отдельно: + +- либо в приватных crates; +- либо в приватных приложениях и сервисах; +- либо в отдельных private repositories с собственной поставкой. + +Рекомендуемая схема: + +- `crank` — public Community repo; +- `crank-enterprise` — private self-hosted extensions; +- `crank-cloud` — private cloud control plane и hosted-only logic. + +## 6. Техническая стратегия разделения + +### 6.1. Capability-first design + +Открытый код должен опираться на capability model: + +- edition capabilities; +- protocol capabilities; +- auth capabilities; +- security capabilities. + +Это позволяет: + +- скрывать недоступные функции в UI; +- не смешивать Community и commercial code paths; +- добавлять private implementations без форка всей архитектуры. + +### 6.2. Extension seams + +В public repo должны быть только контракты и точки расширения: + +- trait boundaries; +- service contracts; +- edition flags; +- protocol registry abstraction; +- token issuer abstraction; +- feature availability checks. + +Private code должен подключаться как реализация этих контрактов, а не как условные ветки по всему коду Community. + +### 6.3. Server-side enforcement + +Критичные коммерческие ограничения должны исполняться только на сервере: + +- edition capabilities; +- token issuance policy; +- licensing checks; +- metering; +- protocol availability; +- per-operation security rules. + +Фронтенд может только отображать состояние. Он не должен быть единственным местом, где проверяется "можно / нельзя". + +## 7. Что не является реальной защитой + +Нельзя считать надежной защитой: + +- минификацию frontend-кода; +- обфускацию JavaScript; +- "сложность" Rust binary как основную линию защиты; +- попытку скрыть коммерческую логику в публичном репозитории через feature flags, если исходный код уже доступен. + +Все это может немного повысить порог извлечения, но не решает задачу защиты коммерческого IP. + +## 8. Реальная защита коммерческого кода + +### 8.1. Не публиковать исходный код + +Основное правило: + +- коммерческий исходный код не должен попадать в public repo. + +### 8.2. Поставлять private artifacts + +Коммерческий контур должен поставляться как: + +- private container images; +- private binary artifacts; +- private `Helm` charts; +- private configuration bundles. + +### 8.3. Подписывать артефакты + +Для коммерческой поставки должны использоваться: + +- подписанные контейнерные образы; +- проверяемая provenance metadata; +- versioned private releases. + +### 8.4. Хранить ключевую логику на сервере + +Наиболее чувствительные части должны оставаться на серверной стороне: + +- licensing; +- token issuance; +- cloud metering; +- enterprise access policy; +- hosted control plane logic. + +### 8.5. Не включать private UI в OSS bundle + +Если функция коммерческая, ее UI не должен полноценно поставляться в Community build. + +Допустимы: + +- capability-driven hiding; +- ограниченный teaser copy. + +Недопустимы: + +- полностью рабочие commercial screens в open-source bundle; +- наличие private API contracts без server-side gating. + +## 9. Что нужно сделать в кодовой базе + +Для подготовки к коммерческой реализации в open-source коде должны появиться: + +- edition capability model; +- protocol capability model; +- auth capability model; +- интерфейсы для token issuer и enterprise access services; +- server-side policy checks; +- UI gating по capability flags; +- отдельные delivery manifests для Community. + +## 10. Связанные документы + +- `docs/product-editions.md` +- `docs/agent-auth-model.md` +- `docs/module-decomposition.md` +- `docs/frontend-roadmap.md` +- `docs/refactoring-roadmap.md` +- `docs/implementation-plan.md` diff --git a/docs/development-rules.md b/docs/development-rules.md index 2d774bd..3360c30 100644 --- a/docs/development-rules.md +++ b/docs/development-rules.md @@ -174,6 +174,15 @@ - домен не знает про HTTP, SQL, storage и transport; - adapters и repositories реализуют контракты, заданные ближе к домену. +## 7.4. Open-core правило + +Для проекта фиксируется дополнительное правило: + +- код `Community` живет в public repository; +- коммерческий код не должен попадать в public repository "на будущее"; +- в public code допустимы только extension seams, capability flags и контракты для private implementations; +- edition gating должно проверяться на серверной стороне, а не только в UI. + ## 8. Git workflow ## 8.1. Remote diff --git a/docs/frontend-roadmap.md b/docs/frontend-roadmap.md new file mode 100644 index 0000000..eef6442 --- /dev/null +++ b/docs/frontend-roadmap.md @@ -0,0 +1,133 @@ +# Frontend roadmap + +## 1. Назначение документа + +Этот документ фиксирует оставшийся фронтенд-backlog после уже выполненных крупных cleanup и refactoring slices. + +Он заменяет старые review-документы и должен использоваться вместе с `TASKS.md`. + +## 2. Что уже считается закрытым + +Выполненными считаются следующие большие треки: + +- базовая локализация и plural rules; +- XSS hardening основных dynamic render paths; +- template safety cleanup; +- CSS state cleanup; +- frontend build pipeline; +- wizard modularization; +- базовая frontend observability and testability; +- websocket test-run polish; +- command palette removal; +- settings honesty initial pass. + +## 3. Что остается актуальным + +### 3.1. Product gating by edition + +UI должен честно отражать различия редакций: + +- Community не должен показывать доступные к настройке `WebSocket`, `SOAP`, `gRPC streaming`, если они не входят в открытую поставку; +- `security_level = elevated` и `security_level = strict` не должны выглядеть рабочими в Community; +- multi-user и enterprise controls должны скрываться или отображаться как capability-locked. + +Основные файлы: + +- `apps/ui/js/wizard.js` +- `apps/ui/js/operations.js` +- `apps/ui/js/agents.js` +- `apps/ui/js/api-keys.js` +- `apps/ui/js/settings.js` +- `apps/ui/js/i18n.js` + +### 3.2. Agent key UX + +После перехода от platform/workspace keys к agent keys UI должен быть доведен до продуктового состояния: + +- список ключей должен уметь показывать привязку к AI-агенту; +- модалка создания ключа должна позволять выбрать режим и область применения; +- copy должен объяснять, зачем нужен agent-scoped key и чем он отличается от будущих токенных режимов. + +Основные файлы: + +- `apps/ui/js/api-keys.js` +- `apps/ui/js/agents.js` +- `apps/ui/html/api-keys.html` +- `apps/ui/html/agents.html` +- `apps/ui/js/i18n.js` + +### 3.3. Mobile-first restructuring for data-heavy pages + +На мобильных экранах нужно убрать desktop-table anti-pattern для: + +- `API Keys` +- `Secrets` +- `Usage` +- частично `Agents` + +Целевой подход: + +- карточки вместо широких таблиц; +- постоянная видимость primary actions; +- отсутствие скрытого горизонтального скролла как обязательного пути. + +Основные файлы: + +- `apps/ui/js/api-keys.js` +- `apps/ui/js/secrets.js` +- `apps/ui/js/usage.js` +- `apps/ui/js/agents.js` +- `apps/ui/css/*` + +### 3.4. Workspace and settings polish + +Остаются открытые UI/UX-проблемы: + +- глобальная терминология `workspace` против `пространство`; +- структура `Account Settings`; +- честное состояние раздела `Notifications`; +- mobile navigation overlap / sticky bug; +- выравнивание dropdown и workspace switcher behavior. + +Основные файлы: + +- `apps/ui/js/settings.js` +- `apps/ui/js/workspace-setup.js` +- `apps/ui/js/auth.js` +- `apps/ui/index.html` +- `apps/ui/css/*` + +### 3.5. Agent creation and operations wizard clarity + +Нужно добить: + +- copy для причин существования отдельных agent endpoints; +- empty states без багов с пустыми запросами; +- пояснения вокруг `slug`; +- более явное описание security level на операции; +- capability-based hiding недоступных протоколов и execution modes. + +Основные файлы: + +- `apps/ui/js/agents.js` +- `apps/ui/js/wizard.js` +- `apps/ui/js/wizard-live.js` +- `apps/ui/js/wizard-model.js` +- `apps/ui/js/i18n.js` + +## 4. Глобальные правила для UI + +- никакой developer-facing copy в пользовательском интерфейсе; +- capability-locked функция либо скрыта, либо помечена честно; +- мобильная версия не должна требовать обязательного горизонтального скролла для primary actions; +- действия создания должны жить в одном месте и не дублироваться в пустом состоянии без причины; +- все новые user-facing строки должны обновляться одновременно в `EN` и `RU`. + +## 5. Правила приемки + +Frontend-задача считается закрытой только если: + +- измененный UI соответствует capability model редакции; +- строка есть в `EN` и `RU`; +- мобильный сценарий проверен вручную или через e2e; +- не остались старые product contradictions в copy. diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index be8999b..90956a0 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -2,217 +2,168 @@ ## 1. Назначение документа -Этот документ фиксирует порядок перехода от текущего состояния проекта к целевой product-ready integration platform модели. +Этот документ фиксирует актуальный порядок подготовки Crank к коммерческой реализации как open-core продукта. + +Документ не повторяет уже выполненные исторические этапы. Он описывает только тот план, по которому проект должен двигаться дальше. + +## 2. Базовый принцип + +Работа делится на два контура: + +1. завершение и hardening открытой редакции `Community`; +2. подготовка архитектурных, продуктовых и delivery-границ для `Enterprise` и `Cloud`. Принцип: -- сначала перепроектирование `as is -> to be`; -- потом foundation под workspace/agent model; -- потом возврат к end-to-end UI сценариям; -- потом observability и access layer; -- потом polish и demo readiness; -- потом расширение до полного protocol platform scope. +- сначала нужно сделать честную и законченную `Community`-основу; +- затем нужно отделить commercial seams и private delivery; +- только после этого имеет смысл реализовывать коммерческие расширения. -## 2. Этап 1. Перепроектирование `As Is -> To Be` +## 3. Этап 1. Open-core product boundary -Цель: +### Цель -- зафиксировать новую доменную модель и page-driven backend contract. +Зафиксировать, что входит в `Community`, а что уходит в `Enterprise` и `Cloud`. -DoD: +### DoD -- зафиксирован `as is -> to be` план; -- page-by-page gap analysis покрывает все целевые экраны; -- разобраны все архитектурные конфликты UI vs current backend; -- документы `architecture`, `data-model`, `database-schema`, `admin-api`, `mcp-interface` синхронизированы. +- документы `product-editions`, `commercial-boundaries`, `architecture`, `module-decomposition` синхронизированы; +- в кодовой базе определена capability model по редакциям; +- UI и API не обещают Community-функции, которых там не должно быть; +- старые review-файлы не требуются как отдельный источник истины. -## 3. Этап 2. Workspace foundation +## 4. Этап 2. Community machine access completion -Цель: +### Цель -- перевести хранение и API на workspace-scoped модель. +Довести базовую модель машинного доступа Community до полностью рабочего состояния. -DoD: +### DoD -- операции и auth profiles принадлежат workspace; -- registry умеет фильтровать данные по workspace; -- есть default workspace migration path. +- у AI-агента есть собственный ключ; +- UI умеет выпускать, показывать, отзывать и удалять agent keys; +- `mcp-server` использует agent-scoped machine access; +- Community поддерживает только `security_level = standard`; +- в системе не остается product-facing assumptions про workspace-wide machine key как основной способ вызова. -## 4. Этап 3. Agent publishing foundation +## 5. Этап 3. Edition capability model -Цель: +### Цель -- ввести `Agent` и agent-scoped MCP publishing. +Подготовить инфраструктуру, которая позволит одной кодовой базе честно различать редакции продукта. -DoD: +### DoD -- можно создать agent и привязать к нему published operations; -- `mcp-server` выдает tools в контексте конкретного agent; -- один agent видит только свой curated toolset. +- существует server-side capability model: + - protocols + - auth modes + - security levels + - workspace/user limits +- `admin-api` возвращает capability flags для UI; +- UI скрывает или честно блокирует premium functionality; +- Community build не содержит ложных "почти доступных" product paths. -## 5. Этап 4. Operations and wizard integration +## 6. Этап 4. Private auth-service seam -Цель: +### Цель -- посадить operations catalog и wizard на реальные backend contracts. +Подготовить публичный код к private реализации короткоживущих и одноразовых токенов. -DoD: +### DoD -- каталог операций и wizard работают без `localStorage` overrides; -- operation edit/delete/publish/test выполняются через backend; -- все протоколы работают в рамках одного UI flow. +- зафиксированы контракты для: + - `POST /mcp-auth/v1/token` + - `POST /mcp-auth/v1/token/one-time` +- в `mcp-server` существует abstraction для проверки токенов; +- `admin-api` и `mcp-interface` знают про эти контракты документированно; +- Community при этом остается полностью работоспособной без private token service. -## 6. Этап 5. Agents UI and backend +## 7. Этап 5. Commercial protocol split -Цель: +### Цель -- реализовать agent-centric слой. +Перестать считать весь текущий protocol surface частью открытой редакции. -DoD: +### DoD -- agent CRUD работает; -- binding operations к agent работает; -- published agent появляется в MCP runtime. +- определен canonical Community protocol set; +- premium protocol families и premium execution modes вынесены в коммерческий план; +- UI capability-gated по протоколам; +- release model для Community не требует shipping premium flow как рабочей open-source функции. -## 7. Этап 6. Agent-scoped machine access +## 8. Этап 6. Frontend launch readiness -Цель: +### Цель -- реализовать machine access на уровне AI-агента и ввести обязательный уровень защиты операции. +Довести UI до коммерчески пригодного состояния для `Community` и подготовить edition-aware product surface. -DoD: +### DoD -- UI умеет выпускать и отзывать ключи конкретного AI-агента; -- длинноживущий машинный доступ больше не описывается ключом рабочей области; -- у операции появляется обязательный `security_level`; -- machine access не смешивается с upstream auth profiles; -- Community поддерживает базовый режим со статическим ключом AI-агента. +- agent key UX доведен до продукта; +- mobile layouts для data-heavy страниц больше не ломают primary actions; +- terminology и localization согласованы; +- settings и workspace flows не содержат misleading или half-functional sections; +- UI понимает разницу между Community и premium capabilities. -## 8. Этап 7. Token exchange and short-lived auth +## 9. Этап 7. Enterprise access and governance -Цель: +### Цель -- внедрить расширенные режимы доступа для операций повышенной чувствительности. +Реализовать коммерческий access/governance слой в private delivery. -DoD: +### DoD -- существует конечная точка выдачи токена по ключу агента; -- есть модель одноразового токена для чувствительных вызовов; -- токены ограничены по сроку жизни, области действия и числу использований; -- операции уровня `elevated` нельзя вызвать по статическому ключу; -- операции уровня `strict` можно вызвать только по одноразовому токену. +- есть `SSO`; +- есть `2FA`; +- есть расширенная `RBAC`; +- есть `audit log`; +- существует private delivery path для self-hosted customers. -## 9. Этап 8. Observability +## 10. Этап 8. Cloud control plane -Цель: +### Цель -- реализовать логи и usage. +Подготовить hosted-редакцию как отдельный продуктовый контур. -DoD: +### DoD -- `Logs` page и `Usage` page работают на реальных данных; -- есть продуктовые endpoints, а не только application logs; -- rollups и detail views согласованы с UI. +- есть metering; +- есть billing integration; +- есть hosted tenant model; +- есть cloud deployment and support tooling; +- capability model синхронизирована с hosted plans. -## 10. Этап 9. Alpine UI integration +## 11. Этап 9. Release and distribution hardening -Цель: +### Цель -- довести `apps/ui` до полной работы на реальном backend. +Подготовить безопасную поставку открытой и коммерческой редакций. -DoD: +### DoD -- `apps/ui` содержит целевой Alpine.js UI; -- mock JSON больше не используется на критическом пути; -- UI, backend и docs синхронизированы. +- Community публикуется из public GitHub repository; +- Enterprise поставляется через private container registry и private manifests; +- Cloud использует отдельный private operational contour; +- build provenance и подпись артефактов документированы; +- коммерческий код не требуется публиковать в public repository. -## 11. Этап 10. Secret store and upstream auth +## 12. Этап 10. Live staging and demo readiness -Цель: +### Цель -- заменить UI placeholder-модель `${secrets.*}` на рабочий backend/runtime слой secrets. +Поддерживать реальный стенд и демонстрационный сценарий в состоянии, пригодном для продажи и демонстрации. -DoD: +### DoD -- есть workspace-scoped `Secrets` resource; -- secret values хранятся только в зашифрованном виде; -- `AuthProfile` ссылается на `secret_id`, а не на строковый placeholder; -- runtime умеет применять bearer/basic/api-key auth к реальному upstream request; -- wizard имеет auth selector и quick-create flow для secrets/auth profiles. +- live authenticated staging pass воспроизводим; +- deploy smoke и operator docs совпадают с реальной поставкой; +- demo user flow подтвержден на реальном окружении; +- регрессии в publish/call/auth flow ловятся до релиза. -## 12. Этап 11. Hardening and demo readiness +## 13. Связанные документы -Цель: - -- довести продукт до стабильного демо-сценария. - -DoD: - -- end-to-end demo воспроизводим; -- deployment и healthchecks стабильно зелёные; -- документация и продуктовый сценарий совпадают. - -## 13. Этап 12. MCP streaming proxy support - -Цель: - -- довести Crank до controlled streaming model поверх MCP `Streamable HTTP`. - -DoD: - -- `mcp-server` соответствует transport semantics `2025-06-18`; -- execution modes `window`, `session`, `async_job` формально описаны и реализованы; -- REST SSE и gRPC server-streaming поддерживаются в bounded форме; -- UI умеет конфигурировать streaming limits, aggregation и lifecycle; -- e2e сценарии покрывают window/session/job calls. - -## 14. Этап 13. WebSocket upstream support - -Цель: - -- добавить полноценный WebSocket upstream adapter в общую execution model. - -DoD: - -- есть WebSocket target model; -- runtime поддерживает bounded `window`, `session` и `async_job`; -- heartbeat, reconnect и subscription lifecycle конфигурируются явно; -- docs, UI и e2e синхронизированы. - -## 15. Этап 14. SOAP support - -Цель: - -- добавить SOAP как enterprise-oriented protocol family. - -DoD: - -- есть WSDL/XSD-driven target model; -- runtime умеет строить SOAP envelopes и нормализовать SOAP Faults; -- operator может выбрать service, port и operation; -- test-run, publish и observability работают так же, как для остальных протоколов. - -## 16. Этап 15. Detailed streaming specs - -Цель: - -- довести streaming-docs до function-level и field-level design спецификации. - -DoD: - -- есть отдельные docs для streaming admin-api, runtime, UI и capability matrix; -- protocol docs не противоречат общей execution model; -- roadmap и `TASKS.md` синхронизированы с full-detail architecture. - -## 17. Этап 16. Streaming implementation slices - -Цель: - -- перевести streaming design в agent-friendly execution plan по файлам, тестам и DoD. - -DoD: - -- есть отдельный implementation spec по всем streaming slices; -- указаны file-level changes; -- указаны обязательные тесты и acceptance criteria; -- state transitions и sequence outlines зафиксированы явно. +- `docs/product-editions.md` +- `docs/commercial-boundaries.md` +- `docs/frontend-roadmap.md` +- `docs/refactoring-roadmap.md` +- `docs/agent-auth-model.md` diff --git a/docs/mcp-interface.md b/docs/mcp-interface.md index fdbeb93..9eb3bfa 100644 --- a/docs/mcp-interface.md +++ b/docs/mcp-interface.md @@ -110,6 +110,11 @@ - операция задает обязательный `security_level`, и `mcp-server` обязан отклонять вызов, если представленный credential слабее требуемого уровня; - детальная модель уровней зафиксирована в `docs/agent-auth-model.md`. +Важно: + +- Community-сборка должна работать полностью без private token services; +- commercial token flows должны подключаться через capability-gated server-side integrations. + Уровни защиты операции: - `standard` diff --git a/docs/module-decomposition.md b/docs/module-decomposition.md index 6cbecf5..c41a94f 100644 --- a/docs/module-decomposition.md +++ b/docs/module-decomposition.md @@ -159,3 +159,41 @@ Антипаттерн: не превращать `mcp-server` во второй `admin-api`. + +## 5. Разделение open-source и commercial модулей + +### 5.1. Что остается в public repository + +В этом репозитории должны жить: + +- Community domain model; +- Community runtime; +- Community UI; +- extension seams для коммерческих возможностей; +- capability model по редакциям. + +### 5.2. Что должно выноситься в private delivery + +За пределами public repo должны жить: + +- short-lived и one-time token issuers; +- enterprise access services; +- `SSO`, `2FA`, `RBAC`, `audit log`; +- metering и billing; +- cloud control plane; +- premium protocol families, если они не входят в Community edition. + +### 5.3. Техническое правило + +Public code не должен содержать скрытую private business logic. + +Допустимо: + +- trait boundaries; +- capability flags; +- edition-aware service contracts. + +Недопустимо: + +- полноценная коммерческая реализация в open-source исходниках; +- UI, который реально включает premium flow без server-side gating. diff --git a/docs/product-editions.md b/docs/product-editions.md new file mode 100644 index 0000000..649e9d3 --- /dev/null +++ b/docs/product-editions.md @@ -0,0 +1,192 @@ +# Редакции продукта + +## 1. Назначение документа + +Этот документ фиксирует продуктовую модель Crank как open-core платформы и определяет, какие возможности входят в открытую редакцию, а какие относятся к коммерческим редакциям. + +Документ нужен для трех целей: + +- не смешивать в одном репозитории открытый и коммерческий контур без явных границ; +- синхронизировать продуктовые ограничения с архитектурой, API и UI; +- дать реализации четкий ориентир по edition gating и delivery model. + +## 2. Базовая модель + +Crank развивается как три редакции: + +1. `Community` — открытая self-hosted редакция; +2. `Enterprise` — коммерческая self-hosted редакция; +3. `Cloud` — коммерческая vendor-hosted редакция. + +Принцип: + +- открытая редакция должна быть самодостаточной и полезной сама по себе; +- коммерческие редакции должны расширять продукт, а не ломать открытую основу; +- различия между редакциями должны быть видны в capability model, документации, UI и delivery pipeline. + +## 3. Community + +### 3.1. Целевая аудитория + +`Community` предназначена для: + +- индивидуального пользователя; +- AI-энтузиаста; +- небольшой команды, которой нужен один рабочий MCP endpoint без enterprise-функций. + +### 3.2. Что входит + +- self-hosted развертывание; +- одна рабочая область; +- один пользователь; +- один AI-агент; +- протоколы: + - `REST / HTTP` + - `GraphQL` + - `gRPC unary` +- создание и публикация операций; +- журналы вызовов и метрики использования; +- секреты и профили аутентификации для внешних сервисов; +- статический ключ AI-агента; +- только `security_level = standard`; +- контейнерное развертывание; +- документация, демо-сценарий и базовый CI. + +### 3.3. Что не входит + +- несколько рабочих областей; +- несколько пользователей; +- `SSO`, `2FA`, развитая `RBAC`, `audit log`; +- короткоживущие и одноразовые машинные токены; +- `WebSocket`, `SOAP`, `gRPC streaming`; +- режимы `window`, `session`, `async_job`; +- биллинг и usage metering для SaaS; +- cloud control plane. + +## 4. Enterprise + +### 4.1. Целевая аудитория + +`Enterprise` предназначена для компаний, которые: + +- разворачивают Crank на своей инфраструктуре; +- хотят полный набор протоколов; +- требуют усиленную безопасность, командный доступ и аудит; +- готовы платить за сопровождение и коммерческие расширения. + +### 4.2. Что добавляется к Community + +- несколько рабочих областей; +- несколько пользователей; +- `SSO`, `2FA`, расширенная `RBAC`, `audit log`; +- короткоживущие машинные токены; +- одноразовые токены для критичных операций; +- `security_level = elevated` и `security_level = strict`; +- `WebSocket`, `SOAP`, `gRPC streaming`; +- streaming execution modes; +- расширенные административные и эксплуатационные функции; +- поставка через приватные образы и `Helm`-чарты. + +## 5. Cloud + +### 5.1. Целевая аудитория + +`Cloud` предназначена для команд, которым нужна: + +- готовая размещенная платформа; +- минимизация собственной эксплуатационной нагрузки; +- SaaS-модель с управляемыми обновлениями и платными лимитами. + +### 5.2. Что добавляется к Enterprise + +- vendor-hosted развертывание; +- usage metering; +- тарифы и лимиты; +- cloud control plane; +- эксплуатационный мониторинг и обновления со стороны поставщика; +- биллинг и tenant-oriented support tooling. + +## 6. Матрица возможностей + +| Возможность | Community | Enterprise | Cloud | +| --- | --- | --- | --- | +| Hosting | self-hosted | self-hosted | vendor-hosted | +| Workspace count | 1 | many | many | +| User count | 1 | many | many | +| Agent count | 1 | many | many | +| REST / GraphQL / gRPC unary | yes | yes | yes | +| WebSocket / SOAP / gRPC streaming | no | yes | yes | +| Streaming modes | no | yes | yes | +| Static agent key | yes | yes | yes | +| Short-lived MCP token | no | yes | yes | +| One-time MCP token | no | yes | selectively | +| `security_level = standard` | yes | yes | yes | +| `security_level = elevated` | no | yes | yes | +| `security_level = strict` | no | yes | limited by policy | +| Logs and usage | yes | yes | yes | +| SSO / 2FA / RBAC / audit | no | yes | yes | +| Billing / metering | no | no | yes | + +## 7. Правила для реализации + +### 7.1. Community не должен зависеть от private code + +Открытая редакция должна: + +- собираться из этого репозитория полностью; +- иметь завершенный сценарий демо и эксплуатации; +- не требовать закрытых сервисов для базового машинного доступа. + +### 7.2. Коммерческие возможности должны быть явно отделены + +Коммерческие функции не должны появляться в Community как полурабочие заглушки. Для каждой такой функции требуется одно из двух: + +- capability flag с честным недоступным состоянием; +- полное отсутствие функции из open-source сборки. + +### 7.3. Различия по редакциям должны быть зафиксированы в четырех местах + +1. в продуктовой документации; +2. в архитектурной декомпозиции; +3. в административном API и MCP-контрактах; +4. в UI через capability gating и честный copy. + +## 8. Правила для машинной аутентификации + +Продуктовые редакции различаются и по входящему машинному доступу: + +- `Community`: + - только статический ключ AI-агента; + - только операции уровня `standard`; +- `Enterprise`: + - статический ключ AI-агента; + - короткоживущий токен; + - одноразовый токен; +- `Cloud`: + - статический ключ как режим совместимости; + - короткоживущий токен как основной режим; + - одноразовый токен — только там, где это поддерживается продуктовой политикой. + +Подробная модель описана в `docs/agent-auth-model.md`. + +## 9. Правила для UI + +Открытый UI не должен: + +- обещать недоступные в Community возможности как почти готовые; +- показывать enterprise/cloud controls без capability gating; +- использовать ложные CTA для недоступных функций. + +Если функция не входит в текущую редакцию, UI должен делать одно из двух: + +- не показывать ее вовсе; +- показывать ее как явно недоступную и документированно требующую другую редакцию. + +## 10. Связанные документы + +- `docs/architecture.md` +- `docs/module-decomposition.md` +- `docs/agent-auth-model.md` +- `docs/frontend-roadmap.md` +- `docs/refactoring-roadmap.md` +- `docs/implementation-plan.md` diff --git a/docs/refactoring-roadmap.md b/docs/refactoring-roadmap.md new file mode 100644 index 0000000..122723a --- /dev/null +++ b/docs/refactoring-roadmap.md @@ -0,0 +1,111 @@ +# Refactoring roadmap + +## 1. Назначение документа + +Этот документ фиксирует оставшийся технический backlog после уже выполненных больших backend и frontend refactoring tracks. + +Он не повторяет уже завершенные изменения, а описывает то, что еще нужно для product-ready open-core платформы. + +## 2. Что уже считается выполненным + +Закрытыми считаются следующие крупные инженерные направления: + +- модульное разбиение `crank-registry`; +- compile-time SQL verification для статических запросов; +- `HKDF` вместо прямого `SHA-256` derivation; +- typed timestamps; +- `Display` для typed ids; +- explicit PostgreSQL pool configuration; +- structured runtime and registry errors; +- correlation IDs; +- runtime backpressure и request throttling; +- distributed transport session store; +- frontend build pipeline и wizard modularization. + +## 3. Оставшиеся технические треки + +### 3.1. Open-core boundary extraction + +Цель: + +- отделить Community runtime от будущих commercial implementations. + +Что нужно: + +- capability model по редакциям; +- protocol gating; +- auth gating; +- extension seams для private implementations; +- отдельные delivery manifests для Community. + +Ключевые места: + +- `crates/crank-core` +- `apps/admin-api` +- `apps/mcp-server` +- `apps/ui` +- `docs/product-editions.md` +- `docs/commercial-boundaries.md` + +### 3.2. Agent-scoped machine auth completion + +Цель: + +- довести current docs-first auth model до рабочего Community implementation. + +Что нужно: + +- полноценный `AgentKey` lifecycle в `admin-api` и UI; +- отказ от remaining workspace/platform-key assumptions; +- enforcement `security_level = standard` в Community flow; +- capability scaffolding для `elevated` и `strict`. + +### 3.3. Private token-service seam + +Цель: + +- подготовить публичный код к private реализации short-lived и one-time token flows. + +Что нужно: + +- stable HTTP contracts; +- server-side trait boundary; +- token verification abstraction in `mcp-server`; +- capability-aware UI contract. + +### 3.4. Commercial protocol split + +Цель: + +- перестать считать все уже написанные protocol families частью Community surface. + +Что нужно: + +- определить Community-supported protocol set; +- вынести premium protocols и premium execution modes в private delivery plan; +- синхронизировать UI, docs и release process. + +### 3.5. Release and distribution hardening + +Цель: + +- подготовить reproducible public Community release и private commercial delivery. + +Что нужно: + +- public release flow for GitHub; +- private image flow for Enterprise; +- artifact signing / provenance; +- clear packaging split for Community vs commercial. + +## 4. Правила приоритизации + +Сначала выполняются треки, которые формируют границу продукта: + +1. open-core boundary extraction; +2. Community agent-scoped auth completion; +3. private token-service seam; +4. frontend edition gating; +5. release and distribution hardening. + +После этого уже можно безопасно идти в enterprise/cloud-specific implementation.