crank/__REFACTORING.md

Refactoring Plan

Purpose

This document defines the refactoring track required to improve maintainability, safety, and production-readiness in the current codebase. It is intentionally implementation-oriented: each area is split into concrete slices, target modules, function ownership, migration order, and acceptance criteria.

The refactoring scope covers:

1. splitting crates/crank-registry/src/postgres.rs into domain modules;
2. migrating static SQL to sqlx::query! / sqlx::query_as! where possible;
3. replacing SHA-256 key derivation in SecretCrypto with HKDF;
4. replacing string timestamps in domain models with typed timestamps;
5. adding Display support to define_id!;
6. introducing explicit PostgreSQL pool configuration;
7. removing duplicated SQL timestamp formatting;
8. making runtime and API error payloads more structured;
9. introducing correlation IDs across transport, runtime, and adapters;
10. adding rate limiting and backpressure controls;
11. replacing in-memory MCP session state with a multi-instance-safe store.

Guiding Rules

• Keep domain logic separate from SQL storage concerns.
• Preserve current behavior while shrinking file size and coupling.
• Move in vertical slices: storage module split first, then SQL typing, then timestamp migration.
• Every slice must be independently reviewable and mergeable.
• Prefer compatibility shims during migration instead of breaking multiple crates at once.
• Add or update tests in the same slice as the refactoring.

Refactoring Tracks

Track A: Registry Modularization

Problem

crates/crank-registry/src/postgres.rs currently concentrates almost all database logic:

• bootstrap and auth;
• workspaces and memberships;
• invitations;
• operations and versions;
• agents and bindings;
• samples and descriptors;
• API keys;
• secrets;
• stream sessions and async jobs;
• invocation logs and usage;
• YAML import jobs.

This creates:

• a permanent git conflict hotspot;
• slow code navigation;
• oversized review diffs;
• accidental coupling between unrelated domains.

Target Layout

crates/crank-registry/src/

error.rs
lib.rs
migrations.rs
model.rs
postgres/
  mod.rs
  connect.rs
  auth.rs
  workspace.rs
  invitation.rs
  secret.rs
  stream.rs
  api_key.rs
  operation.rs
  agent.rs
  observability.rs
  yaml_import.rs
  shared.rs

Ownership by Module

postgres/mod.rs

Responsibilities:

• define PostgresRegistry;
• wire submodules;
• expose the public impl surface;
• keep only high-level constructor entry points.

Functions staying here:

• new
• connect
• connect_with_options
• migrate

Functions moving out:

• everything domain-specific.

postgres/connect.rs

Responsibilities:

• connection setup;
• pool initialization;
• migration invocation;
• pool option translation from config to PgPoolOptions.

Functions:

• connect
• connect_with_options
• migrate
• internal helpers:
  • build_pool_options
  • build_connect_options

postgres/auth.rs

Responsibilities:

• auth users;
• user sessions;
• membership access checks;
• auth profiles.

Functions:

• upsert_bootstrap_user
• get_auth_user_by_email
• get_auth_user_by_id
• update_user_profile
• update_user_password
• create_user_session
• get_user_session
• touch_user_session
• revoke_user_session
• set_user_session_current_workspace
• user_has_workspace_access
• save_auth_profile
• get_auth_profile
• list_auth_profiles
• list_auth_profiles_referencing_secret

Internal helpers to extract:

• map_auth_user_row
• map_session_row
• map_auth_profile_row

postgres/workspace.rs

Responsibilities:

• workspaces;
• memberships;
• workspace CRUD.

Functions:

• list_workspaces
• list_workspaces_for_user
• ensure_membership
• list_memberships
• update_membership_role
• delete_membership
• get_workspace
• create_workspace
• update_workspace
• delete_workspace

Internal helpers:

• workspace_exists
• map_workspace_row
• map_membership_row

postgres/invitation.rs

Responsibilities:

• invitations lifecycle.

Functions:

• list_invitations
• create_invitation
• delete_invitation

postgres/secret.rs

Responsibilities:

• secrets;
• secret versions;
• secret touch/rotation.

Functions:

• list_secrets
• get_secret
• get_current_secret_version
• create_secret
• rotate_secret
• delete_secret
• touch_secret

postgres/stream.rs

Responsibilities:

• stream sessions;
• async jobs;
• state transitions;
• cleanup.

Functions:

• create_stream_session
• get_stream_session
• update_stream_session_state
• close_stream_session
• list_stream_sessions
• delete_expired_stream_sessions
• create_async_job
• get_async_job
• update_async_job_status
• cancel_async_job
• list_async_jobs
• delete_expired_async_jobs

Internal helpers:

• assert_stream_session_transition
• assert_async_job_transition
• map_stream_session_row
• map_async_job_row

postgres/api_key.rs

Responsibilities:

• platform API key lifecycle.

Functions:

• list_platform_api_keys
• get_platform_api_key_by_secret_for_workspace_slug
• create_platform_api_key
• revoke_platform_api_key
• delete_platform_api_key
• touch_platform_api_key

postgres/operation.rs

Responsibilities:

• operations;
• versions;
• published operations;
• samples;
• descriptors.

Functions:

• create_operation
• create_version
• update_operation_draft
• archive_operation
• delete_operation
• has_published_agent_bindings_for_operation
• list_operations
• list_operation_usage_summaries
• list_operation_agent_refs
• get_operation_summary
• get_operation_version
• list_operation_versions
• publish_operation
• get_published_operation
• list_published_operations
• save_sample_metadata
• list_sample_metadata
• save_descriptor_metadata
• list_descriptor_metadata
• insert_version_row

Internal helpers:

• map_operation_summary_row
• map_operation_version_row
• map_published_operation_row
• serialize_optional_json

postgres/agent.rs

Responsibilities:

• agents;
• agent versions;
• bindings;
• publish lifecycle.

Functions:

• list_agents
• get_agent_summary
• create_agent
• get_agent_version
• save_agent_bindings
• update_agent_summary
• delete_agent
• publish_agent
• unpublish_agent
• archive_agent
• get_published_agent_tools_by_slug
• list_agent_bindings
• insert_agent_version_row
• replace_agent_bindings_rows

postgres/observability.rs

Responsibilities:

• invocation logs;
• usage rollups and breakdowns.

Functions:

• create_invocation_log
• list_invocation_logs
• get_invocation_log
• summarize_usage
• list_usage_timeline
• list_usage_by_operation
• get_usage_for_operation
• list_usage_by_agent
• get_usage_for_agent

postgres/yaml_import.rs

Responsibilities:

• YAML import jobs.

Functions:

• create_yaml_import_job
• finish_yaml_import_job
• get_yaml_import_job

postgres/shared.rs

Responsibilities:

• SQL snippets reused by several modules;
• common timestamp select fragments during the migration period;
• small row-mapping helpers;
• pagination helpers.

Planned content:

• Page<T> helpers if they remain storage-local;
• constants for table-independent fragments;
• transitional helpers until typed timestamps remove string formatting entirely.

Slice Plan for Track A

Slice A1: File layout and constructors

• create postgres/ directory;
• move connect, connect_with_options, migrate, and struct wiring;
• keep all public signatures unchanged.

Slice A2: Workspace/Auth/API-key split

• move auth.rs, workspace.rs, invitation.rs, api_key.rs;
• leave behavior unchanged;
• add targeted module tests if not present.

Slice A3: Operation/Agent split

• move operation.rs, agent.rs, yaml_import.rs;
• keep SQL unchanged for this slice;
• ensure published-operation and published-agent behavior is unchanged.

Slice A4: Secret/Stream/Observability split

• move secret.rs, stream.rs, observability.rs;
• keep transition logic and cleanup behavior unchanged.

Slice A5: Remove legacy monolith

• remove remaining methods from root postgres.rs;
• rename module structure so postgres/mod.rs is the only entry point.

Done Criteria for Track A

• no postgres.rs monolith remains;
• no single registry source file exceeds roughly 800–1200 lines;
• all existing registry tests pass unchanged;
• lib.rs still re-exports PostgresRegistry without downstream breakage.

Track B: Compile-Time SQL Verification

Problem

Most static SQL currently uses sqlx::query(...) / query_as::<_, T>(...) with runtime validation only. That leaves:

• column name typos;
• mismatched types;
• bad nullability assumptions;
• schema drift

to runtime instead of compile time.

Goal

Use sqlx::query! and sqlx::query_as! for every static query with stable shape.

Keep runtime builders only where the query shape is truly dynamic.

Query Classification

Must migrate to query! / query_as!

Use macros for:

• single-row fetches by ID;
• list queries with static columns;
• insert/update/delete with static statement shape;
• existence checks;
• count queries;
• cleanup queries with fixed predicates.

Expected modules:

• auth.rs
• workspace.rs
• invitation.rs
• secret.rs
• stream.rs
• api_key.rs
• most of agent.rs
• most of yaml_import.rs

May stay runtime-based

Keep dynamic APIs only when necessary:

• optional filters that materially change SQL shape;
• optional sort columns chosen from a whitelist;
• bulk dynamic IN lists where macro usage becomes unreadable;
• QueryBuilder-based multi-row inserts/updates.

Expected modules:

• operation.rs list queries with optional filters;
• observability.rs usage aggregations and breakdowns;
• some search/filter endpoints.

Required Infrastructure

Add SQLx offline support

Repository-level requirements:

• use sqlx prepare to generate checked metadata;
• keep checked metadata in version control if the team wants deterministic CI;
• wire verification into CI or just verify.

Concrete additions:

• sqlx-data.json or the current SQLx metadata equivalent for the chosen version;
• DATABASE_URL or test DB wiring for cargo sqlx prepare.

Introduce row structs where needed

For query_as!, add storage-local row structs for cases where domain structs should not be coupled directly to SQL column shape.

Examples:

• WorkspaceRow
• AuthUserRow
• OperationSummaryRow
• InvocationLogRow
• UsageTimelineRow

Rule:

• domain structs remain domain-oriented;
• row structs remain SQL-oriented.

Function-Level Migration Rules

For query_as!

Use query_as! when:

• the query result maps cleanly to a typed row struct;
• all returned columns are explicit;
• nullability is known.

For query!

Use query! when:

• no reusable row struct is needed;
• the query is a simple insert, update, delete, or exists fetch.

For dynamic list queries

Use:

• sqlx::QueryBuilder<Postgres> with explicit allowed sort/filter sets;
• storage-local mapper functions from row to domain object.

Migration Order

Slice B1: auth, workspace, invitation

Low-risk, fixed-shape queries.

Slice B2: secret, stream, api_key, yaml_import

Still mostly fixed-shape.

Slice B3: operation, agent

Introduce mixed strategy: macros for fixed queries, builder for dynamic lists.

Slice B4: observability

Likely the most dynamic area. Convert only static pieces first, leave complex aggregations to QueryBuilder.

Done Criteria for Track B

• all fixed-shape SQL uses compile-time checked macros;
• dynamic SQL is explicitly documented and isolated;
• CI fails on SQL drift before runtime tests;
• no new static query is accepted through sqlx::query(...) without justification.

Track C: SecretCrypto KDF Correction

Problem

Current code in crates/crank-runtime/src/secret_crypto.rs derives the AES key by hashing the master key with SHA-256:

• no KDF semantics;
• no labeled derivation context;
• no future rotation/versioning path.

Goal

Replace direct hashing with HKDF-SHA256.

Target Design

SecretCrypto::new(master_key: &str) will:

1. validate non-empty master key;
2. derive a 32-byte AES key using Hkdf<Sha256>;
3. use explicit info context, for example:
   • b"crank.secret-envelope.v1"
4. initialize Aes256Gcm from the derived bytes.

Module Changes

File:

• crates/crank-runtime/src/secret_crypto.rs

Changes:

• replace Sha256::digest(trimmed.as_bytes()) with HKDF expansion;
• preserve the public constructor signature;
• keep ciphertext format unchanged if possible;
• if ciphertext format must change later, introduce envelope version before doing so.

Required Tests

• encrypt_decrypt_roundtrip_still_works
• empty_master_key_is_rejected
• same_master_key_derives_stable_key
• different_master_keys_produce_different_ciphertexts

Done Criteria for Track C

• no direct SHA-256 KDF remains;
• SecretCrypto uses HKDF-SHA256 with explicit context;
• all existing secret encrypt/decrypt flows remain compatible.

Track D: Typed Timestamps

Problem

Domain and storage structs use String timestamps. SQL converts timestamps via repeated to_char(...).

That causes:

• repeated formatting logic in SQL;
• weak type guarantees;
• inability to do time arithmetic without reparsing;
• broad surface for formatting drift.

Goal

Use time::OffsetDateTime in domain and storage models, with RFC 3339 serialization at the transport boundary.

Target Type Strategy

Domain layer

Use:

#[serde(with = "time::serde::rfc3339")]
pub created_at: OffsetDateTime

and the same for updated_at, expires_at, used_at, and similar fields.

Storage row layer

Also use OffsetDateTime.

Do not format timestamps in SQL except for transitional compatibility shims.

Transport layer

JSON still emits RFC 3339 strings through Serde.

Files Affected

Domain crates

• crates/crank-core/src/workspace.rs
• crates/crank-core/src/access.rs
• crates/crank-core/src/auth.rs
• crates/crank-core/src/agent.rs
• crates/crank-core/src/operation.rs
• crates/crank-core/src/observability.rs
• crates/crank-core/src/secret.rs
• crates/crank-core/src/stream_session.rs

Registry models

• crates/crank-registry/src/model.rs

Admin API transport/service DTOs

• apps/admin-api/src/service.rs
• any route DTO files that mirror time fields directly

Migration Strategy

Slice D1: storage row types

• change storage row structs to OffsetDateTime;
• remove to_char(...) from fixed-shape SQL;
• keep service DTOs as strings temporarily if needed.

Slice D2: domain models

• convert crank-core models to typed timestamps with Serde helpers;
• update constructors and tests.

Slice D3: API/service transport

• remove string timestamp shims from service DTOs where possible;
• ensure JSON remains stable.

Slice D4: cleanup

• remove transitional formatting helpers;
• remove now_string() helpers in favor of typed timestamps.

Acceptance Criteria

• no created_at: String or updated_at: String remains in domain/storage models;
• no to_char(created_at...) / to_char(updated_at...) remains in registry SQL except perhaps in legacy compatibility code slated for removal in the same slice;
• JSON contract stays RFC 3339.

Track E: Display for ID Newtypes

Problem

define_id! currently requires .as_str() everywhere for formatting and messages.

Goal

Add std::fmt::Display support to every generated ID newtype.

File

• crates/crank-core/src/ids.rs

Change

Add inside define_id!:

impl std::fmt::Display for $name {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(f, "{}", self.0)
    }
}

Follow-Up Cleanup

After macro support lands:

• replace formatting sites like workspace_id.as_str() inside messages;
• keep .as_str() only where an actual &str API requires it.

Acceptance Criteria

• all ID newtypes implement Display;
• formatting code no longer uses .as_str() inside format!, println!, or error messages unless &str is actually required.

Track F: PostgreSQL Pool Configuration

Problem

The pool is currently created via default PgPoolOptions, which means the runtime behavior is implicit and under-configured.

Goal

Make pool behavior explicit and configurable from environment.

Target Config Model

Add a typed config struct, for example:

pub struct PostgresPoolConfig {
    pub max_connections: u32,
    pub min_connections: u32,
    pub acquire_timeout_ms: u64,
    pub idle_timeout_ms: u64,
    pub max_lifetime_ms: u64,
}

Runtime Environment

Add env variables:

• POSTGRES_MAX_CONNECTIONS
• POSTGRES_MIN_CONNECTIONS
• POSTGRES_ACQUIRE_TIMEOUT_MS
• POSTGRES_IDLE_TIMEOUT_MS
• POSTGRES_MAX_LIFETIME_MS

Files

• crates/crank-registry/src/postgres/connect.rs
• apps/admin-api/src/main.rs
• apps/mcp-server/src/main.rs
• config docs and examples

Behavior

connect_with_options should accept either:

• direct PgConnectOptions plus explicit pool config;
• or an internal default config built from env.

Defaults should be conservative and explicit, for example:

• max_connections = 20
• min_connections = 2
• acquire_timeout = 5s
• idle_timeout = 10m
• max_lifetime = 30m

Exact values can be tuned later, but they must no longer be hidden defaults.

Acceptance Criteria

• pool limits and timeouts are explicit;
• docs reflect the new variables;
• startup logs or config inspection make active pool settings visible.

Track G: Timestamp Formatting Deduplication

Problem

The same to_char(... at time zone 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS"Z"') fragment is copied across many queries.

Goal

Eliminate duplicated formatting logic.

Strategy

This track is mostly resolved by Track D, but there is a transition window.

During transition:

• centralize any remaining formatting in postgres/shared.rs;
• define named SQL fragments only once;
• remove inline duplicated snippets.

After Track D:

• remove formatting fragments entirely.

Acceptance Criteria

• duplicated inline timestamp formatting is no longer scattered across query strings;
• final state has typed timestamps instead of SQL formatting.

Track H: Error Structure Normalization

Problem

Some runtime and API error variants carry only a details: String, which makes programmatic handling and consistent observability harder.

Goal

Introduce structured error payloads where the current model is too string-heavy.

Scope

Priority types:

• RuntimeError
• registry storage errors where context is stringly typed;
• API error mapping around secret/runtime/streaming faults.

Design Rules

• keep thiserror;
• use structured fields in variants whenever stable context exists;
• reserve free-form details only for opaque third-party messages.

Examples:

Instead of:

RuntimeError::SecretCrypto { details: String }

prefer:

RuntimeError::SecretCrypto {
    operation: &'static str,
    details: String,
}

or dedicated variants such as:

• InvalidSecretEnvelope
• SecretDecryptFailed
• SecretSerializeFailed

Slice Plan

Slice H1

• normalize SecretCrypto and streaming/session/job errors.

Slice H2

• normalize adapter errors for REST/gRPC/WebSocket/SOAP where stable categories exist.

Slice H3

• normalize API mapping and error codes.

Acceptance Criteria

• core runtime error variants expose structured context;
• API can map key failure classes deterministically;
• logs and telemetry can group errors by category instead of by ad hoc message text.

Track I: Correlation IDs and Cross-Layer Tracing

Problem

Today, a request can enter through:

• admin-api HTTP routes;
• mcp-server JSON-RPC over Streamable HTTP;
• internal runtime execution;
• protocol adapters (REST, gRPC, WebSocket, SOAP)

without a single stable correlation key that survives the entire path.

There are already partial pieces in the codebase:

• request_id exists in observability models and storage;
• MCP mappings can inject $.mcp.correlation_id;
• some protocol fixtures mention correlation_id.

But there is no guaranteed end-to-end propagation model. When an adapter fails, logs cannot always be tied back to:

• the incoming HTTP request;
• the MCP session;
• the runtime operation execution;
• the invocation log row.

Goal

Introduce a first-class correlation model that is generated once at ingress and propagated across:

• transport;
• service/runtime orchestration;
• adapter calls;
• persistence and logs.

Target Model

Introduce two related identifiers:

• request_id
  • unique per incoming HTTP or MCP request;
  • used for ingress/egress tracing and log correlation.
• correlation_id
  • stable business-level identifier propagated into upstream payloads when configured;
  • may equal request_id by default, but remains distinct conceptually.

Flow Rules

admin-api

At HTTP ingress:

• read X-Request-Id if supplied and valid;
• otherwise generate a UUID v7 request ID;
• store it in request extensions;
• include it in structured logs;
• pass it into runtime execution context;
• persist it into invocation_logs.request_id when a test run or runtime execution creates a log row.

mcp-server

At JSON-RPC ingress:

• derive a request-scoped request_id;
• bind it to the current transport session and JSON-RPC request;
• propagate it into runtime execution and adapter context;
• return it in logs and diagnostics.

Runtime

Add an execution context struct, for example:

pub struct ExecutionContext {
    pub request_id: String,
    pub correlation_id: String,
    pub workspace_id: WorkspaceId,
    pub agent_id: Option<AgentId>,
    pub operation_id: OperationId,
    pub mcp_session_id: Option<String>,
}

This context must be passed through:

• unary execution;
• window execution;
• session tool start/poll/stop;
• async job start/status/result/cancel.

Adapters

Each adapter must receive execution context and apply it consistently:

• REST
  • add X-Request-Id
  • add X-Correlation-Id
  • allow explicit mapping into headers/query/body where configured
• gRPC
  • attach metadata keys such as x-request-id, x-correlation-id
• WebSocket
  • inject correlation metadata into subscribe payloads when configured
• SOAP
  • inject correlation value into configured header/body mappings

Files and Functions

New or expanded files

• apps/admin-api/src/request_context.rs
• apps/mcp-server/src/request_context.rs
• crates/crank-runtime/src/context.rs
• adapter request builders in each protocol crate

Functions to add or refactor

admin-api

• extract_or_generate_request_id(...)
• request_context_middleware(...)
• runtime_request_context(...)

mcp-server

• extract_mcp_request_context(...)
• runtime_request_context_from_message(...)

crank-runtime

• ExecutionContext
• PreparedRequest / RuntimeOperation updates to carry context
• execute_* functions accept &ExecutionContext

Acceptance Criteria

• every runtime invocation has a request ID;
• every adapter call can log the request ID;
• invocation logs persist the same request ID;
• operator can trace a failing adapter call back to the original HTTP or MCP request.

Track J: Rate Limiting and Backpressure

Problem

There is currently no explicit protection layer for:

• excessive API request rates;
• excessive tool invocation rates;
• excessive stream/session polling;
• unbounded adapter output pressure.

This is a production-readiness gap, especially now that the platform supports:

• unary execution;
• windowed streaming;
• session tools;
• async jobs;
• WebSocket and SSE upstream adapters.

Goal

Introduce explicit controls for request rate, concurrency, and bounded output pressure.

Scope

API-level controls

Apply to:

• admin-api routes;
• mcp-server transport endpoints.

Dimensions:

• per-IP
• per-session
• per-workspace
• per-platform-key

Runtime-level controls

Apply to:

• concurrent unary executions;
• concurrent window executions;
• concurrent live stream sessions;
• concurrent async jobs;
• polling frequency for session/job tools.

Adapter-level backpressure

Apply to:

• REST SSE
• gRPC server-streaming
• WebSocket windows/sessions

Controls:

• max buffered items
• max buffered bytes
• max window duration
• max session polls
• max per-message bytes

Target Design

admin-api

Add middleware with a keyed limiter, for example:

• by authenticated user ID;
• by workspace ID;
• by remote IP for unauthenticated endpoints.

mcp-server

Add a limiter keyed by:

• platform key ID;
• workspace slug;
• MCP session ID;
• optionally agent slug.

crank-runtime

Add a runtime guard layer, for example:

pub struct RuntimeLimits {
    pub max_concurrent_unary: usize,
    pub max_concurrent_window: usize,
    pub max_concurrent_sessions: usize,
    pub max_concurrent_jobs: usize,
    pub max_polls_per_session: u32,
}

And an execution gate:

• acquire permit before execution;
• release permit on completion;
• reject with stable error code if limits are exceeded.

Implementation Slices

Slice J1: API request throttling

• add middleware and config for admin-api;
• add middleware and config for mcp-server.

Slice J2: Runtime concurrency limits

• add semaphores/permits around execution paths.

Slice J3: Streaming/session pressure limits

• cap poll frequency and active session count;
• cap pending buffer sizes in adapters.

Acceptance Criteria

• platform can reject overload deterministically instead of degrading unpredictably;
• stream/session polling is bounded;
• slow clients cannot force unbounded buffering in streaming adapters.

Track K: Distributed MCP Session Store

Problem

apps/mcp-server/src/session.rs currently keeps transport/session state in process memory through Arc<RwLock<HashMap<...>>>.

This is acceptable only for a single instance. It breaks when:

• multiple mcp-server instances run behind a load balancer;
• a container restarts and loses in-memory session state;
• GET/POST/DELETE transport requests land on different instances.

Goal

Replace the in-memory session store with a multi-instance-safe shared store.

Target Design

Introduce a SessionStore trait:

#[async_trait]
pub trait SessionStore {
    async fn create_transport_session(&self, session: TransportSessionRecord) -> Result<...>;
    async fn get_transport_session(&self, session_id: &str) -> Result<...>;
    async fn touch_transport_session(&self, session_id: &str, now: OffsetDateTime) -> Result<...>;
    async fn delete_transport_session(&self, session_id: &str) -> Result<...>;
    async fn append_pending_message(&self, session_id: &str, message: Value) -> Result<...>;
    async fn drain_pending_messages(&self, session_id: &str) -> Result<Vec<Value>, ...>;
    async fn cleanup_expired_transport_sessions(&self, now: OffsetDateTime) -> Result<u64, ...>;
}

Provide two implementations:

• InMemorySessionStore
  • test-only and local dev fallback
• PostgresSessionStore
  • production default

Storage Model

Add tables such as:

• mcp_transport_sessions
• mcp_transport_outbox

Suggested fields:

mcp_transport_sessions

• id
• workspace_id
• agent_id
• platform_api_key_id
• transport_kind
• status
• created_at
• updated_at
• expires_at

mcp_transport_outbox

• id
• session_id
• sequence_no
• payload_json
• created_at

Semantics

POST initialize

• create transport session record in shared store.

GET SSE stream

• attach to existing session record;
• stream messages from outbox or polling bridge;
• support reconnect without requiring process-local state.

POST JSON-RPC

• resolve session from shared store;
• append outbound messages to outbox.

DELETE

• mark session closed in shared store;
• best-effort cleanup of outbox rows.

Files and Modules

Add:

• apps/mcp-server/src/session/mod.rs
• apps/mcp-server/src/session/store.rs
• apps/mcp-server/src/session/in_memory.rs
• apps/mcp-server/src/session/postgres.rs

Potential shared data types:

• crates/crank-core/src/transport_session.rs

Migration Order

Slice K1: trait and in-memory implementation

• preserve current behavior but hide it behind a trait.

Slice K2: persistent schema and Postgres store

• add migrations and registry/storage support.

Slice K3: transport handlers on shared store

• wire GET/POST/DELETE through the trait-backed store.

Slice K4: cleanup and failover semantics

• cleanup job for expired transport sessions;
• reconnect tests across store-backed state.

Acceptance Criteria

• transport session state survives process restart if store is persistent;
• GET/POST/DELETE requests can hit different instances safely;
• in-memory session state is no longer the production bottleneck.

Delivery Order

Recommended order:

1. feat/refactoring-plan
2. feat/id-display-support
3. feat/postgres-pool-config
4. feat/postgres-registry-modularization
5. feat/sqlx-compile-time-verification
6. feat/secret-crypto-hkdf
7. feat/typed-timestamps
8. feat/error-structure-normalization
9. feat/correlation-id-propagation
10. feat/runtime-rate-limiting-and-backpressure
11. feat/distributed-mcp-session-store
12. feat/post-refactor-cleanup

Reasoning:

• Display is tiny and immediately reduces noise.
• pool config is operational and isolated.
• module split should happen before large query migrations, otherwise the macro migration lands into the monolith and becomes harder to review.
• HKDF is isolated and low-risk.
• typed timestamps should land after storage modules are split and query migration is underway.

Per-Slice Verification

Every slice must run:

• just fmt-check
• just check
• just clippy
• targeted tests for the touched area

Additionally:

• SQL macro migration slices must run SQLx prepare/verification;
• timestamp slices must run serialization regression tests;
• secret crypto slice must run compatibility and roundtrip tests.

Success Definition

The refactoring track is complete when:

• registry storage code is split into domain modules;
• static SQL is compile-time validated;
• SecretCrypto uses HKDF instead of SHA-256 as a KDF;
• domain/storage timestamps are typed;
• ID newtypes implement Display;
• PostgreSQL pool settings are explicit and configurable;
• timestamp formatting duplication is removed;
• error variants carry structured context where it matters.

At that point, maintainability improves materially without changing the product model or feature scope.