bsodfather/crank
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cache: add grpc read-only response caching

Browse Source
This commit is contained in:
a.tolmachev
2026-05-04 09:34:34 +00:00
parent b2c5e28cba
commit a61064ba3a
13 changed files with 306 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TASKS.md
+29 -78
View File
@@ -2,84 +2,6 @@
## Current
### `feat/optional-cache-layer`
Status: in_progress
Goal:
- добавить в продукт optional cache/coordination layer с `Valkey/Redis` backend без обязательной зависимости от него.
Main code areas:
- `crates/crank-core`
- `crates/crank-runtime`
- `apps/admin-api`
- `apps/mcp-server`
- `deploy/community/*`
- `docs/runtime-config.md`
- `docs/product-editions.md`
- `docs/commercial-boundaries.md`
Implementation slices:
1. Зафиксировать public cache contracts:
- response cache
- rate-limit store
- token replay / one-time token store
- ephemeral coordination state
2. Подготовить Community fallback path:
- без `Valkey/Redis` все должно работать
- degraded mode documented explicitly
3. Добавить optional `Valkey` service в Community deployment manifests как рекомендованный, но не обязательный runtime component.
4. Подготовить `Valkey/Redis` backend для:
- managed Cloud default
- enterprise optional cluster-aware deployment
5. Протянуть capability/config model так, чтобы решение "использовать или нет внешний cache" принимал оператор, а не кодовая база.
6. Зафиксировать safe cache boundaries:
- platform cache и response cache не смешиваются;
- ключи response cache изолируются минимум по `workspace + agent + operation + request fingerprint`;
- ключи coordination state изолируются минимум по `workspace + agent + cache scope`;
- при отсутствии внешнего cache все эти контуры продолжают работать через in-memory fallback.
7. Довести response cache не только до Community baseline, но и до первых коммерчески ценных read-only hot paths:
- `REST GET`
- `GraphQL query`
при сохранении той же изоляции и opt-in policy.
DoD:
- без `Valkey/Redis` Community и self-hosted deployment остаются рабочими;
- с `Valkey/Redis` платформа умеет снижать runtime/load pressure и хранить ephemeral coordination state;
- `Cloud` может использовать shared cache layer по умолчанию;
- `Enterprise` может подключать cache layer на своей инфраструктуре, включая кластерный вариант;
- runtime/docs/manifests не делают cache обязательным для базового запуска.
- cache boundaries и key namespaces документированы так, чтобы разные агенты и рабочие области не пересекались друг с другом.
- response cache покрывает первый коммерчески значимый protocol set без перехода к небезопасному "cache everything":
- `REST GET`
- `GraphQL query`
- дальнейшее расширение на `gRPC unary` возможно только для явно read-only / cacheable операций.
Verification:
- targeted runtime/admin/mcp tests for cache contracts and fallback mode;
- deploy manifest validation.
Progress:
- done:
- product/docs/runtime plan already define cache layer as optional and edition-aware
- public cache contracts and runtime cache config now exist in `crank-core` / `crank-runtime`
- Community fallback in-memory implementations now exist for response cache, rate-limit state, replay guard, and coordination state
- Community deployment manifest now includes optional `valkey` profile and cache env wiring without making cache mandatory
- shared `Valkey/Redis` backend layer now exists in `crank-runtime` with config-driven store factory for managed / enterprise contours
- admin-api and mcp-server ingress rate limiting now use shared external cache state when `CRANK_CACHE_BACKEND` is `valkey` or `redis`
- `ExecutionConfig.response_cache` is now part of the public operation model, and `admin-api` validates the safe Community baseline: only explicit `REST GET` operations without `auth_profile_ref`
- cache boundary and namespace rules are now documented for `platform / coordination cache` versus `response cache`
- runtime executor now performs actual response cache reads/writes for eligible `REST GET` operations with keys isolated by `workspace + agent + operation + request fingerprint`
- response cache keys now also include `operation version`, so a new published version does not reuse stale cached payloads from the previous one
- mcp-server published tool catalog now uses shared coordination snapshots across instances, while preserving the same `workspace + agent` isolation model
- response cache runtime/admin contracts now also cover `GraphQL query` as the first commercial protocol extension beyond the `REST GET` baseline
- pending:
- preserve the same isolation model while extending response cache to the next monetizable read-only path: explicit `gRPC unary` read operations
- wire replay guard into the first real token or nonce flow once commercial machine-token issuance stops being a `Community` stub
- wire coordination state into additional runtime or transport paths beyond the shared published catalog snapshot
## Planned
### `feat/open-core-repo-boundary`
Status: in_progress
@@ -127,6 +49,35 @@ Progress:
- remaining extension seams and packaging split still need to move from planning into concrete `crank-community` / `crank-enterprise` / `crank-cloud` manifests
- next management gate is to actually create the `3` target repositories before physical split starts
## Planned
### `feat/commercial-machine-token-flow`
Status: ready
Goal:
- реализовать короткоживущие и одноразовые машинные токены как коммерческий auth contour, а не как Community stub.
Main code areas:
- private `enterprise/cloud` token services
- public contracts already fixed in:
- `crates/crank-core`
- `apps/admin-api`
- `apps/mcp-server`
- `docs/agent-auth-model.md`
Implementation slices:
1. short-lived token issuance
2. one-time token issuance
3. replay guard / nonce coordination on top of cache layer
4. runtime verification and policy enforcement by `security_level`
5. capability-gated UI/admin flows for commercial editions
DoD:
- replay guard is no longer a stub abstraction and is wired into a real token flow;
- `elevated/strict` machine access works end-to-end in commercial contours;
- Community remains on static agent keys only.
### `feat/enterprise-access-governance`
Status: ready
apps/admin-api/src/app.rs
+2
View File
@@ -3605,6 +3605,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
}),
@@ -3658,6 +3659,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "ServerEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo_stream"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
});
apps/admin-api/src/service.rs
+98 -4
View File
@@ -5061,13 +5061,23 @@ fn validate_response_cache_policy(
));
}
if execution_config.streaming.is_some() {
return Err(ApiError::validation_with_context(
"response cache is supported only for unary operations".to_owned(),
json!({
"field": "execution_config.response_cache",
}),
));
}
match target {
Target::Rest(rest_target) if rest_target.method == crank_core::HttpMethod::Get => {}
Target::Graphql(graphql_target)
if graphql_target.operation_type == crank_core::GraphqlOperationType::Query => {}
Target::Grpc(grpc_target) if grpc_target.read_only => {}
_ => {
return Err(ApiError::validation_with_context(
"response cache is supported only for REST GET and GraphQL query operations"
"response cache is supported only for REST GET, GraphQL query, and read-only unary gRPC operations"
.to_owned(),
json!({
"field": "execution_config.response_cache",
@@ -5093,8 +5103,8 @@ mod tests {
use std::collections::BTreeMap;
use crank_core::{
ExecutionConfig, GraphqlOperationType, GraphqlTarget, HttpMethod, ResponseCachePolicy,
RestTarget, Target,
ExecutionConfig, GraphqlOperationType, GraphqlTarget, GrpcTarget, HttpMethod,
ResponseCachePolicy, RestTarget, StreamingConfig, Target, TransportBehavior,
};
use super::validate_response_cache_policy;
@@ -5145,7 +5155,7 @@ mod tests {
assert!(matches!(error, crate::error::ApiError::Validation { .. }));
assert_eq!(
error.to_string(),
"response cache is supported only for REST GET and GraphQL query operations"
"response cache is supported only for REST GET, GraphQL query, and read-only unary gRPC operations"
);
}
@@ -5162,6 +5172,90 @@ mod tests {
assert!(result.is_ok());
}
#[test]
fn accepts_response_cache_for_read_only_unary_grpc() {
let target = Target::Grpc(GrpcTarget {
server_addr: "http://127.0.0.1:50051".to_owned(),
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: true,
descriptor_ref: "desc_echo".into(),
descriptor_set_b64: "ZmFrZQ==".to_owned(),
});
let result = validate_response_cache_policy(&target, &cacheable_execution_config());
assert!(result.is_ok());
}
#[test]
fn rejects_response_cache_for_non_read_only_grpc() {
let target = Target::Grpc(GrpcTarget {
server_addr: "http://127.0.0.1:50051".to_owned(),
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: "desc_echo".into(),
descriptor_set_b64: "ZmFrZQ==".to_owned(),
});
let error =
validate_response_cache_policy(&target, &cacheable_execution_config()).unwrap_err();
assert!(matches!(error, crate::error::ApiError::Validation { .. }));
assert_eq!(
error.to_string(),
"response cache is supported only for REST GET, GraphQL query, and read-only unary gRPC operations"
);
}
#[test]
fn rejects_response_cache_for_streaming_grpc() {
let target = Target::Grpc(GrpcTarget {
server_addr: "http://127.0.0.1:50051".to_owned(),
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "ServerEcho".to_owned(),
read_only: true,
descriptor_ref: "desc_echo".into(),
descriptor_set_b64: "ZmFrZQ==".to_owned(),
});
let mut execution_config = cacheable_execution_config();
execution_config.streaming = Some(StreamingConfig {
mode: crank_core::ExecutionMode::Window,
transport_behavior: TransportBehavior::ServerStream,
window_duration_ms: Some(1_000),
poll_interval_ms: None,
upstream_timeout_ms: Some(1_000),
idle_timeout_ms: None,
max_session_lifetime_ms: None,
max_items: Some(1),
max_bytes: None,
aggregation_mode: crank_core::AggregationMode::RawItems,
summary_path: None,
items_path: None,
cursor_path: None,
status_path: None,
done_path: None,
redacted_paths: Vec::new(),
truncate_item_fields: false,
max_field_length: None,
drop_duplicates: false,
sampling_rate: None,
tool_family: crank_core::ToolFamilyConfig::default(),
});
let error = validate_response_cache_policy(&target, &execution_config).unwrap_err();
assert!(matches!(error, crate::error::ApiError::Validation { .. }));
assert_eq!(
error.to_string(),
"response cache is supported only for unary operations"
);
}
}
fn enrich_operation_summary(
apps/mcp-server/src/main.rs
+2
View File
@@ -3210,6 +3210,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
}),
@@ -3268,6 +3269,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "ServerEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
});
crates/crank-adapter-grpc/src/client.rs
+3
View File
@@ -229,6 +229,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: test_support::descriptor_set_b64(),
};
@@ -252,6 +253,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "ServerEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: test_support::descriptor_set_b64(),
};
@@ -288,6 +290,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: test_support::descriptor_set_b64(),
};
crates/crank-adapter-grpc/src/test_support.rs
+52
View File
@@ -1,5 +1,9 @@
use base64::{Engine as _, engine::general_purpose::STANDARD};
use futures_util::stream;
use std::sync::{
Arc,
atomic::{AtomicUsize, Ordering},
};
use tokio::net::TcpListener;
use tonic::{Request, Response, Status, transport::Server};
@@ -121,6 +125,54 @@ pub async fn spawn_metadata_echo_server() -> String {
format!("http://{}", address)
}
pub async fn spawn_counting_unary_echo_server(request_count: Arc<AtomicUsize>) -> String {
struct CountingEchoServiceImpl {
request_count: Arc<AtomicUsize>,
}
#[tonic::async_trait]
impl echo::echo_service_server::EchoService for CountingEchoServiceImpl {
async fn unary_echo(
&self,
request: Request<echo::EchoRequest>,
) -> Result<Response<echo::EchoResponse>, Status> {
let message = request.into_inner().message;
let count = self.request_count.fetch_add(1, Ordering::SeqCst) + 1;
Ok(Response::new(echo::EchoResponse {
message: format!("{message}-{count}"),
}))
}
type ServerEchoStream = std::pin::Pin<
Box<dyn futures_util::Stream<Item = Result<echo::EchoResponse, Status>> + Send>,
>;
async fn server_echo(
&self,
_request: Request<echo::EchoRequest>,
) -> Result<Response<Self::ServerEchoStream>, Status> {
Ok(Response::new(Box::pin(stream::empty())))
}
}
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let incoming = tonic::transport::server::TcpIncoming::from(listener);
tokio::spawn(async move {
Server::builder()
.add_service(echo::echo_service_server::EchoServiceServer::new(
CountingEchoServiceImpl { request_count },
))
.serve_with_incoming(incoming)
.await
.unwrap();
});
format!("http://{}", address)
}
pub fn descriptor_set_b64() -> String {
STANDARD.encode(echo::FILE_DESCRIPTOR_SET)
}
crates/crank-core/src/operation.rs
+2
View File
@@ -55,6 +55,8 @@ pub struct GrpcTarget {
pub package: String,
pub service: String,
pub method: String,
#[serde(default)]
pub read_only: bool,
pub descriptor_ref: DescriptorId,
pub descriptor_set_b64: String,
}
crates/crank-runtime/src/executor.rs
+100
View File
@@ -757,9 +757,14 @@ fn log_runtime_event(
}
fn response_cache_ttl(operation: &RuntimeOperation) -> Option<Duration> {
if operation.execution_config.streaming.is_some() {
return None;
}
let is_cacheable_protocol = match &operation.target {
Target::Rest(target) => target.method == HttpMethod::Get,
Target::Graphql(target) => target.operation_type == crank_core::GraphqlOperationType::Query,
Target::Grpc(target) => target.read_only,
_ => false,
};
if !is_cacheable_protocol {
@@ -1130,6 +1135,79 @@ mod tests {
assert_eq!(request_count.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn caches_read_only_grpc_unary_responses_within_agent_scope() {
let request_count = Arc::new(AtomicUsize::new(0));
let server_addr =
grpc_test_support::spawn_counting_unary_echo_server(Arc::clone(&request_count)).await;
let executor = RuntimeExecutor::new()
.with_response_cache_store(Arc::new(InMemoryResponseCacheStore::default()));
let operation = test_cached_grpc_operation(&server_addr);
let first_context = RuntimeRequestContext::from_request_id("req_cache_grpc_1")
.with_response_cache_scope("ws_cache", "agent_sales");
let second_context = RuntimeRequestContext::from_request_id("req_cache_grpc_2")
.with_response_cache_scope("ws_cache", "agent_sales");
let first_output = executor
.execute_with_context(
&operation,
&json!({ "message": "hello" }),
Some(&first_context),
)
.await
.unwrap();
let second_output = executor
.execute_with_context(
&operation,
&json!({ "message": "hello" }),
Some(&second_context),
)
.await
.unwrap();
assert_eq!(first_output, json!({ "message": "hello-1" }));
assert_eq!(second_output, json!({ "message": "hello-1" }));
assert_eq!(request_count.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn skips_grpc_response_cache_without_read_only_flag() {
let request_count = Arc::new(AtomicUsize::new(0));
let server_addr =
grpc_test_support::spawn_counting_unary_echo_server(Arc::clone(&request_count)).await;
let executor = RuntimeExecutor::new()
.with_response_cache_store(Arc::new(InMemoryResponseCacheStore::default()));
let mut operation = test_cached_grpc_operation(&server_addr);
if let Target::Grpc(target) = &mut operation.target {
target.read_only = false;
}
let first_context = RuntimeRequestContext::from_request_id("req_cache_grpc_skip_1")
.with_response_cache_scope("ws_cache", "agent_sales");
let second_context = RuntimeRequestContext::from_request_id("req_cache_grpc_skip_2")
.with_response_cache_scope("ws_cache", "agent_sales");
let first_output = executor
.execute_with_context(
&operation,
&json!({ "message": "hello" }),
Some(&first_context),
)
.await
.unwrap();
let second_output = executor
.execute_with_context(
&operation,
&json!({ "message": "hello" }),
Some(&second_context),
)
.await
.unwrap();
assert_eq!(first_output, json!({ "message": "hello-1" }));
assert_eq!(second_output, json!({ "message": "hello-2" }));
assert_eq!(request_count.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn emits_runtime_tracing_with_request_context() {
let base_url = spawn_runtime_server().await;
@@ -2244,6 +2322,26 @@ mod tests {
})
}
fn test_cached_grpc_operation(server_addr: &str) -> RuntimeOperation {
let mut operation = test_grpc_operation(server_addr);
operation.operation_id = OperationId::new("op_grpc_cached_runtime");
operation.tool_name = "echo_unary_grpc_cached".to_owned();
if let Target::Grpc(target) = &mut operation.target {
target.read_only = true;
}
operation.execution_config.response_cache =
Some(crank_core::ResponseCachePolicy { ttl_ms: 5_000 });
operation.tool_description = ToolDescription {
title: "Unary Echo gRPC Cached".to_owned(),
description: "Reads a cacheable unary gRPC payload".to_owned(),
tags: vec!["grpc".to_owned(), "cache".to_owned()],
examples: vec![ToolExample {
input: json!({ "message": "hello" }),
}],
};
operation
}
fn test_grpc_operation(server_addr: &str) -> RuntimeOperation {
RuntimeOperation::from(Operation {
id: OperationId::new("op_grpc_runtime"),
@@ -2259,6 +2357,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "UnaryEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
}),
@@ -2340,6 +2439,7 @@ mod tests {
package: "echo".to_owned(),
service: "EchoService".to_owned(),
method: "ServerEcho".to_owned(),
read_only: false,
descriptor_ref: DescriptorId::new("desc_echo"),
descriptor_set_b64: grpc_test_support::descriptor_set_b64(),
}),
docs/admin-api.md
+4 -3
View File
@@ -139,10 +139,11 @@
- `REST`
- `security_level = standard`
- `execution_config.response_cache` допускается только для:
- `REST`
- `GET`
- или `GraphQL query`
- `REST GET`
- `GraphQL query`
- `gRPC unary`, если `GrpcTarget.read_only = true`
- операций без `auth_profile_ref`
- операций без `streaming`
- `execution_config.response_cache` не означает глобальный shared cache на все вызовы системы:
- response cache должен быть изолирован минимум по `workspace + agent + operation + operation version + request fingerprint`
- попытка создать, обновить или импортировать операцию с неподдерживаемым `protocol` или `security_level` должна завершаться `validation_error` еще на стороне `admin-api`, а не только скрываться в UI.
docs/data-model.md
+8 -4
View File
@@ -463,6 +463,7 @@
- `package`
- `service`
- `method`
- `read_only`
- `descriptor_ref`
- `descriptor_set_b64`
@@ -534,14 +535,17 @@
- policy задается явно на операции;
- response cache допускается только для read-only вызовов;
- в открытой редакции первый поддержанный путь:
- `REST`
- `GET`
- без `auth_profile_ref`
- текущие поддержанные пути:
- `REST GET`
- `GraphQL query`
- `gRPC unary` только при `GrpcTarget.read_only = true`
- все варианты только без `auth_profile_ref`
- операции со `streaming` не допускают response cache даже при наличии policy;
- cache key должен строиться не глобально, а минимум в контексте:
- `workspace`
- `agent`
- `operation`
- `operation version`
- `request fingerprint`
Это позволяет избежать неявного кэширования ответов, зависящих от upstream credentials.
docs/implementation-plan.md
+2 -1
View File
@@ -201,7 +201,8 @@
- response cache уже покрывает:
- `REST GET` как Community baseline;
- `GraphQL query` как первый коммерчески ценный read-only protocol path.
- дальнейшее расширение response cache идет только по явно обоснованным read-only сценариям, начиная с `gRPC unary`, а не как общий cache для всех протоколов подряд.
- `gRPC unary` для явно read-only операций через `GrpcTarget.read_only`.
- дальнейшее расширение response cache идет только по отдельно обоснованным read-only сценариям, а не как общий cache для всех протоколов подряд.
## 14. Этап 11. Live staging and demo readiness
docs/mcp-interface.md
+2
View File
@@ -240,7 +240,9 @@ Crank должен корректно работать, если MCP client де
- если операция явно включает `execution_config.response_cache`, текущий поддержанный runtime path ограничен:
- `REST GET`
- `GraphQL query`
- `gRPC unary` только при `GrpcTarget.read_only = true`
- без `auth_profile_ref`
- без `streaming`
- cache keys изолируются минимум по `workspace + agent + operation + operation version + request fingerprint`
- streaming operations публикуются как bounded tools или tool families;
- reload published tools без пересборки сервиса;
docs/runtime-config.md
+2 -1
View File
@@ -187,9 +187,10 @@ Demo/deployment:
- `REST GET`;
- `GraphQL query`.
- `gRPC unary` только при `GrpcTarget.read_only = true`.
Это не означает автоматическое кэширование всех protocol families. Следующим кандидатом на расширение
может быть только явно read-only `gRPC unary`.
может быть только следующий отдельно обоснованный protocol path, а не "cache everything".
Эти контуры не должны смешивать ключи друг с другом.