Files
crank/apps/mcp-server/tests/integration/mcp_completion/response_metrics.rs
T
bsodfather 54ce770682
CI / Rust Checks (pull_request) Failing after 13m41s
CI / UI Checks (pull_request) Skipped
CI / Frontend E2E (pull_request) Skipped
CI / Community Image Smoke (pull_request) Skipped
CI / Deploy (pull_request) Skipped
Community Release / release (push) Failing after 13m33s
feat(epic3): deliver the accepted Epic 3 scope through Story 3.17
Recovery delivery of the accumulated, TL-accepted Epic 3 work that had never been
committed to Git (stories 3.6-3.17, see the crank board). Story 3.18 is not accepted
yet and is deliberately excluded: its logging surface, its capability registration,
its oracle projections and its tests are not part of this tree.

Accepted scope
- 3.6 typed static Resource Drafts; 3.7 publication and exact binding of static
  Resource Versions; 3.8 MCP discovery/read and revocation of static Resources;
- 3.9 Resource import/export in API and UI plus TLS dependency maintenance;
  3.10 REST-backed Resource templates; 3.11 shared REST read runtime and the MCP
  revision/capability foundation;
- 3.12 parameterised Prompt Drafts with preview; 3.13 Prompt publication and exact
  version binding to Agents; 3.14 MCP prompts; 3.15 Prompt import/export;
- 3.16 composite Prompts with published Resources (dependency content scan plus
  dynamic read through the accepted 3.11 runtime, admin preview and MCP prompts/get);
- 3.17 bounded MCP completion suggestions from typed published schemas and the
  resources/list admission fix for bound template publications.

Accepted evidence being reused (unchanged code paths)
- 3.16 candidate: core 7, admin-api 22, registry 4, mcp-server 4, UI 6; one final
  `just verify` run 1651 passed / 0 failed / 2 ignored.
- 3.17 candidate: full run 1672 passed / 0 failed / 2 ignored, QA targeted reruns and
  a real MCP Inspector 2.12.0 client smoke.

Delivery engineering in this commit
- CI: one event per change (feature branches are gated by their pull request, push
  runs only on main), story events run the changed packages, the full workspace suite
  stays the feature/release gate; no cache/sharding without a measured runner check.
- Release: the packaged archives, checksums and SBOM are now uploaded to the Gitea
  release entry of the tag instead of dying with the runner workspace.

Not included on purpose: Story 3.18 work, build caches, runtime databases, local
credentials and scratch evidence.

Refs: crank kanban t_e39d7263; baseline 1b6db09bd1.
2026-10-25 09:08:22 +00:00

117 lines
3.8 KiB
Rust

//! AT-3.17-02: completion identifiers never reach metric labels.
//!
//! The MCP request metric is recorded with a closed `McpMethod` label and the completion service emits
//! only fixed labels, counts and the served revision. This test proves it against a real recorder: the
//! rendered series of a snapshot taken after completion traffic carries no Prompt name, URI template,
//! argument name, suggestion value or Agent slug, and every `method` label value stays inside the
//! fixed enum.
use std::{collections::BTreeSet, time::Duration};
use metrics_util::debugging::DebuggingRecorder;
use serde_json::{Value, json};
use super::fixtures::*;
/// The closed `method` label set of `crank_mcp_requests_total` (crank-metrics `McpMethod`).
const METHOD_LABELS: [&str; 9] = [
"initialize",
"initialized",
"ping",
"tools_list",
"tools_call",
"notification",
"unsupported",
"response",
"invalid",
];
fn request(reference: Value, argument: &str, value: &str) -> Value {
json!({"ref": reference, "argument": {"name": argument, "value": value}})
}
#[tokio::test]
async fn completion_identifiers_never_reach_metric_labels() {
crank_observability::bind_metric_service(crank_observability::MetricService::McpServer)
.expect("explicit MCP service binding before the recorder");
let recorder = DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
recorder
.install()
.expect("isolated completion test recorder");
let endpoint = completion_endpoint().await;
for revision in [MODERN_REVISION, LEGACY_REVISION] {
for params in [
// A matched published value, a partially matched token and an unavailable reference: the
// three shapes that a naive implementation would be tempted to label with.
request(
json!({"type": "ref/prompt", "name": "greeting"}),
"tone",
"warm",
),
request(
json!({"type": "ref/resource", "uri": BOUND_TEMPLATE_URI}),
"region",
"apac",
),
request(
json!({"type": "ref/prompt", "name": "no-such-prompt"}),
"tone",
"no-such-token",
),
] {
let _ = endpoint.on_revision(revision, params).await;
}
}
tokio::time::sleep(Duration::from_millis(25)).await;
let snapshot = snapshotter.snapshot().into_vec();
assert!(
!snapshot.is_empty(),
"the completion traffic must be observable at all, otherwise this check proves nothing"
);
let rendered = format!("{snapshot:?}");
for forbidden in [
"greeting",
"prm_completion_greeting",
"completion-agent",
"completion-orders",
"no-such-prompt",
"no-such-token",
"warm",
"apac",
"tone",
"region",
] {
assert!(
!rendered.contains(forbidden),
"a completion identifier leaked into metric labels: {forbidden:?}"
);
}
let mut observed_methods = BTreeSet::new();
for (key, _, _, _) in &snapshot {
if key.key().name() != "crank_mcp_requests_total" {
continue;
}
for label in key.key().labels() {
if label.key() == "method" {
observed_methods.insert(label.value().to_owned());
}
}
}
assert!(
!observed_methods.is_empty(),
"the completion requests must produce `crank_mcp_requests_total` series"
);
for method in &observed_methods {
assert!(
METHOD_LABELS.contains(&method.as_str()),
"the `method` label must stay inside the fixed taxonomy, got {method:?} \
from {observed_methods:?}"
);
}
}