CI / Rust Checks (pull_request) Failing after 13m41s
CI / UI Checks (pull_request) Skipped
CI / Frontend E2E (pull_request) Skipped
CI / Community Image Smoke (pull_request) Skipped
CI / Deploy (pull_request) Skipped
Community Release / release (push) Failing after 13m33s
Recovery delivery of the accumulated, TL-accepted Epic 3 work that had never been
committed to Git (stories 3.6-3.17, see the crank board). Story 3.18 is not accepted
yet and is deliberately excluded: its logging surface, its capability registration,
its oracle projections and its tests are not part of this tree.
Accepted scope
- 3.6 typed static Resource Drafts; 3.7 publication and exact binding of static
Resource Versions; 3.8 MCP discovery/read and revocation of static Resources;
- 3.9 Resource import/export in API and UI plus TLS dependency maintenance;
3.10 REST-backed Resource templates; 3.11 shared REST read runtime and the MCP
revision/capability foundation;
- 3.12 parameterised Prompt Drafts with preview; 3.13 Prompt publication and exact
version binding to Agents; 3.14 MCP prompts; 3.15 Prompt import/export;
- 3.16 composite Prompts with published Resources (dependency content scan plus
dynamic read through the accepted 3.11 runtime, admin preview and MCP prompts/get);
- 3.17 bounded MCP completion suggestions from typed published schemas and the
resources/list admission fix for bound template publications.
Accepted evidence being reused (unchanged code paths)
- 3.16 candidate: core 7, admin-api 22, registry 4, mcp-server 4, UI 6; one final
`just verify` run 1651 passed / 0 failed / 2 ignored.
- 3.17 candidate: full run 1672 passed / 0 failed / 2 ignored, QA targeted reruns and
a real MCP Inspector 2.12.0 client smoke.
Delivery engineering in this commit
- CI: one event per change (feature branches are gated by their pull request, push
runs only on main), story events run the changed packages, the full workspace suite
stays the feature/release gate; no cache/sharding without a measured runner check.
- Release: the packaged archives, checksums and SBOM are now uploaded to the Gitea
release entry of the tag instead of dying with the runner workspace.
Not included on purpose: Story 3.18 work, build caches, runtime databases, local
credentials and scratch evidence.
Refs: crank kanban t_e39d7263; baseline 1b6db09bd1.
117 lines
3.8 KiB
Rust
117 lines
3.8 KiB
Rust
//! AT-3.17-02: completion identifiers never reach metric labels.
|
|
//!
|
|
//! The MCP request metric is recorded with a closed `McpMethod` label and the completion service emits
|
|
//! only fixed labels, counts and the served revision. This test proves it against a real recorder: the
|
|
//! rendered series of a snapshot taken after completion traffic carries no Prompt name, URI template,
|
|
//! argument name, suggestion value or Agent slug, and every `method` label value stays inside the
|
|
//! fixed enum.
|
|
|
|
use std::{collections::BTreeSet, time::Duration};
|
|
|
|
use metrics_util::debugging::DebuggingRecorder;
|
|
use serde_json::{Value, json};
|
|
|
|
use super::fixtures::*;
|
|
|
|
/// The closed `method` label set of `crank_mcp_requests_total` (crank-metrics `McpMethod`).
|
|
const METHOD_LABELS: [&str; 9] = [
|
|
"initialize",
|
|
"initialized",
|
|
"ping",
|
|
"tools_list",
|
|
"tools_call",
|
|
"notification",
|
|
"unsupported",
|
|
"response",
|
|
"invalid",
|
|
];
|
|
|
|
fn request(reference: Value, argument: &str, value: &str) -> Value {
|
|
json!({"ref": reference, "argument": {"name": argument, "value": value}})
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn completion_identifiers_never_reach_metric_labels() {
|
|
crank_observability::bind_metric_service(crank_observability::MetricService::McpServer)
|
|
.expect("explicit MCP service binding before the recorder");
|
|
let recorder = DebuggingRecorder::new();
|
|
let snapshotter = recorder.snapshotter();
|
|
recorder
|
|
.install()
|
|
.expect("isolated completion test recorder");
|
|
|
|
let endpoint = completion_endpoint().await;
|
|
for revision in [MODERN_REVISION, LEGACY_REVISION] {
|
|
for params in [
|
|
// A matched published value, a partially matched token and an unavailable reference: the
|
|
// three shapes that a naive implementation would be tempted to label with.
|
|
request(
|
|
json!({"type": "ref/prompt", "name": "greeting"}),
|
|
"tone",
|
|
"warm",
|
|
),
|
|
request(
|
|
json!({"type": "ref/resource", "uri": BOUND_TEMPLATE_URI}),
|
|
"region",
|
|
"apac",
|
|
),
|
|
request(
|
|
json!({"type": "ref/prompt", "name": "no-such-prompt"}),
|
|
"tone",
|
|
"no-such-token",
|
|
),
|
|
] {
|
|
let _ = endpoint.on_revision(revision, params).await;
|
|
}
|
|
}
|
|
tokio::time::sleep(Duration::from_millis(25)).await;
|
|
|
|
let snapshot = snapshotter.snapshot().into_vec();
|
|
assert!(
|
|
!snapshot.is_empty(),
|
|
"the completion traffic must be observable at all, otherwise this check proves nothing"
|
|
);
|
|
|
|
let rendered = format!("{snapshot:?}");
|
|
for forbidden in [
|
|
"greeting",
|
|
"prm_completion_greeting",
|
|
"completion-agent",
|
|
"completion-orders",
|
|
"no-such-prompt",
|
|
"no-such-token",
|
|
"warm",
|
|
"apac",
|
|
"tone",
|
|
"region",
|
|
] {
|
|
assert!(
|
|
!rendered.contains(forbidden),
|
|
"a completion identifier leaked into metric labels: {forbidden:?}"
|
|
);
|
|
}
|
|
|
|
let mut observed_methods = BTreeSet::new();
|
|
for (key, _, _, _) in &snapshot {
|
|
if key.key().name() != "crank_mcp_requests_total" {
|
|
continue;
|
|
}
|
|
for label in key.key().labels() {
|
|
if label.key() == "method" {
|
|
observed_methods.insert(label.value().to_owned());
|
|
}
|
|
}
|
|
}
|
|
assert!(
|
|
!observed_methods.is_empty(),
|
|
"the completion requests must produce `crank_mcp_requests_total` series"
|
|
);
|
|
for method in &observed_methods {
|
|
assert!(
|
|
METHOD_LABELS.contains(&method.as_str()),
|
|
"the `method` label must stay inside the fixed taxonomy, got {method:?} \
|
|
from {observed_methods:?}"
|
|
);
|
|
}
|
|
}
|