Audit Infrastructure

Audit Infrastructure

Per-request observability for every public host entry-point: component-meta resolution, compile, semantic analysis, type resolution, workspace ops, LSP handlers, MCP tool invocations, and bundler-batch summaries. Each audited request produces one RequestAuditRecord envelope carrying timing, memory, store counters, scheduler attribution, per-file reads, optional semantic footprint, and a strongly-typed kind-specific payload.

For end-user API reference and debug workflows see docs/audit-footprint/.

Architecture Overview — Substrate Vs Session

Optional capture policy and availability

The binding REQUIRED / REQUIRED-budget / REQUIRED-lifetime / OPTIONAL policy, per-owner inventory format, default-off semantic-observe feature and generator constraints live in docs/arch/semantic-observe.md. Inventory files are crates/*/observe-inventory/*.md; extend the owning file, not a shared table. Required validity, budgets, diagnostic data and current occupancy/ownership charges remain independent of capture.

verter_audit::observe::CaptureAvailability::compiled() reports Unavailable when semantic-observe is off and Available when on. observe::capture returns None without calling its collector when off; it returns the collected payload when on, never fabricated zero metrics. ObserveMode supplies the uncaptured/captured vocabulary; root selection and existing audit-endpoint migration remain with the execution/consolidation owner. The feature currently implies legacy measurement gates without removing them. Integration tests in crates/verter_audit/tests/cases/observe_feature_closure.rs check resolver-2 production and dev-unified closures on host/WASM; run the audit tests both with and without --features semantic-observe.

Audit state is split between a leaf substrate crate (verter_audit) and the session crate (verter_session). verter_audit may depend only on verter_span plus ecosystem crates — never on verter_session or any other verter_* crate.

Isolation enforced by: verter_audit_no_upward_deps guard (rejects any verter_* dep in verter_audit/Cargo.toml other than verter_span) and audit_substrate_isolation guard (rejects any use verter_* under crates/verter_audit/src/ other than verter_span).

RequestAuditRecord Envelope

Top-level record (crates/verter_audit/src/record.rs):

RequestKind Variants

Typed Payload Accessors

RequestAuditRecord typed accessors (each returns None when kind_payload is not the matching variant):

• component_meta_payload() -> Option<&ComponentMetaPayload>
• type_resolution_payload() -> Option<&TypeResolutionPayload>
• compile_payload() -> Option<&CompilePayload>
• semantic_analysis_payload() -> Option<&SemanticAnalysisPayload>
• workspace_payload() -> Option<&WorkspacePayload>
• lsp_payload() -> Option<&LspRequestPayload>
• mcp_payload() -> Option<&McpToolPayload>
• bundler_batch_payload() -> Option<&BundlerBatchPayload>
• typeinfo_graph_payload() -> Option<&TypeInfoGraphPayload>
• flow_return_inference_payload() -> Option<&FlowReturnInferencePayload>

FlowReturnInference (U6 flow-return substrate)

RequestKind::FlowReturnInference audits the demand-sliced flow-return entry VerterHost::get_flow_return_type_with_audit(function, demand) (crates/verter_session/src/host_flow_return_audit.rs), which resolves ONE SemanticQueryKey::FlowReturn through the shared dispatch and returns AuditedResult<Arc<FlowReturnResult>, FlowReturnError> — the carrier's audit field is populated on BOTH arms. FlowReturnInferencePayload (crates/verter_audit/src/payloads/flow_return.rs) carries function_symbol, three per-request counters mirroring the cold-path structured events one to one, and the typed partiality reason:

The counters report THAT a request did cold work, hit a budget, or held on a cycle. partiality: Option<FlowPartialityTag> reports WHY it came back incomplete, and is None for the complete, warm-admissible outcome (and on the default-filled filtered / audit-disabled record, where no payload was collected at all):

partiality reports exactly ONE reason, never a set: the producer's typed outcome already reduced every observed gap to the FIRST in source order, so a function carrying several distinct gaps still names only the earliest. Read it as "the reason this request was partial", never as "the complete inventory of what is missing".

FlowDegradationTag and FlowFailureTag are CLOSED MIRRORS of the session's FlowReturnDegradation / FlowGap and FlowReturnFailure vocabularies, so the leaf audit substrate keeps no back-edge to verter_session. Both flatten their domain's nested closed enums — FlowReturnDegradation::FlowGap(_) reduces through the gap variant (GapGuardNarrowing, GapNominalRelation, GapClosureCapture, GapAbruptCompletion, GapUnmodeledExpression), and FlowReturnFailure's Unsupported / CallResolution / Budget arms reduce through their inner reason (UnsupportedLoop, CallUndecidable, BudgetWorkExceeded, …) — so every distinct reason keeps its own wire spelling instead of collapsing into a catch-all bucket. FlowFailureTag additionally carries UnstableState for the host's own FlowReturnError::UnstableState refusal, so the Err arm never reports an unexplained no-value.

The projection lives at the ONE producer, observed_partiality in crates/verter_session/src/host_flow_return_audit.rs, and maps through exhaustive matches (a new domain variant is a compile error, never a silently collapsed reason). It is READ-ONLY telemetry: it runs after the outcome is bound, and no admission decision, warm/cold classification, or cache identity reads it back.

Cold-vs-warm contract: a warm family hit emits NO FlowReturnStarted and bumps NO counter (cold_computes == 0 is the counter-side witness), and allocates no audit payload without an active accumulator. A degraded success never warms at all, so it reports its partiality on every call. Guards: crates/verter_session/tests/cases/g_type/flow_return_audit_contract.rs (cold/warm event + payload contract, the partial-vs-complete partiality contract, and the filter-driven projected-vs-unprojected equivalence — denying KindBit::FlowReturnInference takes the Noop arm and removes the projection outright, and the served value, degradation verdict and warm/cold sequence are unchanged) and crates/verter_session/tests/cases/g_misc0/flow_return_audit_tls_propagation.rs (TLS observer propagation across the dispatch's worker hops); the wire surface is pinned by crates/verter_audit/tests/cases/ts_bindings.rs.

AuditedResult<T, E> Carrier

AuditedResult<T, E> (crates/verter_audit/src/audited_result.rs) pairs the outcome — success T or typed error E — with the RequestAuditRecord captured while producing it. #[serde(tag = "kind")] discriminated enum (Ok { value, audit } / Err { error, audit }); both arms carry the record so the envelope survives regardless of outcome.

Lives in verter_audit, not verter_protocol: it is generic over T/E (protobuf cannot express) and embeds RequestAuditRecord — putting it in the protobuf-authoritative verter_protocol would invert the dependency or force a hand-written TS mirror. Rides the ts-rs path, exporting as export type AuditedResult<T, E> into packages/types/audit.generated.ts; packages/typeinfo imports the generated type. The typeinfo native session's _with_audit methods return AuditedResult<Arc<...>, TypeInfoRequestError>.

Surface: ok(value, audit) / err(error, audit) constructors; audit(), as_result(), into_parts(), into_result(), map(), map_err(). Home + export rule pinned by audited_result_lives_in_audit_and_exports_through_generated_ts (crates/verter_session/tests/cases/g_block/typeinfo_audit_contract_guards.rs).

Producer Entry-Points

Every public audited entry-point follows the same lifecycle: stamp a request id, build a RequestContext keyed by the matching RequestKind, construct an AuditRequestRegistration BEFORE installing the TLS guard, run the producer body under either RequestContextGuard (active) or install_noop_observer() (filtered), assemble the typed payload from per-request counters, and finalise through the registration. Filtered kinds short-circuit to None; the producer body always runs regardless of audit state.

Component-Meta

VerterHost::get_component_meta_with_resolution(canonical_id, mode) returns (Option<ComponentMetaAnalysis>, Option<ResolvedComponentMetaState>). The audit record is published into the host's bounded AuditRecordsStore and drained via HostAuditRuntime::take_record(request_id).

AuditedRequest builder (crates/verter_session/src/audited_request.rs) wraps one call in a request-scoped audit harness, resets per-thread counters, validates exactly one request was created, and returns (ComponentMetaAnalysis, ResolvedComponentMetaState, RequestAuditRecord) as a triple. AuditedRequestBuilder::resolve_component_meta is the test-facing convenience; AuditedRequestBuilder::run_custom lets a closure issue arbitrary single-request audited work.

Compile

VerterHost::compile_with_audit(canonical_id, target) -> (VerterCompileResult, Option<RequestAuditRecord>) and compile_with_audit_options(canonical_id, target, verter_options) for explicit force_vapor / force_js control. The target bitset maps to CompileTargetTag (Vdom, Ide, Vapor) on kind. Producer-side instrumentation in verter_compiler emits record_phase_timing at parse/transform/codegen/css_analysis/sourcemap boundaries and record_event(CompileCodeTransformOp) at every CodeTransform operation — the session-side RequestContext accumulates these into per-request atomics that assemble_compile_payload reads at finalize time.

Semantic Analysis

VerterHost::analyze_with_audit(canonical_id) -> (Option<AnalysisReady>, Option<RequestAuditRecord>). Probes FileArtifactStore cache before constructing the registration so from_cache is unaffected by audit work. Audit-disabled fast path runs materialize_analysis_ready with no RequestContextGuard.

Type Resolution

VerterHost::resolve_type_with_audit(query: SemanticQueryKey, canonical_hint: &str) -> (Option<TypeResolutionResult>, Option<RequestAuditRecord>). Drives one ProjectSemanticDispatch::execute(query) inside the audit window. TypeResolutionPayload reports the caller's projection mode (derived from the query variant) plus per-mode counters mined off the active RequestContext.

Workspace

VerterHost::audit_workspace_op(op: WorkspaceOp) -> RequestAuditRecord. Drives WorkspaceAccess::audit_op(op) under audit. Constructs the AuditRequestRegistration first so the registry slot precedes the workspace traversal. Returns the record unconditionally; Noop arm only suppresses the records-store side effect.

LSP

verter_lsp::audit_harness::run_with_audit(host, method, target_identity, position, body, populate) wraps each LSP handler future in:

1. An LspAuditSession keyed by LspMethodTag and RequestTargetIdentity (constructed via VerterHost::lsp_audit_begin). Registered URIs use the registry's exact stored identity; request-before-registration uses the raw URI; NotApplicable is reserved for operations with no single document target.
2. The same explicit request_deadlines policy used when audit is disabled. Production defaults every request deadline to zero (unbounded); audit never adds a feature/provider timeout.
3. finalize_ok(payload) on success or RPC error. audit_supersede is an observational latency SLO only: exceeding it emits telemetry but does not cancel or alter the response. Explicit client cancellation may still finalize a session with finalize_cancelled() through the cancellation lifecycle.
4. Optional drain to VERTER_LSP_AUDIT_TRACE_OUT (JSON-lines append, configurable via env var).

Audit-disabled fast path runs the body under the identical explicit request-deadline policy without registration cost. The audit-on and audit-off paths are therefore semantically equivalent.

Position-bound LSP payloads carry the same additive tagged identity in PositionInfo::target_identity. PositionInfo::canonical_id remains the legacy projection; an unregistered URI never becomes NotApplicable.

MCP

VerterHost::audit_mcp_tool_call(tool_name, canonical_id, args_size_bytes, f) -> (T, Option<RequestAuditRecord>) wraps a closure FnOnce(&Arc<Self>) -> McpToolOutcome<T> under audit. McpToolOutcome { value, result_size_bytes, error } carries the two facts the wrapper cannot infer (response size and optional error message). A non-empty canonical_id is tagged RegisteredCanonical; an empty value represents a tool with no single file target and is tagged NotApplicable while the retained legacy field stays empty. Sub-requests inherit the MCP request's id as parent_request_id via the scheduler-side TLS slot.

AuditRequestRegistration Lifecycle

Every audited entry-point allocates exactly one AuditRequestRegistration (crates/verter_session/src/host_audit_runtime.rs):

AuditRequestRegistration ::= Active(ActiveRegistration) | Noop

• Active — captures a Weak<RequestContext> slot in HostAuditRuntime::active_requests. finalize(record) atomically removes the slot and publishes the record into AuditRecordsStore (idempotent — first call wins). Drop defensively sweeps the slot when finalize did not run (panic/cancellation paths).
• Noop — returned when AuditConfig::consumer_filter rejects the request's RequestKind. Holds no state; finalize returns false and emits no record.

The three lifecycle methods on HostAuditRuntime (register_active_request, finalize_active_request, drop_active_request) are crate-private and have exactly ONE in-tree call site each, all in host_audit_runtime.rs. The audit_request_registration_lifecycle architecture guard mechanically enforces this.