Telemetry Best Practices

Telemetry Best Practices

This extension uses @microsoft/vscode-azext-utils (callWithTelemetryAndErrorHandling, IActionContext.telemetry) to emit telemetry events. Telemetry is public — assume every property and measurement value is shipped to a backend and visible to whoever has access to the dashboards. Treat every new property as a privacy decision.

These rules apply to the entire codebase (extension host, webviews, language services, helper packages), not to any single feature area.

When to Use This Skill

Apply it whenever code:

• Calls callWithTelemetryAndErrorHandling(...), callWithTelemetryAndErrorHandlingSync(...), or registers a command (which wraps the same).
• Assigns to context.telemetry.properties.* or context.telemetry.measurements.*.
• Defines a helper (e.g. report*Stats, track*, *Telemetry) that mutates an IActionContext.
• Adds or renames a telemetry event string (the first argument to callWithTelemetryAndErrorHandling).

Hard Rules — Never Emit PII / EUII

The following must never appear in telemetry.properties or telemetry.measurements, and never as part of an event name:

• File names, base names, file paths, directory names, workspace names.
• User IDs, resource names (other than the Azure account name — see below), database names, container/collection names, connection strings, hostnames, endpoints, ports.
• Query text, document contents, schema/DDL contents, error messages containing any of the above, free-form user input, prompts, AI responses.
• Email addresses, user names, machine names, IP addresses.
• Deterministic identifiers derived from the above, including hashes, truncated hashes, and normalized hash values. Cryptographic hashes are not inherently reversible, but guessable inputs can be recovered by hashing candidate values, and stable outputs permit correlation across sessions.

Not PII (always allowed): the AI model identifiers modelId, modelFamily, modelVendor (vendor-published values from vscode.LanguageModelChat), bounded enums you control, durations, counts, and ratios.

Allowed exceptions: OII identifiers under predefined property names

A small set of Azure identifiers are OII (Organization Identifiable Information), not PII, and may be emitted as-is — but only under the predefined property names listed below. The telemetry pipeline (and @microsoft/vscode-azext-utils) recognizes these names and handles them accordingly (subscription scoping, sanitization of the resource path, organization-tier classification, etc.).

context.telemetry.properties.subscriptionId = subscriptionId;
context.telemetry.properties.tenantId = tenantId;
context.telemetry.properties.accountName = account.name;
context.telemetry.properties.resourceId = new vscode.TelemetryTrustedValue(resourceId.rawId);

Rules:

• Use exactly these key names. Logging the same value under any other key (subId, accountId, armId, cosmosAccount, …) bypasses the special handling and counts as PII.
• resourceId must be wrapped in new vscode.TelemetryTrustedValue(...).
• Do not decompose the resource id and emit its parts (resourceGroup, databaseName, containerName, …) under separate properties — only the four keys above are allowed; everything else is PII.
• tenantId is allowed OII, not personal data. Do not remove its predefined telemetry property or explicitly add tenant IDs to context.valuesToMask. The library's built-in error sanitization may still redact GUIDs.
• Still push the other listed values to context.valuesToMask so they are redacted from error messages emitted alongside the event.

Safe alternatives when you need to correlate or categorize

• A non-persistent, in-memory session id generated with crypto.randomUUID() per session/operation.
• A bounded enum (e.g. 'mongo' | 'postgres' | 'sqlserver'), never a free-form string.
• A boolean flag (e.g. hasCustomInstructions) instead of the actual value.

Keep hashes needed for local storage keys or comparisons local. For cohort selection, report only the boolean selection result, not the underlying machine-ID hash or its normalized numeric value. Reuse existing random session IDs rather than adding another identifier when they already provide the required operation correlation.

Red flags to look for in reviews

context.valuesToMask — Defense in Depth, Not a Substitute

IActionContext.valuesToMask supplies sensitive strings for sanitization by @microsoft/vscode-azext-utils. The library sanitizes string telemetry.properties before emission, using more aggressive masking for keys containing error or exception. Other string properties receive less aggressive masking but still use the registered values. Non-error TelemetryTrustedValue properties bypass this sanitization, and numeric telemetry.measurements are not masked. Registering a raw value does not mask its deterministic derivatives, such as hashes or normalized hash values.

Masks belong to the action context on which they are registered. Nested telemetry calls and separate tRPC invocations must register their own masks before operations that can fail; caller or webview masks are not automatically inherited.

For NoSQL connections, reuse maskConnectionTelemetry to register connection values and every configured credential's account key or managed-identity client ID, including fallback credentials. Entra tenant IDs are allowed OII and are not registered as masks. Add operation-specific masks (such as query text) separately. Explicit client-ID masks avoid relying solely on the sanitizer's GUID detection.

Use it as a safety net for sensitive values that your code touches and might end up in a thrown error or log line you don't fully control:

// Extension host: push directly to the action context
context.valuesToMask.push(connectionString);
context.valuesToMask.push(masterKey, endpoint, databaseId, containerId);
context.valuesToMask.push(account.subscription.subscriptionId);
context.valuesToMask.push(userProvidedName); // database/container/resource names entered in a wizard

// Webview-bridged events: register on the per-webview TelemetryContext instead
telemetryContext.addMaskedValue(connectionString);
telemetryContext.addMaskedValue([endpoint, databaseId, containerId]);

Rules

• Always push a value to valuesToMask as soon as you obtain it if it could end up in an error path. This includes:
  • Strict secrets that must never appear in telemetry: connection strings, keys, tokens, query text, document contents, partition keys, user-entered names (database, container, resource).
  • The OII identifiers subscriptionId, resourceId, and accountName — even though they are emitted as-is under their predefined keys, they should still be masked from error messages. Tenant IDs are exempt from explicit masking.
• This is defense in depth, not a license to put a strict secret into telemetry properties. Never do properties.connectionString = cs and rely on masking; emit only privacy-safe values by construction.
• Push non-empty strings only. Empty/whitespace values match everything and corrupt logs (the central Telemetry.ts filter already drops falsy values; do not bypass it).
• If a value has multiple equivalent forms a user might see (e.g. a partition key with and without a leading /), push all forms: context.valuesToMask.push(partitionKey, partitionKey.slice(1));
• Wizard prompt/validateInput steps that capture user input should push the captured value before the step returns. See CosmosDBContainerNameStep, CosmosDBConnectionStringStep, CosmosDBPartitionKeyStep for the pattern.
• Branch data providers and tree item factories should mask resource ids/names at construction time (see CosmosDBBranchDataProvider, AccountInfo).
• For values that should be masked across all events emitted by a single webview, register them once on that webview's TelemetryContext via addMaskedValue(value) (see src/Telemetry.ts); the mask list is then applied automatically to every reportWebviewEvent / reportWebviewError call from that webview.