Enable Dsm

Claude runs

DSM_LABELS="<DSM_APP_LABELS space-separated>"   # e.g. "order-producer billing-consumer"; empty if the DSM target has no podSelector
OK=0
END=$(( $(date +%s) + 480 ))
while [ "$(date +%s)" -lt "$END" ]; do
  STATE=$(kubectl get pods -n <AGENT_NAMESPACE> -l agent.datadoghq.com/component=cluster-agent \
    -o jsonpath='{range .items[*]}{.metadata.name}{"|"}{.metadata.deletionTimestamp}{"|"}{.status.conditions[?(@.type=="Ready")].status}{"|"}{.spec.containers[0].env[?(@.name=="DD_APM_INSTRUMENTATION_TARGETS")].value}{"\n"}{end}')
  SEL=$(kubectl get mutatingwebhookconfigurations \
    -o jsonpath='{range .items[*].webhooks[?(@.name=="datadog.webhook.lib.injection")]}{.objectSelector}{end}')
  TOTAL=$(printf '%s\n' "$STATE" | grep -c .)
  GOOD=$(printf '%s\n' "$STATE" | grep -c '^[^|]*||True|.*DD_DATA_STREAMS_ENABLED')
  for L in $(printf '%s' "$DSM_LABELS"); do
    [ "$(printf '%s\n' "$STATE" | grep -c "\"$L\"")" -eq "$TOTAL" ] || GOOD=-1
  done
  if [ "$TOTAL" -gt 0 ] && [ "$GOOD" -eq "$TOTAL" ] && printf '%s' "$SEL" | grep -q NotIn; then
    OK=1; break
  fi
  sleep 10
done
if [ "$OK" = 1 ]; then
  sleep 30   # lets the old pod leave the webhook Service and the webhook config propagate
  echo "Cluster Agent is on the new config and the SSI webhook is active"
else
  echo "TIMED OUT waiting for the Cluster Agent: do not restart applications"
fi

Replace the DSM_LABELS placeholder before running it. The loop passes when every Cluster Agent pod is Ready, none is terminating, each carries the DSM setting and every service in DSM_APP_LABELS, and the injection webhook has its SSI selector. Until then, pods are admitted by a Cluster Agent with the old config, or by a webhook that still has the pre-SSI opt-in selector: only the Cluster Agent holding the leader lock updates the webhook configuration, and leadership can take a minute or more to move after a rollout. That matters most when SSI is first switched on, so expect the wait to take up to about two minutes then and under a minute otherwise. The webhook uses failurePolicy: Ignore, so those pods start without the change and nothing reports an error.

The first injection after a Cluster Agent restart also looks up the SDK image digests from the Datadog registry and caches them for an hour. On a slow network that lookup can exceed the webhook's 10-second timeout, so that first pod starts uninjected even after the loop passes. The first-restart check below catches this.

If it prints TIMED OUT, do not restart applications. Check kubectl describe pod and the logs of the newest Cluster Agent pod, then go to troubleshoot-ssi.

Applying label changes to a Deployment (for example Unified Service Tags from enable-ssi) also rolls its pods immediately. Apply those only after this wait.

Confirm with the user before restarting. Tell the user: "I need to restart <DSM_SERVICES> for the DSM setting to reach the pods. This will cause a brief outage. Ready to proceed?" Wait for confirmation.

Restart one Deployment in DSM_SERVICES first and run the checks below on it. Only restart the rest once it shows the SSI init containers and the DSM variable. For each Deployment in DSM_SERVICES, in its own namespace:

Claude runs

kubectl rollout restart deployment/<DEPLOYMENT_NAME> -n <NAMESPACE>
kubectl rollout status deployment/<DEPLOYMENT_NAME> -n <NAMESPACE> --timeout=180s
kubectl get pod -l <DSM_LABEL_KEY>=<APP_LABEL> -n <NAMESPACE> --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{range .items[-1:].spec.containers[*]}{.name}{"="}{.env[?(@.name=="DD_DATA_STREAMS_ENABLED")].value}{"\n"}{end}'

If the application container shows =true, DSM is configured for that service.

ERROR: Empty. Check the newest pod's init containers and the target it matched:

kubectl get pod -l <DSM_LABEL_KEY>=<APP_LABEL> -n <NAMESPACE> --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1:].spec.initContainers[*].name}{"\n"}{.items[-1:].metadata.annotations.internal\.apm\.datadoghq\.com/applied-target}{"\n"}'

• No datadog-lib-*-init at all → most often the first injection timed out on the registry digest lookup. Wait 30 seconds and restart that Deployment once more; the digest is now cached. If it is still uninjected, re-run the wait above.
• The applied target is default but the pod's labels match the DSM selectors → the pod was admitted by a Cluster Agent with the old config. Re-run the wait and restart.
• Otherwise the pod doesn't match the DSM target. Compare the target's namespaceSelector / podSelector with the pod's namespace and labels.

Then confirm that workloads outside DSM_SERVICES are still instrumented. A server-side dry run sends a test pod through the injection webhook without creating or restarting anything:

Claude runs

kubectl run dsm-admission-check -n <A_NAMESPACE_IN_SSI_SCOPE> --image=busybox --restart=Never \
  --dry-run=server -o jsonpath='{.metadata.annotations.internal\.apm\.datadoghq\.com/applied-target}{"\n"}{.spec.initContainers[*].name}{"\n"}'

Expect the default target and a datadog-lib-*-init container.

ERROR: No init container, or a target other than default. The default target is missing or listed before the DSM target. Fix the order and re-apply.

Java alternative without a restart: on the APM Service Page, Enable DSM turns it on through Remote Configuration.

Step 2b: Linux (SSI on a host)

Add the variable to the same systemd drop-in enable-ssi created for Unified Service Tags.

What you need to do in a terminal

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST>
sudo systemctl edit <SYSTEMD_SERVICE_NAME>

Add below the existing DD_SERVICE / DD_ENV / DD_VERSION lines:

Environment="DD_DATA_STREAMS_ENABLED=true"

For supervisord or pm2, add DD_DATA_STREAMS_ENABLED="true" to the same environment / env block that holds the UST vars. Do not reload yet.

Confirm with the user before restarting. Tell the user: "I need to restart <SYSTEMD_SERVICE_NAME> for DSM to take effect. This will cause a brief outage. Ready to proceed?" Wait for confirmation.

Claude runs

ssh -o StrictHostKeyChecking=no -i <SSH_KEY> <SSH_USER>@<SSH_HOST> \
  "sudo systemctl daemon-reload && sudo systemctl restart <SYSTEMD_SERVICE_NAME> && sleep 3 && \
   sudo cat /proc/\$(systemctl show -p MainPID <SYSTEMD_SERVICE_NAME> | cut -d= -f2)/environ | tr '\0' '\n' | grep DD_DATA_STREAMS"

For supervisord, restart only this program: sudo supervisorctl reread && sudo supervisorctl update <PROGRAM>. For pm2: pm2 restart <APP> --update-env (a plain reload keeps the old environment). Then check the process environment with sudo cat /proc/<PID>/environ | tr '\0' '\n' | grep DD_DATA_STREAMS.

If DD_DATA_STREAMS_ENABLED=true is printed, continue to Step 3.

Step 2c: AWS Lambda

SSI does not apply to Lambda. The function must already use the Datadog Lambda library or extension. Set DD_DATA_STREAMS_ENABLED=true in the function's environment in its IaC (serverless.yml, SAM, CDK, or Terraform), not on the live function. Tell the user to confirm the runtime's minimum Lambda library version in the DSM setup docs. Ask the user before deploying.

Step 3: Verify DSM data arrives

DSM data appears only after a DSM-enabled service produces or consumes a message. Allow a few minutes after the first message.

Claude runs

DD_SITE=<DD_SITE> pup metrics query \
  --query "avg:data_streams.latency{service:<SERVICE_NAME>,env:<ENV>} by {pathway_type}" \
  --from 15m --to now

If series are returned, DSM is working for that service. If no message has flowed yet, tell the user DSM will appear after the first one.

ERROR: No series after traffic has flowed for 5+ minutes:

• Confirm the service actually sent or received messages in that window (check its traces for a produce/consume span)
• Confirm the client library and tracer version are in the language table above
• If DD_DATA_STREAMS_ENABLED did not reach a .NET 3.22+ process, it is still in default mode, which skips Kafka/Kinesis messages under 34 bytes and RabbitMQ messages over 128 KB
• Java on kafka-clients 3.7.x shows latency but no lag. Upgrade the client
• SQS: DSM needs one free message attribute (SQS allows 10 per message)
• SNS to SQS: enable SNS raw message delivery (Java SQS v1: set DD_TRACE_SQS_BODY_PROPAGATION_ENABLED=true)

Done

Exit when ALL of the following are true:

• The user explicitly agreed to enable DSM, after hearing the plan rule
• DD_DATA_STREAMS_ENABLED=true is set on the scope the user agreed to, and nowhere else
• Kubernetes: the Cluster Agent is 7.73 or later, and the dry-run admission check matched the default target with an SSI init container
• Data verified in Step 3 (or in onboarding-summary when called from enable-ssi), or the user was told DSM will appear after the first message flows
• Any service that could not be enabled (PHP, Go without Orchestrion, unsupported client, Redis-backed queue) was named to the user, with the reason

Then give the user the link: https://app.<DD_SITE>/data-streams

Security constraints

• Never write a raw API key into any file or chat message
• Never enable DSM without the user's explicit yes
• Always confirm with the user before restarting services or deploying functions
• Do not modify application source code. DSM for supported libraries is configuration only
• DD_DATA_STREAMS_ENABLED is a non-secret boolean. Never place API keys or other secrets in ddTraceConfigs, systemd drop-ins, or IaC environment blocks
• Documentation links are for the user. Do not fetch them at runtime to decide what to run