Fleet Management

What it does

When your agent uses it

• Standing up a Cloud Alloy fleet
• Pushing a config change to 200 collectors
• Hunting why one collector shows REMOTECONFIGSTATUSFAILED
• Validating River syntax before saving

Example prompts

• “production”
• “configure Alloy”
• “remote config the collectors”
• “/fleet-management”

Requirements

• A credential in GRAFANA_CLOUD_API_KEY
• A credential in API_TOKEN

Workflow steps

3 steps, taken from the step headings in SKILL.md.

1. 1Author + validate + deploy a pipeline
2. 2Troubleshoot a REMOTE_CONFIG_STATUS_FAILED collector
3. 3Onboard a new Alloy with the bootstrap block

What it can do on your machine

Read from SKILL.md and the folder at commit 1ccacf2. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • jq
  • curl
  • kubectl

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  Hosts in commands or code, which the agent is likely to contact:

  • prometheus-prod-01-eu-west-0.grafana.net
  • fleet-management-prod-us-east-0.grafana.net

  Also links to:

  • grafana.com
  • github.com

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • GRAFANA_CLOUD_API_KEY
  • API_TOKEN

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Fleet Management loads about 1.3k tokens when it runs, and up to ~3.3k if it reads all its reference files. Until then it costs about 242 tokens; SKILL.md has 147 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~242

When it runs · the whole SKILL.md, loaded when a task matches

~1.3k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~3.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from grafana/skills at commit 1ccacf2, republished under its Apache-2.0 licence (© grafana). 147 words, ~1,344 tokens.

name

fleet-management

description

Manage a fleet of Grafana Alloy collectors with Fleet Management — author Alloy pipelines once, target them via attribute matchers (`env="production"`, regex `region=~"us-.*"`), push remotely via OpAMP without restarting collectors. Covers pipeline create / update / matcher RPCs, collector attribute API, `remotecfg` bootstrap block (standalone + Helm), pre-deploy `alloy fmt` validation, the local Alloy UI at port 12345 for component health, and post-deploy `REMOTE_CONFIG_STATUS_APPLIED` verification. Use when standing up a Cloud Alloy fleet, pushing a config change to 200 collectors, hunting why one collector shows `REMOTE_CONFIG_STATUS_FAILED`, validating River syntax before saving, or wiring `discovery.kubernetes` → `prometheus.remote_write` — even when the user says "configure Alloy", "remote config the collectors", "push pipeline", "OpAMP", "collector is unhealthy", or "manage agent config centrally" without naming Fleet Management.

license

Apache-2.0

# 1. Save the pipeline to a local file (lint catches typos before remote)
cat > pipeline.alloy <<'EOF'
prometheus.scrape "default" {
  targets    = []
  forward_to = [prometheus.remote_write.grafana_cloud.receiver]
  scrape_interval = "60s"
}

prometheus.remote_write "grafana_cloud" {
  endpoint {
    url = "https://prometheus-prod-01-eu-west-0.grafana.net/api/prom/push"
    basic_auth {
      username = "<METRICS_USERNAME>"
      password = env("GRAFANA_CLOUD_API_KEY")
    }
  }
}
EOF

# 2. Validate syntax LOCALLY before sending to Fleet Management
alloy fmt pipeline.alloy            # rewrites in place or errors with line number
alloy validate pipeline.alloy       # full semantic check (newer Alloy releases)

# 3. Create the pipeline via API (see references/api.md for the payload schema)
BASE=https://fleet-management-prod-us-east-0.grafana.net
TOKEN=<STACK_ID>:<API_TOKEN>
PAYLOAD=$(jq -n --rawfile c pipeline.alloy '{
  name:"k8s-metrics", contents:$c,
  matchers:[{name:"env",value:"production",type:"EQUAL"}]
}')
curl -s -X POST "$BASE/pipeline.v1.PipelineService/CreatePipeline" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d "$PAYLOAD" | jq

# 4. Verify it rolled out — every targeted collector should report APPLIED within 1-2 polls
curl -s -X POST "$BASE/collector.v1.CollectorService/ListCollectors" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}' \
  | jq '.collectors[] | select(.attributes[]?.value=="production")
        | {name, remoteConfigStatus}'
# Expect every row: remoteConfigStatus == "REMOTE_CONFIG_STATUS_APPLIED"

# 1. Find failed collectors and surface the error message
curl -s -X POST "$BASE/collector.v1.CollectorService/ListCollectors" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}' \
  | jq '.collectors[] | select(.remoteConfigStatus=="REMOTE_CONFIG_STATUS_FAILED")
        | {name, msg:.remoteConfigStatusMessage}'

# 2. Re-validate the offending pipeline locally
alloy fmt pipeline.alloy

# 3. Inspect Alloy directly — UI at port 12345 shows per-component health
#    http://<COLLECTOR_HOST>:12345 → Graph / Components / Clustering tabs
kubectl -n monitoring logs -l app.kubernetes.io/name=alloy --tail=100 | grep -iE 'remote|error'

# 4. After fixing + re-pushing, re-list collectors and confirm the row flips to APPLIED.

remotecfg {
  url = "https://<FLEET_MANAGEMENT_HOST>"
  basic_auth { username = "<STACK_ID>"; password = env("GRAFANA_CLOUD_API_KEY") }
  poll_frequency = "1m"
  attributes = { "env" = env("ENVIRONMENT"), "team" = "platform" }
}

# Verify after start
curl -s http://localhost:12345/api/v0/web/components \
  | jq '.[] | select(.id=="remotecfg") | {id, health:.health.state}'
# health.state == "healthy"

© grafana, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 2 other files (references) in skills/grafana-cloud/fleet-management of grafana/skills.

• SKILL.md
• references/api.md
• references/bootstrap.md

Open the folder on GitHubat commit 1ccacf2