Grafana Dashboards

Grafana Dashboards v2 (Org Monitoring by sfdx-hardis)

Everything needed to build, modify and validate the v2 Grafana dashboards without rediscovering the constraints. The generic Grafana skills (dashboarding, promql, loki, alerting-irm in .claude/skills/) cover Grafana itself; THIS skill covers the sfdx-hardis-specific rules.

Source of truth

• Dashboards: docs/grafana/dashboards-v2/*.json - GENERATED FILES, never edit them directly.
• Generator: docs/grafana/dashboards-v2/generator.mjs. Edit it, then run node generator.mjs from that folder to rewrite all JSONs.
• Panel id lock: docs/grafana/panel-ids-v2.json - generated, commit it with the dashboards, never hand-edit. Keeps every panel id stable across insertions and reorderings (see the DON'Ts).
• Alert pack: docs/grafana/alerts-v2/sfdx-hardis-alerts.yaml (hand-written YAML, all rules isPaused: true).
• Lint suite: test/grafana-dashboards-v2.test.ts - enforces most rules below; run with npx mocha "test/grafana-dashboards-v2.test.ts".
• Docs page: docs/salesforce-monitoring-grafana-v2.md.
• Installer command: hardis:org:configure:grafana-dashboards (src/common/grafana/grafanaDashboardsInstaller.ts) fetches the dashboards from GitHub raw at runtime and imports them via the Grafana HTTP API. When adding or removing a dashboard JSON, update GRAFANA_V2_FALLBACK_DASHBOARD_FILES there (enforced by the lint suite).
• v1 dashboards (docs/grafana/dashboards/) are frozen: never modify them.

Data model reminder

sfdx-hardis pushes to two backends (see src/common/notifProvider/apiProvider.ts):

• Loki (logs): stream labels source="sfdx-hardis", type (notification type key), orgIdentifier, gitIdentifier, severity. The log line is a JSON body: metric, _metrics, _metricsKeys, _logElements (detail rows), _title, _logBodyText, _dateTime, _jobUrl.
• Prometheus/Mimir (metrics): each metrics key of a notification becomes <Key>_metric (plus _max / _percent variants for object-form values), labeled with source, type, orgIdentifier, gitIdentifier.

PII: in CI, sensitive fields in _logElements are pseudonymized (user_xxxxxxxxxx, id_xxxxxxxxxx, ip_xxxxxxxxxx) by src/common/utils/anonymizeUtils.ts (levels: standard / strict, actor fields readable at standard). Dashboards must never rely on readable usernames.

Two emission rules decide whether a panel can show a number at all:

• A value of exactly 0 IS emitted. apiProvider used to test min/max/percent/value for truthiness, so a metric sitting at zero produced no field and the series disappeared: a limit at 0% read as "no data" instead of charting a zero. Fixed in 7.24; older CLI versions still drop it, which is what RECENT_CLI_NOTE warns about.
• A feature that is not provisioned emits NO metric, rather than a zero (no AiUsageCreditsTotal on an org without a Data 360 consumption model). A confident 0 would claim "nothing consumed" when the truth is "not available here". Such panels need a noValue empty state ("Requires Data 360"), never a threshold that treats 0 as good news.

DOs

• DO wrap every Prometheus selector in a lookback window: last_over_time(M{...}[2d]), avg_over_time(M{...}[7d]), etc. Metrics arrive ONCE PER DAY; a bare selector returns "no data" (5-minute default lookback). Enforced by lint.
• DO aggregate per org: max by (orgIdentifier) (...) on every stat/gauge query (min by for time-to-exhaustion forecasts). Orgs can expose several series for one metric (two gitIdentifier values after a monitoring repo/branch rename); without aggregation, stat panels show two confusing values. For per-limit series use max by (__name__).
• DO use the datasource variables ${ds_prom} / ${ds_loki} for every panel and target (helpers DS_PROM/DS_LOKI in the generator). They are hidden (hide: 2) - there is always exactly one Prometheus and one Loki receiving sfdx-hardis data.
• DO put a detail link on every number: each stat/gauge/bargauge must be clickable via fieldConfig.defaults.links. Helpers: indicatorLink('<TYPE>') (generic Indicator Detail dashboard filtered on the notification type), detailLink('<dash-slug>') (another org dashboard), viewPanelLink(...) / linkStatToPanel(...) (full-screen view of a table on the same dashboard, used on fleet for silent orgs / orgs without backup). Enforced by lint (exempt: "Stats generation date", "Latest value", the dtl-indicator dashboard).
• DO propagate variables AND the time range in links: append ${ds_prom:queryparam}&${ds_loki:queryparam}&${__url_time_range} (already in the helpers) or ${__all_variables}&${__url_time_range} for same-dashboard viewPanel links. The time range is not optional: a stat viewed over 180d shows the last non-null value in that window, so its detail link must open the same window or a stale value lands on an empty page.
• DO show averages and trends: new numeric indicators should get avg/day (7d/30d) context where relevant (avgStat helper), not just the latest value.
• DO respect the fleet Environment filter: every fleet-dashboard query must include the $env matcher ({source="sfdx-hardis", $env, orgIdentifier=~"$org"}). It injects orgIdentifier!~".*sandbox.*" for "Production only" (RE2 has no lookbehind, hence matcher-as-variable-value).
• DO use whitelist organize transformations on Loki tables (includeByName) so backend noise columns (labels, traceID, detected_level) can never appear. __proxy_source__ is excluded centrally by the organize() helper.
• DO reuse the v1-proven jsonArrayToRows('<field>') chain to render a JSON array field of the latest Loki entry as table rows (_logElements, topFailingFlows, ...).
• DO keep tables vertical: one row per item, 2-4 columns. Wide "one column per item" layouts cause horizontal scrollbars (the days-until-limit table uses reduce seriesToRows + legendFormat per query for this).
• DO keep stat titles short (fits a 4-unit-wide box, ~15 chars): "Apex avg/day (30d)", not "Apex errors average per day (30 days)". Longer context goes in the panel description. The ~15 char budget is tied to the box width, so scale it with gridPos.w: on a wider panel a longer title is fine, and when a title reads as clipped mid-word the fix is to widen the panel or shorten the words, never to let Grafana truncate it.
• DO handle no-data: noValue text on stats (e.g. "Schedule dora-report"), range mappings for degenerate values ("Limit exceeded" for negative forecast days, "No growth" above 3650), and a description mentioning "Requires a recent sfdx-hardis version" for panels fed by newly added metrics (RECENT_CLI_NOTE).
• DO validate live before delivering: .env contains GRAFANA_TOKEN, a service account of the team's Grafana instance, whose URL goes in GRAFANA_API_URL in the same .env (never in the repository). Test queries through the datasource proxy (/api/datasources/proxy/uid/grafanacloud-prom/api/v1/query, .../grafanacloud-logs/loki/api/v1/query), then import into folder uid sfdx-hardis-v2 via POST /api/dashboards/db with overwrite: true.
• DO run the lint suite after regenerating - it checks JSON validity, v2 uids/tags, ds variables, no hardcoded stack references, *_over_time wrappers, unique panel ids, and detail links on every number.