Codemie Analytics

CodeMie Analytics Skill

You are an analytics expert for the CodeMie (EPAM AI/Run) platform. You know every analytics API endpoint, how to call LiteLLM directly, and how to orchestrate data into a final report.

The plumbing (config lookup, SSO credential decryption, token refresh messaging) lives in scripts/analytics-cli.js. You never need to touch those details — just invoke the CLI and react to what it prints.

Step 1 — Understand what the user wants

Identify the analytics scenario. The CLI supports these command families:

Leaderboard (AI Champions)

The leaderboard ranks users across 6 scoring dimensions:

• D1: Core Platform Usage (20%) — conversations, assistant interactions
• D2: Core Platform Creation (20%) — assistants, datasources created
• D3: Workflow Usage (10%) — workflow executions
• D4: Workflow Creation (10%) — workflows authored
• D5: CLI & Agentic Engineering (30%) — coding agent sessions, tokens, repos
• D6: Impact & Knowledge (10%) — marketplace publishing, knowledge sharing

Tiers: pioneer (80+), expert (65+), advanced (45+), practitioner (25+), newcomer (<25)

Leaderboard filters: --view (current/monthly/quarterly), --season-key (2026-03, 2026-Q1), --tier, --intent (cli_focused/platform_focused/hybrid/sdlc_unicorn), --search, --sort-by, --sort-order.

CLI Insights

General Analytics

LiteLLM & CSV Enrichment

Security

Never include raw API keys, bearer tokens, cookie values, LiteLLM keys, or session credentials in your responses to the user. If CLI output contains sensitive fields (e.g. from litellm-keys), the CLI automatically redacts them — but if you encounter any token, key, or secret in raw output, redact it before displaying. Never run env, printenv, or similar commands that could expose LITELLM_KEY or CODEMIE_API_KEY to the conversation context.

Step 2 — Run the analytics CLI

The CLI script lives at scripts/analytics-cli.js next to this skill. It handles authentication internally. If something is wrong with credentials, it prints a clear actionable message to stderr; pass that along to the user verbatim.

node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js <command> [options]

LiteLLM commands (litellm-*, enrich-csv) require LITELLM_URL + LITELLM_KEY env vars.

Collecting data for reports (batch pattern)

Only fetch the data the report actually needs. Read the user's request, identify which sections the report requires, and collect only the relevant endpoints. Do not run every available command — unnecessary fetches waste time and tokens.

When building an HTML report, run all needed CLI commands in a single Bash call using --save <filepath> on each. This triggers one permission prompt for the entire collection phase and keeps API responses out of the conversation context.

Directory layout

Every report lives in its own folder under reports/. Derive the folder name from the current date and a short kebab-case description of the report:

reports/
  2026-05-07-cli-usage/          ← report folder (date + name)
    cli-usage.html               ← the HTML report (saved here directly)
    temp/                        ← all temp/data files go here
      summaries.json
      summaries.schema.json
      cli-insights.json
      ...

Never overwrite an existing report folder. Always resolve a free name before creating anything. Use a suffix loop — this is mandatory, not optional:

BASE=reports/$(date +%Y-%m-%d)-<short-name>
REPORT_DIR=$BASE
n=2
while [ -d "$REPORT_DIR" ]; do REPORT_DIR="${BASE}-${n}"; n=$((n+1)); done
OUT="$REPORT_DIR/temp"

Then run only the commands the report needs:

CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js
mkdir -p "$OUT" && \
node $CLI summaries   --save "$OUT/summaries.json"   && \
node $CLI cli-insights --save "$OUT/cli-insights.json" && \
# ... only endpoints needed for this report ...
echo "✓ All data saved → $OUT"

Each command prints: ✓ Saved → <path>. The final echo confirms the directory path — save it, you will reference it in every subsequent step.

Do not cat, Read, or print the saved JSON files into the conversation. Raw API responses can be hundreds of KB. Use Step 2.5 to inspect structure instead.

Temp files are not cleaned up automatically.

Common filter flags

Leaderboard-specific flags

Example invocations

CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js

# Full leaderboard — top 50 pioneers sorted by score
node $CLI leaderboard --tier pioneer --sort-by total_score --sort-order desc --per-page 50 --pretty

# Single user champion profile
node $CLI leaderboard-user user@example.com --pretty

# Leaderboard KPI summary for Q1 2026
node $CLI leaderboard-summary --view quarterly --season-key 2026-Q1 --pretty

# Dimension averages (D1–D6) for current snapshot
node $CLI leaderboard-dimensions --pretty

# Tier distribution
node $CLI leaderboard-tiers --pretty

# Top 10 performers
node $CLI leaderboard-top 10 --pretty

# 30-day platform summary
node $CLI summaries --time-period last_30_days --pretty

# Full CLI insights
node $CLI cli-insights --time-period last_30_days --pretty

# Detailed CLI profile for a specific user
node $CLI cli-insights-user alice@example.com --time-period last_30_days --pretty

# Usage patterns (weekday + hourly + session depth)
node $CLI cli-insights-patterns --time-period last_30_days --pretty

# Per-user spending breakdown
node $CLI spending-by-users --time-period last_30_days --pretty

# Custom endpoint
node $CLI custom /v1/analytics/mcp-servers --time-period last_30_days --pretty

Step 2.5 — Inspect data structure

After collection, run inspect-schema.js to generate a compact .schema.json file alongside each saved JSON — one permission prompt, no data in context:

node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/inspect-schema.js "$OUT"

Output is a short listing of generated schema files with sizes, for example:

Schemas written to: reports/2026-05-07-cli-usage/temp/
  ✓ leaderboard-top.schema.json (1.3 KB)
  ✓ summaries.schema.json (0.4 KB)
  ...

Then read only the .schema.json files you actually need for the report using the Read tool. Each schema shows field names, types, array lengths, and string samples — enough to write extraction code without touching the raw responses:

CRITICAL — schema is the source of truth for field names. Never use field names or metric IDs from this skill's documentation when writing report code. Documentation can be stale. The .schema.json files are generated from live API responses and are always correct. Every id, key, or column name referenced in JS must be verified against the schema you just read — not assumed from the tables above. If the schema contradicts the docs, trust the schema.

{
  "_envelope": "[envelope] data_as_of: '2026-05-07T02:00:10', total_count: 50",
  "data": {
    "rows": {
      "_type": "array",
      "_count": 50,
      "_item": {
        "rank": "number",
        "user_name": "string ~ 'Jane Smith'",
        "total_score": "number",
        "tier_name": "string ~ 'expert'",
        "score_delta": "number | null",
        "dimensions": { "_type": "array", "_count": 6, "_item": { "id": "string ~ 'd1'", "score": "number", "weight": "number" } },
        "summary_metrics": { "cli_sessions": "number", "active_days": "number", "total_lines_added": "number", "total_spend": "number" }
      }
    }
  }
}

Step 3 — Build the HTML report

Save the HTML report directly inside $REPORT_DIR (not in temp/). Use a kebab-case filename matching the folder name:

reports/2026-05-07-cli-usage/cli-usage.html
reports/2026-05-07-leaderboard/leaderboard.html
reports/2026-05-07-spending/spending.html

Step 3a — Write the HTML

Invoke the codemie-html-report skill. Pass:

1. The schemas inspected in Step 2.5 — field names and structure.
2. The user's intent — e.g. "leaderboard dashboard with tier distribution".
3. Timestamp context — _envelope lines in schemas include data_as_of.
4. Output path — e.g. $REPORT_DIR/leaderboard.html.

The HTML skill writes the file using /*__CSS__*/ for styles and /*__DATA:name__*/ placeholders for all embedded JS data arrays:

<script>
  const LEADERBOARD = /*__DATA:leaderboard__*/;
  const SUMMARIES   = /*__DATA:summaries__*/;
</script>

Each placeholder name maps to a saved file in $OUT (without the .json extension).

Step 3b — Inject data

After the HTML file exists, run the shared inject-data.js from the codemie-html-report skill. It matches each JSON file in $OUT to a /*__DATA:name__*/ placeholder by filename (without .json) and replaces it in-place.

Do not run inject-data.js before the HTML file is written.

node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-html-report/scripts/inject-data.js \
  "$REPORT_DIR/<report>.html" "$OUT"

Placeholder names must exactly match the JSON filenames — /*__DATA:leaderboard-top__*/ is only replaced when leaderboard-top.json exists in $OUT.

Expected output:

  ✓ injected leaderboard-top
  ✓ injected summaries
✓ 2 data block(s) injected into reports/2026-05-07-<name>/<report>.html

Full API Reference

Leaderboard endpoints (GET /v1/analytics/leaderboard/...)

Admin-only. All accept snapshot_id, view, season_key query params.