Agent skill

Olakai Agent Project Setup

by andrewyng in andrewyng/context-hub

Builds a new AI agent with Olakai monitoring from the start: CLI login, SDK integration, per-agent KPI configuration and an end-to-end check that data flows.

MITAuto-check passedAI & LLM Engineering

Install Olakai Agent Project Setup

skills CLI
$ npx skills add andrewyng/context-hub --skill new-project -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install andrewyng/context-hub new-project --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/andrewyng/context-hub.git skills-src && mkdir -p .claude/skills && cp -r skills-src/content/olakai/skills/new-project .claude/skills/new-project && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
new-project
GitHub stars
14k
Token cost
~5k tokens
SKILL.md length
1,084 words
Files
1
Skills in repo
8
Repo updated
First seen
Licence
MIT

At a glance

Builds a new AI agent with Olakai monitoring from the start: CLI login, SDK integration, per-agent KPI configuration and an end-to-end check that data flows.

  • Works in 5 steps: Design the Agent Architecture → Configure Olakai Platform → Implement SDK Integration → …
  • Starting a new agent that should report KPIs to Olakai from day one
  • SKILL.md covers Prerequisites, Why Custom KPIs Are Essential, Understanding the customData… and Step 1: Design the Agent…, plus 5 more sections
  • Calls npm, jq and pip; needs OLAKAI_API_KEY and OPENAI_API_KEY

What it does

Setup begins with the Olakai CLI installed through npm and authenticated with `olakai login`, plus an SDK API key generated per agent from the CLI. The skill stresses that without custom KPIs you only get basic token counts, so each agent should define a few KPIs that answer how you know it is performing well. KPIs belong to one agent and cannot be shared, so every later agent needs its own.

It explains the data path: `customData` sent from the SDK is matched against a CustomDataConfig schema, its fields become context variables, KPI formulas compute from them, and the results come back as `kpiData`. The rules it lists include that only registered CustomDataConfig fields can be used in formulas, formulas are case-insensitive, and NUMBER fields need real numbers rather than strings. Built-in variables such as `Prompt` are always available.

When your agent uses it

  • Starting a new agent that should report KPIs to Olakai from day one
  • Defining custom data fields and KPI formulas for an agent's events
  • Debugging why a KPI shows no value after events are sent

Example prompts

  • “Set up a new support-triage agent with Olakai monitoring and a resolution-rate KPI.”
  • “My KPI formula returns nothing even though I send stepCount in customData. Find out why.”
  • “Validate end to end that events from this agent reach Olakai and produce kpiData.”

Requirements

  • npm, to install the Olakai CLI
  • An Olakai login through the CLI
  • A per-agent API key generated with the CLI

Workflow steps

5 steps, taken from the step headings in SKILL.md.

  1. Design the Agent Architecture
  2. Configure Olakai Platform
  3. Implement SDK Integration
  4. Test-Validate-Iterate Cycle
  5. Production Checklist

What it can do on your machine

Read from SKILL.md and the folder at commit 67dcbeb. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • npm
    • jq
    • pip
    • curl

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use npm, pip and curl, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names these keys or tokens, usually read from environment variables:

    • OLAKAI_API_KEY
    • OPENAI_API_KEY

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Olakai Agent Project Setup loads about 5k tokens when it runs. Until then it costs about 37 tokens; SKILL.md has 1,084 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~37
When it runs · the whole SKILL.md, loaded when a task matches
~5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from andrewyng/context-hub at commit 67dcbeb, republished under its MIT licence (© andrewyng). 1,084 words, ~4,995 tokens.

Download SKILL.mdSave it as .claude/skills/new-project/SKILL.md (or your agent's skills folder).
name
new-project
description
Build a new AI agent with Olakai monitoring from scratch — project setup, SDK integration, KPI configuration, and end-to-end validation
metadata.revision
1
metadata.updated-on
2026-03-10
metadata.source
maintainer
metadata.tags
olakai,new-project,agent,monitoring,kpi,governance

Build a New AI Agent Project with Olakai

This skill guides you through creating a new AI agent that is fully integrated with Olakai for analytics, KPI tracking, and governance.

Prerequisites

Before starting, ensure:

  1. Olakai CLI installed: npm install -g olakai-cli
  2. CLI authenticated: olakai login
  3. API key for SDK (generated per-agent via CLI — see Step 2.2)

Why Custom KPIs Are Essential

Olakai's core value is tracking business-specific KPIs for your AI agents. Without KPIs, you're tracking events without gaining actionable insights.

What you can measure with KPIs:

  • Business outcomes (items processed, success rates, revenue impact)
  • Operational data (step counts, retry rates, execution time)
  • Quality indicators (error rates, user satisfaction signals)

Without KPIs configured:

  • No dashboard KPIs beyond basic token counts
  • No aggregated performance views
  • No alerting thresholds
  • No ROI calculations

Every agent should have 2-4 KPIs that answer: "How do I know this agent is performing well?"

KPIs created here belong to this specific agent only. If you later create additional agents, each one needs its own KPI definitions — KPIs cannot be shared or reused across agents.

Understanding the customData to KPI Pipeline

Before diving into implementation, understand how data flows through Olakai:

SDK customData → CustomDataConfig (Schema) → Context Variable → KPI Formula → kpiData
How It Works
  1. customData (SDK): Raw JSON you send with each event
  2. CustomDataConfig (Platform): Schema defining which fields are processed
  3. Context Variables: CustomDataConfig fields become available for formulas
  4. KPI Formula: Expression that computes a value (e.g., SuccessRate * 100)
  5. kpiData (Response): Computed KPI values returned with each event
Critical Rules
RuleConsequence
Only CustomDataConfig fields become variablesUnregistered customData fields are NOT usable in KPIs
Formula evaluation is case-insensitivestepCount, STEPCOUNT, StepCount all work in formulas
NUMBER configs need numeric valuesDon't send "5" (string), send 5 (number)
KPIs are unique per agentEach KPI belongs to exactly one agent — create separately for each
Built-in Context Variables (Always Available)
VariableTypeDescription
PromptstringThe prompt text sent to the LLM
ResponsestringThe LLM response text
Documents countnumberNumber of attached documents
PII detectedbooleanWhether PII was detected
PHI detectedbooleanWhether PHI was detected
CODE detectedbooleanWhether code was detected
SECRET detectedbooleanWhether secrets were detected

Step 1: Design the Agent Architecture

1.1 Determine Agent Type

Agentic AI (Multi-step autonomous workflows):

  • Research agents, document processors, data pipelines
  • Track as SINGLE events aggregating all internal LLM calls
  • Focus on workflow-level KPIs (total tokens, total time, success/failure)

Assistive AI (Interactive chatbots/copilots):

  • Customer support agents, coding assistants, Q&A systems
  • Track EACH interaction as separate events
  • Focus on conversation-level KPIs (per-message tokens, response quality)
1.2 Design Your KPI Schema (CRITICAL)

Design your KPIs BEFORE writing any SDK code. This ensures only meaningful data is sent and tracked.

Step A: Identify Business Questions

What do stakeholders need to know about this agent?

  • "How many items does it process per run?"
  • "What's the success/failure rate?"
  • "How efficient is each execution?"
Step B: Map Questions to Data Fields
Business QuestionField NameTypeKPI FormulaAggregation
ThroughputItemsProcessedNUMBERItemsProcessedSUM
ReliabilitySuccessRateNUMBERSuccessRate * 100AVERAGE
Error countSuccessRateNUMBERIF(SuccessRate < 1, 1, 0)SUM
CorrelationExecutionIdSTRING(for filtering only)-
Step C: Plan Your customData Structure
typescript
// ONLY include fields you'll register as CustomDataConfigs
customData: {
  // Business KPIs
  ItemsProcessed: number,  // Count of items handled
  SuccessRate: number,     // 0-1 success ratio

  // Performance KPIs
  StepCount: number,       // Number of workflow steps

  // Identification (for filtering, not KPIs)
  ExecutionId: string,     // Correlation ID
}

IMPORTANT: Only include fields you will register as CustomDataConfigs. Unregistered fields are stored but cannot be used in KPIs.

What NOT to Include in customData

The Olakai platform automatically tracks these — do NOT duplicate them:

Already TrackedWhereDon't Send As customData
Session IDMain payloadsessionId
Agent IDAPI key associationagentId
User emailuserEmail parameteremail, userEmail
TimestampEvent metadatatimestamp, createdAt
Request timerequestTime parameterduration, latency
Token counttokens parametertokenCount
ModelAuto-detectedmodel, modelName
ProviderClient configprovider

customData is ONLY for:

  1. KPI variables — Fields you'll use in formula calculations
  2. Tagging/filtering — Fields you'll filter by in queries

Step 2: Configure Olakai Platform

2.1 Create a Workflow (Required)

Every agent MUST belong to a workflow, even if it's the only agent in that workflow.

bash
olakai workflows create --name "Your Workflow Name" --json
# Output: { "id": "wfl_xxx...", "name": "Your Workflow Name" }
Show full SKILL.md (427 more words)Show less
2.2 Create the Agent in Olakai
bash
olakai agents create \
  --name "Your Agent Name" \
  --description "What this agent does" \
  --workflow WORKFLOW_ID \
  --with-api-key \
  --json

# Returns agent details including apiKey:
# {
#   "id": "cmkbteqn501kyjy4yu6p6xrrx",
#   "name": "Your Agent Name",
#   "workflowId": "wfl_xxx...",
#   "apiKey": "sk_agent_xxxxx..."   <-- Use this in your SDK
# }

Agent-Workflow Hierarchy:

Workflow: "Customer Support Pipeline"
├── Agent: "Ticket Classifier"
├── Agent: "Response Generator"
└── Agent: "Quality Checker"

Workflow: "Document Processing"
└── Agent: "Document Summarizer"  ← single-agent workflows are valid
2.3 Create Custom Data Configurations (BEFORE Writing SDK Code)

This step MUST be completed before Step 3 (SDK Integration). Only fields registered here can be used in KPI formulas.

ONLY create configs for data you'll use in KPIs or for filtering. Don't create configs for data already tracked automatically.

bash
# For numeric fields (can be used in KPI calculations)
olakai custom-data create --agent-id YOUR_AGENT_ID --name "ItemsProcessed" --type NUMBER
olakai custom-data create --agent-id YOUR_AGENT_ID --name "SuccessRate" --type NUMBER
olakai custom-data create --agent-id YOUR_AGENT_ID --name "StepCount" --type NUMBER

# For string fields (for filtering/grouping, not calculations)
olakai custom-data create --agent-id YOUR_AGENT_ID --name "ExecutionId" --type STRING

# Verify all configs are created
olakai custom-data list --agent-id YOUR_AGENT_ID
2.4 Create KPI Definitions
Quick Start with Templates

Instead of writing formulas from scratch, use predefined classifier templates:

bash
# List available templates
olakai kpis templates

# Create a classifier KPI from a template
olakai kpis create --name "User Satisfaction" \
  --calculator-id classifier --template-id sentiment_scorer \
  --scope CHAT --agent-id $AGENT_ID

# Create a time-saved estimator
olakai kpis create --name "Time Saved" \
  --calculator-id classifier --template-id time_saved_estimator \
  --scope CHAT --agent-id $AGENT_ID
Custom Formula KPIs
bash
# Variable passthrough
olakai kpis create \
  --name "Items Processed" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "ItemsProcessed" \
  --unit "items" \
  --aggregation SUM

# Percentage calculation
olakai kpis create \
  --name "Success Rate" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "SuccessRate * 100" \
  --unit "%" \
  --aggregation AVERAGE

# Conditional counting
olakai kpis create \
  --name "Error Count" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "IF(SuccessRate < 1, 1, 0)" \
  --unit "errors" \
  --aggregation SUM

# Validate formulas before creating
olakai kpis validate --formula "ItemsProcessed" --agent-id YOUR_AGENT_ID

Step 3: Implement SDK Integration

3.1 TypeScript Implementation

Install dependencies:

bash
npm install @olakai/sdk openai

Initialize and track:

typescript
import { olakaiConfig, olakai } from "@olakai/sdk";
import OpenAI from "openai";

// Initialize Olakai
olakaiConfig({
  apiKey: process.env.OLAKAI_API_KEY!,
  debug: process.env.NODE_ENV === "development",
});

// Create LLM client
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Use wrapped client — monitoring happens automatically
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userPrompt }],
});

Agentic workflow with manual event tracking:

taskExecutionId — Cross-Agent Task Correlation. Generate ONE taskExecutionId per task and share it across all agents in a multi-agent workflow. This links events from different agents into a single logical task for analytics.

typescript
async function runAgent(input: string): Promise<string> {
  const startTime = Date.now();
  const executionId = crypto.randomUUID();
  const taskExecutionId = crypto.randomUUID();
  let totalTokens = 0;
  let stepCount = 0;
  let itemsProcessed = 0;

  try {
    // Step 1: Planning
    stepCount++;
    const plan = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: `Plan: ${input}` }],
    });
    totalTokens += plan.usage?.total_tokens ?? 0;

    // Step 2: Process items
    const items = parseItems(plan.choices[0].message.content);
    for (const item of items) {
      stepCount++;
      const result = await openai.chat.completions.create({
        model: "gpt-4o",
        messages: [{ role: "user", content: `Process: ${item}` }],
      });
      totalTokens += result.usage?.total_tokens ?? 0;
      itemsProcessed++;
    }

    // Step 3: Summarize
    stepCount++;
    const summary = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: "Summarize results" }],
    });
    totalTokens += summary.usage?.total_tokens ?? 0;

    const finalResponse = summary.choices[0].message.content ?? "";

    // Track the complete workflow as a single event
    // Only send fields that have CustomDataConfigs (from Step 2.3)
    olakai("event", "ai_activity", {
      prompt: input,
      response: finalResponse,
      tokens: totalTokens,
      requestTime: Date.now() - startTime,
      taskExecutionId,
      task: "Data Processing & Analysis",
      customData: {
        ExecutionId: executionId,
        StepCount: stepCount,
        ItemsProcessed: itemsProcessed,
        SuccessRate: 1.0,
      },
    });

    return finalResponse;
  } catch (error) {
    // Track failed execution — same fields, different values
    olakai("event", "ai_activity", {
      prompt: input,
      response: `Error: ${error instanceof Error ? error.message : "Unknown"}`,
      tokens: totalTokens,
      requestTime: Date.now() - startTime,
      taskExecutionId,
      task: "Data Processing & Analysis",
      customData: {
        ExecutionId: executionId,
        StepCount: stepCount,
        ItemsProcessed: itemsProcessed,
        SuccessRate: 0,
      },
    });
    throw error;
  }
}
3.2 Python Implementation

Install dependencies:

bash
pip install olakai-sdk openai

Initialize and track:

python
import os
from olakaisdk import olakai_config, olakai, OlakaiEventParams
from openai import OpenAI

# Initialize Olakai
olakai_config(os.getenv("OLAKAI_API_KEY"))

# Create OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

Agentic workflow:

python
import time
import uuid

def run_agent(input_text: str) -> str:
    start_time = time.time()
    execution_id = str(uuid.uuid4())
    task_execution_id = str(uuid.uuid4())
    total_tokens = 0
    step_count = 0
    items_processed = 0

    try:
        # Your workflow steps here...
        step_count += 1
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": input_text}]
        )
        total_tokens += response.usage.total_tokens
        final_response = response.choices[0].message.content

        # Track successful execution
        olakai("event", "ai_activity", OlakaiEventParams(
            prompt=input_text,
            response=final_response,
            tokens=total_tokens,
            requestTime=int((time.time() - start_time) * 1000),
            taskExecutionId=task_execution_id,
            task="Data Processing & Analysis",
            customData={
                "ExecutionId": execution_id,
                "StepCount": step_count,
                "ItemsProcessed": items_processed,
                "SuccessRate": 1.0,
            }
        ))

        return final_response

    except Exception as e:
        olakai("event", "ai_activity", OlakaiEventParams(
            prompt=input_text,
            response=f"Error: {str(e)}",
            tokens=total_tokens,
            requestTime=int((time.time() - start_time) * 1000),
            taskExecutionId=task_execution_id,
            task="Data Processing & Analysis",
            customData={
                "ExecutionId": execution_id,
                "StepCount": step_count,
                "ItemsProcessed": items_processed,
                "SuccessRate": 0,
            }
        ))
        raise
3.3 REST API Direct Integration

For other languages or custom integrations:

bash
curl -X POST "https://app.olakai.ai/api/monitoring/prompt" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "prompt": "User input here",
    "response": "Agent response here",
    "app": "your-agent-name",
    "task": "Data Processing & Analysis",
    "tokens": 1500,
    "requestTime": 5000,
    "customData": {
      "ExecutionId": "abc-123",
      "StepCount": 5,
      "ItemsProcessed": 10,
      "SuccessRate": 1.0
    }
  }'

Step 4: Test-Validate-Iterate Cycle

Always validate your implementation by running a test and inspecting the actual event data.

4.1 Run Your Agent

Execute your agent with test data to generate at least one event.

4.2 Fetch and Inspect the Event
bash
olakai activity list --agent-id YOUR_AGENT_ID --limit 1 --json
olakai activity get EVENT_ID --json
4.3 Validate Each Component

Check customData is present and correct:

bash
olakai activity get EVENT_ID --json | jq '.customData'

Check KPIs are numeric (not strings):

bash
olakai activity get EVENT_ID --json | jq '.kpiData'

CORRECT — numeric values:

json
{
  "Items Processed": 10,
  "Success Rate": 100
}

WRONG — string values (broken formula):

json
{
  "Items Processed": "itemsProcessed"
}

Fix: olakai kpis update KPI_ID --formula "YourVariable"

WRONG — null values: Fix by verifying:

  1. CustomDataConfig exists: olakai custom-data list --agent-id ID
  2. Field name case matches exactly (case-sensitive)
  3. SDK actually sends the field in customData
4.4 Validation Flow
1. Run agent (generate event)
           ↓
2. Fetch event: olakai activity get ID --json
           ↓
3. Check customData present? NO → Fix SDK code
           ↓
4. Check kpiData numeric? NO → Fix formula
           ↓
5. Check kpiData not null? NO → Create CustomDataConfig or fix field name
           ↓
✅ All validations pass — implementation complete

Step 5: Production Checklist

Before deploying to production:

  • API key stored securely in environment variables
  • Error handling wraps all LLM calls
  • Failed executions still report events (with SuccessRate: 0)
  • All custom data fields have corresponding CustomDataConfig entries
  • KPI formulas validated and showing numeric values (not strings)
  • SDK configured with appropriate retries and timeouts
  • Sensitive data redaction enabled if needed

KPI Formula Reference

Supported Operators
CategoryOperators
Arithmetic+, -, *, /
Comparison<, <=, =, <>, >=, >
LogicalAND, OR, NOT
ConditionalIF(condition, true_val, false_val), MAP(value, match1, out1, default)
MathABS, MAX, MIN, AVERAGE, TRUNC
Null handlingISNA(value), ISDEFINED(value), NA()
Common Formula Patterns
bash
--formula "ItemsProcessed"                              # passthrough
--formula "SuccessRate * 100"                            # percentage (0-1 to 0-100)
--formula "IF(SuccessRate < 1, 1, 0)"                    # conditional counting
--formula "IF(PII detected, 1, 0)"                       # built-in variable
--formula "IF(ISDEFINED(MyField), MyField, 0)"           # null-safe
--formula "IF(AND(StepCount > 5, SuccessRate < 0.9), 1, 0)"  # compound conditions
Aggregation Types
AggregationUse ForExample
SUMTotals, countsTotal items processed across all runs
AVERAGERates, percentagesAverage success rate

Task Categories Reference

Use these predefined task categories for the task field:

CategoryExample Use
Research & IntelligenceCompetitive intelligence, market research
Data Processing & AnalysisData extraction, statistical analysis
Content DevelopmentBlog writing, technical documentation
Content RefinementEditing, proofreading
Customer ExperienceComplaint resolution, ticket triage
Software DevelopmentCode generation, code review, debugging
Strategic PlanningRoadmap development, scenario planning

Quick Reference

bash
# CLI Commands
olakai login                                              # Authenticate
olakai workflows create --name "Name" --json              # Create workflow
olakai agents create --name "Name" --workflow ID --with-api-key  # Register agent
olakai custom-data create --agent-id ID --name X --type NUMBER   # Create custom field
olakai kpis create --formula "X" --agent-id ID            # Create KPI
olakai activity list --agent-id ID                        # View events
typescript
// TypeScript SDK
import { olakaiConfig, olakai } from "@olakai/sdk";
olakaiConfig({ apiKey: process.env.OLAKAI_API_KEY });

olakai("event", "ai_activity", {
  prompt: "input",
  response: "output",
  tokens: 1500,
  task: "Data Processing & Analysis",
  customData: { StepCount: 3, Success: 1 },
});
python
# Python SDK
from olakaisdk import olakai_config, olakai, OlakaiEventParams
olakai_config(os.getenv("OLAKAI_API_KEY"))

olakai("event", "ai_activity", OlakaiEventParams(
    prompt="input",
    response="output",
    tokens=1500,
    task="Data Processing & Analysis",
    customData={"StepCount": 3, "Success": 1},
))

© andrewyng, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in content/olakai/skills/new-project of andrewyng/context-hub.

Open the folder on GitHubat commit 67dcbeb

Compare with similar skills

Olakai Agent Project Setup next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Olakai Agent Project Setup compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Olakai Agent Project Setup this skillandrewyng/context-hub14k—~5kAutomated safety check: PassMIT
LangSmith Trace DebuggingComposioHQ/awesome-claude-skills77k8 repos~2.7kAutomated safety check: PassNone
Failproof AI SDK IntegrationFailproofAI/failproofai5.3k—~6kAutomated safety check: PassCustom licence
OmniRoute Cost and Usage CLIdiegosouzapw/OmniRoute75k—~693Automated safety check: PassMIT
Dev BumpNetis/heron102—~983Automated safety check: PassApache-2.0
Langgraph Project Setupsoba-labs/langchain-agent-skills107—~2.4kAutomated safety check: NotesMIT

Similar skills

  • LangSmith Trace Debugging

    ComposioHQ/awesome-claude-skills

    Debugs LangChain and LangGraph agents by pulling recent execution traces with the langsmith-fetch CLI and reporting errors, tool calls, timings and token use.

    77k GitHub starsUsed in 8 repos~2.7k tokens
    AI & LLM EngineeringAuto-check passed
  • Failproof AI SDK Integration

    FailproofAI/failproofai

    Helps instrument a custom Python or TypeScript agent to record events for Failproof AI, verify what gets written, and run an evaluator worker that scores the runs.

    5.3k GitHub stars~6k tokensUpdated 2 days ago
    AI & LLM EngineeringAuto-check passed
  • OmniRoute Cost and Usage CLI

    diegosouzapw/OmniRoute

    View cost breakdowns, token usage, and call logs from the CLI. Filter by provider, model, or date range. Export usage reports and inspect per-connection…

    75k GitHub stars~693 tokensUpdated today
    AI & LLM EngineeringAuto-check passed
  • Dev Bump

    Netis/heron

    Bump Heron version via the VERSION-file SSOT. An agent skill from Netis/heron.

    102 GitHub stars~983 tokensUpdated 5 days ago
    AI & LLM EngineeringAuto-check passed
  • Langgraph Project Setup

    soba-labs/langchain-agent-skills

    Initialize and configure LangGraph projects with proper structure, langgraph.json configuration, environment variables, and dependency management.

    107 GitHub stars~2.4k tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check: notes
  • Agentsop Observability Setup

    agentsope/SkillAlchemy

    Enhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover.

    436 GitHub stars~4.4k tokensUpdated yesterday
    AI & LLM EngineeringAuto-check passed

More from andrewyng/context-hub

All 8 skills in this repo
  • Get API Docs with chub

    andrewyng/context-hub

    Fetches current documentation for third-party APIs and SDKs with the chub CLI before the agent writes code against them, instead of relying on remembered API shapes.

    14k GitHub starsUsed in 1 repo~775 tokens
    Auto-check passed
  • Tavily Search API Integration

    andrewyng/context-hub

    Guides building Tavily integrations for web search, URL extraction, site crawling and AI-assisted research in Python or JavaScript agent and RAG projects.

    14k GitHub stars~1.1k tokensUpdated 4 mo ago
    Auto-check passed
  • Flutter Bloc and Cubit

    andrewyng/context-hub

    Helps choose between Bloc and Cubit in Flutter and sets defaults for packages, lifecycle, widget binding and tests.

    14k GitHub stars~998 tokensUpdated 4 mo ago
    Auto-check passed
  • Sequences parts-mcp tool calls for finding, pricing and checking availability of electronic components, including BOM processing and datasheet navigation.

    14k GitHub stars~1.8k tokensUpdated 4 mo ago
    Auto-check passed
  • Olakai Monitoring Integration

    andrewyng/context-hub

    Adds Olakai monitoring to an existing LLM application with minimal code changes, then configures custom KPIs so the dashboard tracks business outcomes instead of just token counts.

    14k GitHub stars~4.5k tokensUpdated 4 mo ago
    Auto-check passed
  • Playwright Login Flows

    andrewyng/context-hub

    Collects reusable Playwright patterns for logging in during end-to-end tests: password forms, OAuth redirects, saved browser state and TOTP two-factor codes.

    14k GitHub stars~715 tokensUpdated 4 mo ago
    Auto-check passed

Works with

Questions about Olakai Agent Project Setup

What does Olakai Agent Project Setup do?

Builds a new AI agent with Olakai monitoring from the start: CLI login, SDK integration, per-agent KPI configuration and an end-to-end check that data flows. Setup begins with the Olakai CLI installed through npm and authenticated with `olakai login`, plus an SDK API key generated per agent from the CLI. The skill stresses that without custom KPIs you only get basic token counts, so each agent should define a few KPIs that answer how you know it is performing well.

When should I use Olakai Agent Project Setup?

Olakai Agent Project Setup fits situations like: starting a new agent that should report KPIs to Olakai from day one; defining custom data fields and KPI formulas for an agent's events; debugging why a KPI shows no value after events are sent.

How do I install Olakai Agent Project Setup in Claude Code?

Run `npx skills add andrewyng/context-hub --skill new-project -a claude-code`. Or copy the skill folder (content/olakai/skills/new-project in andrewyng/context-hub) into .claude/skills/new-project in your project. Claude Code loads it when a task matches its description.

How do I install Olakai Agent Project Setup in Codex?

Run `npx skills add andrewyng/context-hub --skill new-project -a codex`. Or copy the skill folder (content/olakai/skills/new-project in andrewyng/context-hub) into .agents/skills/new-project in your project. Codex loads it when a task matches its description.

Can I use Olakai Agent Project Setup in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add andrewyng/context-hub --skill new-project -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/new-project, .gemini/skills/new-project, .github/skills/new-project and .opencode/skills/new-project in your project.

What does Olakai Agent Project Setup need to run?

Going by SKILL.md and its folder, Olakai Agent Project Setup needs the command-line tools its instructions call (npm, jq, pip and curl) and credentials named OLAKAI_API_KEY and OPENAI_API_KEY. Our summary lists: npm, to install the Olakai CLI; An Olakai login through the CLI; A per-agent API key generated with the CLI.

Does Olakai Agent Project Setup access the network?

SKILL.md contains no URLs. Its commands use npm, pip and curl, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Olakai Agent Project Setup safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Olakai Agent Project Setup use?

Olakai Agent Project Setup is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Olakai Agent Project Setup use?

About 5k tokens (SKILL.md is roughly 20k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Olakai Agent Project Setup?

Skills that share tags, products or a category with Olakai Agent Project Setup: LangSmith Trace Debugging (ComposioHQ/awesome-claude-skills, 77k stars), Failproof AI SDK Integration (FailproofAI/failproofai, 5.3k stars), OmniRoute Cost and Usage CLI (diegosouzapw/OmniRoute, 75k stars) and Dev Bump (Netis/heron, 102 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Olakai Agent Project Setup?

andrewyng (a GitHub user) maintains it in andrewyng/context-hub, which has 13,973 GitHub stars. The repository holds 8 skills in this directory. The repository was last updated on May 31, 2026.

Source: andrewyng/context-hub on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.