Agent Cost Optimizer

Agent Cost Optimizer

Overview

agent-cost-optimizer provides comprehensive cost tracking, budget enforcement, and ROI measurement for AI agent operations.

Purpose: Control and optimize AI spending while maximizing value delivered

Pattern: Task-based (7 operations for cost management)

Key Innovation: Real-time cost tracking with automatic budget enforcement and cost-effective fallbacks

Industry Context (2025):

• Average AI spending: $85,521/month (36% YoY increase)
• Only 50% of organizations can measure AI ROI
• IT teams struggle with hidden costs

Solution: Comprehensive cost management from tracking to optimization

When to Use

Use agent-cost-optimizer when:

• Tracking AI costs across skills
• Preventing budget overruns
• Measuring ROI (cost vs. value delivered)
• Optimizing model selection (Opus vs. Sonnet vs. Haiku)
• Planning AI budgets
• Cost-effective development
• Enterprise cost accountability

Prerequisites

Required

• AI API access (Anthropic, OpenAI, Google)
• Cost tracking capability (API usage data)

Optional

• Paid.ai or AgentOps integration (advanced cost tracking)
• Prometheus/Grafana (cost visualization)
• Budget approval workflow

Cost Operations

Operation 1: Track Token Usage

Purpose: Monitor token consumption per skill invocation

Process:

1. Initialize Tracking:

   {
     "tracking_id": "track_20250126_1200",
     "skill": "multi-ai-verification",
     "started_at": "2025-01-26T12:00:00Z",
     "tokens": {
       "prompt": 0,
       "completion": 0,
       "total": 0
     },
     "cost": {
       "amount_usd": 0.00,
       "model": "claude-sonnet-4-5"
     }
   }

2. Track During Execution:

   // After each AI call
   trackTokens({
     prompt_tokens: response.usage.input_tokens,
     completion_tokens: response.usage.output_tokens,
     model: 'claude-sonnet-4-5'
   });
   
   // Update running totals

3. Finalize Tracking:

   {
     "tracking_id": "track_20250126_1200",
     "skill": "multi-ai-verification",
     "completed_at": "2025-01-26T12:45:00Z",
     "duration_minutes": 45,
     "tokens": {
       "prompt": 15234,
       "completion": 8932,
       "total": 24166
     },
     "cost": {
       "amount_usd": 0.073,
       "model": "claude-sonnet-4-5",
       "rate": "$3 per million tokens"
     }
   }

4. Save to Cost Log:

   # Append to daily cost log
   cat tracking.json >> .cost-tracking/$(date +%Y-%m-%d).json

Outputs:

• Token usage per invocation
• Cost per invocation
• Model used
• Daily/monthly aggregates

Validation:

• Tracking initialized
• Tokens counted accurately
• Cost calculated correctly
• Logs saved

Time Estimate: Automatic (integrated into skills)

Operation 2: Calculate Costs

Purpose: Compute accurate costs based on provider pricing

Pricing (2025 rates):

Anthropic (Claude):

OpenAI (Codex):

Google (Gemini):

Process:

function calculateCost(usage, model) {
  const pricing = {
    'claude-sonnet-4-5': { input: 3, output: 15 },
    'claude-haiku-4-5': { input: 0.80, output: 4 },
    'claude-opus-4-5': { input: 15, output: 75 },
    // ... more models
  };

  const rates = pricing[model];
  const inputCost = (usage.prompt_tokens / 1_000_000) * rates.input;
  const outputCost = (usage.completion_tokens / 1_000_000) * rates.output;

  return {
    input_cost: inputCost,
    output_cost: outputCost,
    total_cost: inputCost + outputCost,
    currency: 'USD'
  };
}

Outputs:

• Accurate cost per invocation
• Model-specific pricing
• Input vs. output cost breakdown

Operation 3: Enforce Budget Caps

Purpose: Prevent exceeding monthly budget limits

Process:

1. Set Budget:

   {
     "monthly_budget_usd": 100,
     "skill_budgets": {
       "multi-ai-verification": 70,
       "multi-ai-research": 20,
       "multi-ai-testing": 10
     },
     "alert_thresholds": {
       "warning": 0.80,
       "critical": 0.95
     }
   }

2. Check Before Operation:

   async function checkBudget(skill, estimated_cost) {
     const usage = getCurrentMonthUsage();
     const remaining = budget.monthly_budget_usd - usage.total_cost;
   
     if (estimated_cost > remaining) {
       return {
         allowed: false,
         reason: `Budget exceeded: $${usage.total_cost}/$${budget.monthly_budget_usd}`,
         overage: estimated_cost - remaining
       };
     }
   
     if (usage.total_cost / budget.monthly_budget_usd > 0.80) {
       return {
         allowed: true,
         warning: `80% of monthly budget used ($${usage.total_cost}/$${budget.monthly_budget_usd})`
       };
     }
   
     return { allowed: true };
   }

3. Enforce:

   const budgetCheck = await checkBudget('multi-ai-verification', 0.50);
   
   if (!budgetCheck.allowed) {
     console.log(`❌ Budget exceeded: ${budgetCheck.reason}`);
     console.log(`Options:`);
     console.log(`  1. Use cheaper model (Sonnet → Haiku)`);
     console.log(`  2. Skip optional verification layers`);
     console.log(`  3. Request budget increase`);
     return;
   }
   
   if (budgetCheck.warning) {
     console.log(`⚠️ ${budgetCheck.warning}`);
   }
   
   // Proceed with operation

Outputs:

• Budget status checked
• Operations blocked if over budget
• Warnings at 80%
• Cost-effective alternatives suggested

Operation 4: Optimize Model Selection

Purpose: Choose cost-effective model for each task

Decision Matrix:

Auto-Optimization:

function selectModel(task_type, criticality, budget_remaining) {
  // Critical + budget OK → Use Opus
  if (criticality === 'critical' && budget_remaining > 20) {
    return 'claude-opus-4-5';
  }

  // Standard → Use Sonnet
  if (criticality === 'standard') {
    return 'claude-sonnet-4-5';
  }

  // Budget low or simple task → Use Haiku
  if (budget_remaining < 5 || task_type === 'simple') {
    return 'claude-haiku-4-5';
  }

  return 'claude-sonnet-4-5'; // Default
}

Outputs:

• Optimal model selected
• Cost minimized
• Quality maintained

Operation 5: Cost-Effective Caching

Purpose: Avoid re-computing identical operations

Process:

1. Cache Key Generation:

   function generateCacheKey(operation, inputs) {
     // Hash inputs to create unique key
     const content_hash = crypto
       .createHash('sha256')
       .update(JSON.stringify(inputs))
       .digest('hex');
   
     return `${operation}_${content_hash}`;
   }

2. Check Cache Before Operation:

   const cacheKey = generateCacheKey('verify_code', {
     files: ['src/auth.ts'],
     file_hashes: {'src/auth.ts': 'abc123'}
   });
   
   const cached = readCache(cacheKey);
   
   if (cached && !isExpired(cached, 24)) {
     // Use cached result
     console.log('📦 Using cached verification result');
     return cached.result;
   }
   
   // Cache miss → run verification
   const result = await runVerification();
   
   // Save to cache
   saveCache(cacheKey, result, ttl: 24 hours);

3. Cache Structure:

   {
     "cache_key": "verify_code_abc123def456",
     "created_at": "2025-01-26T12:00:00Z",
     "expires_at": "2025-01-27T12:00:00Z",
     "inputs": {
       "files": ["src/auth.ts"],
       "file_hashes": {"src/auth.ts": "abc123"}
     },
     "result": {
       "quality_score": 92,
       "layers_passed": 5,
       "issues": []
     },
     "cost_saved": 0.073
   }

Outputs:

• 90% reduction in re-verification costs
• Instant results for unchanged code
• Cache hit/miss tracking

Validation:

• Cache key correctly identifies identical operations
• File changes invalidate cache
• Expired cache not used
• Cost savings tracked

Operation 6: Measure ROI

Purpose: Calculate return on investment for AI spending

Process:

1. Track Time Saved:

   {
     "task": "Implement user authentication",
     "without_ai": {
       "estimated_hours": 40,
       "developer_rate": 100,
       "total_cost": 4000
     },
     "with_ai": {
       "actual_hours": 11.3,
       "developer_rate": 100,
       "developer_cost": 1130,
       "ai_cost": 2.50,
       "total_cost": 1132.50
     },
     "roi": {
       "time_saved_hours": 28.7,
       "cost_saved": 2867.50,
       "roi_percentage": 253,
       "payback_period_hours": 0.025
     }
   }

2. Calculate ROI:

   function calculateROI(task) {
     const time_saved = task.without_ai.estimated_hours - task.with_ai.actual_hours;
     const cost_saved = task.without_ai.total_cost - task.with_ai.total_cost;
     const roi_percentage = (cost_saved / task.with_ai.ai_cost) * 100;
   
     return {
       time_saved_hours: time_saved,
       cost_saved_usd: cost_saved,
       roi_percentage: roi_percentage,
       payback_period: task.with_ai.ai_cost / (task.without_ai.developer_rate * (time_saved / 40)) // weeks
     };
   }

3. Monthly ROI Report:

   # Monthly ROI Report - January 2025
   
   ## AI Spending
   - Total AI costs: $87.50
   - Breakdown:
     - multi-ai-verification: $62.30 (71%)
     - multi-ai-research: $18.40 (21%)
     - multi-ai-testing: $6.80 (8%)
   
   ## Time Savings
   - Tasks completed: 8
   - Total hours saved: 156 hours
   - Average savings per task: 19.5 hours
   
   ## Cost Savings
   - Developer cost avoided: $15,600 (156h × $100/h)
   - AI costs: $87.50
   - Net savings: $15,512.50
   
   ## ROI
   - ROI: 17,728% ($177 saved per $1 spent)
   - Payback period: 0.03 weeks (immediate)
   - Value multiplier: 178x
   
   ## Recommendations
   - ✅ Current spending highly cost-effective
   - Continue using AI for all qualifying tasks
   - Consider increasing budget (high ROI)

Outputs:

• Comprehensive ROI analysis
• Time and cost savings quantified
• Monthly reports
• Business justification for AI spending