Self Improving Agent Skill

Self-Improving Agent

"An AI agent that learns from every interaction, accumulating patterns and insights to continuously improve its own capabilities." — Based on 2025 lifelong learning research

Overview

This is a universal self-improvement system that learns from ALL task experiences. It implements a complete feedback loop:

• Multi-Memory Architecture: Semantic (patterns/rules) + Episodic (experiences) + Working (session context)
• Self-Correction: Detects and fixes guidance errors
• Self-Validation: Periodically verifies skill accuracy
• Evolution Markers: Traceable changes with source attribution
• Confidence Tracking: Measures pattern reliability over time
• User Confirmation Gate: All skill file modifications require explicit user approval before applying
• Human-in-the-Loop: Collects feedback to validate improvements

Research-Based Design

The Self-Improvement Loop

┌──────────────────────────────────────────────────────────────┐
│                  UNIVERSAL SELF-IMPROVEMENT                   │
├──────────────────────────────────────────────────────────────┤
│                                                              │
│  Task Event → Extract Experience → Abstract Pattern → Update │
│       │               │                 │              │     │
│       ▼               ▼                 ▼              ▼     │
│  ┌────────────────────────────────────────────────────────┐  │
│  │              MULTI-MEMORY SYSTEM                       │  │
│  ├────────────────────────────────────────────────────────┤  │
│  │ Semantic Memory  │ Episodic Memory  │ Working Memory   │  │
│  │ (Patterns/Rules) │ (Experiences)    │ (Current)        │  │
│  │ memory/self-improving/semantic/ │ memory/self-improving/episodic/ │ memory/self-improving/working/  │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │              FEEDBACK LOOP                             │  │
│  │ User Feedback → Confidence Update → Pattern Adapt      │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
└──────────────────────────────────────────────────────────────┘

When This Activates

Automatic Triggers

Manual Triggers

• User says "自我进化", "self-improve", "从经验中学习"
• User says "分析今天的经验", "总结教训", "总结经验"
• User asks to improve a specific skill or workflow

Memory Storage

Workspace Discovery

Before accessing any memory files, the agent MUST first determine the workspace root path:

1. Check environment — Use the workspace path provided by the IDE/environment context
2. Verify structure — Confirm the workspace root by checking for project markers (e.g., .git/, package.json, pom.xml, etc.)
3. All paths below are relative to the workspace root — e.g., {workspace}/memory/self-improving/

Relationship with Agent Memory

The Self-Improving Agent's memory lives inside the Agent's memory/ directory as a dedicated subdirectory. This design ensures:

• No confusion: Agent's own memory (MEMORY.md, memory/YYYY-MM-DD.md) and Self-Improving Agent's memory (memory/self-improving/) are clearly separated by directory structure
• Discoverability: The Agent can browse memory/ and naturally find self-improving insights
• Supplement, not replace: Self-Improving Agent can append high-confidence patterns to Agent's memory files (with user confirmation), enriching the Agent's knowledge

{workspace}/
├── MEMORY.md                          # Agent core memory (Self-Improving Agent can append)
├── memory/
│   ├── YYYY-MM-DD.md                  # Agent daily memory (Self-Improving Agent can append)
│   └── self-improving/                # Self-Improving Agent dedicated memory space
│       ├── semantic/
│       │   └── patterns.json          # Abstract patterns and rules
│       ├── episodic/
│       │   └── YYYY/
│       │       └── YYYY-MM-DD-{task}.json  # Specific experiences
│       ├── working/
│       │   ├── current_session.json   # Active session data
│       │   ├── last_error.json        # Error context for self-correction
│       │   └── session_end.json       # Session end marker for consolidation
│       └── index.json                 # Memory index and metrics

Memory Interaction Rules

Evolution Priority Matrix

Trigger evolution when new reusable knowledge appears:

Multi-Memory Architecture

1. Semantic Memory (memory/self-improving/semantic/patterns.json)

Stores abstract patterns and rules reusable across contexts:

{
  "patterns": {
    "pat-2025-01-11-001": {
      "id": "pat-2025-01-11-001",
      "name": "Pattern Name",
      "source": "user_feedback|implementation_review|retrospective",
      "confidence": 0.95,
      "applications": 5,
      "created": "2025-01-11",
      "last_applied": "2025-01-15",
      "category": "coding_patterns|architecture|debugging|workflow|...",
      "pattern": "One-line summary",
      "problem": "What problem does this solve?",
      "solution": "How to apply this pattern",
      "quality_rules": ["Rule 1", "Rule 2"],
      "target_skills": ["skill-name-1", "skill-name-2"]
    }
  }
}

2. Episodic Memory (memory/self-improving/episodic/)

Stores specific experiences and what happened:

{
  "id": "ep-2025-01-11-001",
  "timestamp": "2025-01-11T10:30:00Z",
  "skill": "debugger|coding-assistant|reviewer|...",
  "task_type": "debugging|coding|review|design|...",
  "situation": "What the user was trying to do",
  "solution": "How the issue was resolved",
  "outcome": "success|partial|failure",
  "root_cause": "Underlying issue if applicable",
  "lesson": "Key takeaway from this experience",
  "related_pattern": "pattern_id if linked",
  "user_feedback": {
    "rating": 8,
    "comments": "User's feedback on the experience"
  }
}

3. Working Memory (memory/self-improving/working/)

Stores current session context — ephemeral data that gets consolidated at session end:

{
  "session_id": "session-2025-01-11-001",
  "started": "2025-01-11T10:00:00Z",
  "tasks_completed": [],
  "errors_encountered": [],
  "patterns_applied": [],
  "pending_extractions": []
}

Self-Improvement Process

Phase 1: Experience Extraction

After any significant task completes, extract:

What happened:
  task_type: {what kind of task}
  task: {what was being done}
  outcome: {success|partial|failure}

Key Insights:
  what_went_well: [what worked]
  what_went_wrong: [what didn't work]
  root_cause: {underlying issue if applicable}

User Feedback:
  rating: {1-10 if provided}
  comments: {specific feedback}

Phase 2: Pattern Abstraction

Convert experiences to reusable patterns. The goal is to go from concrete to abstract — patterns should be general enough to apply across different tasks but specific enough to be actionable.

Abstraction Rules:

If experience_repeats 3+ times:
  pattern_level: critical
  action: Add to "Critical Mistakes" or "Anti-Patterns" section

If solution_was_effective:
  pattern_level: best_practice
  action: Add to "Best Practices" section

If user_rating >= 7:
  pattern_level: strength
  action: Reinforce this approach in relevant skills

If user_rating <= 4:
  pattern_level: weakness
  action: Add to "What to Avoid" section

Phase 3: Skill Updates

IMPORTANT: User Confirmation Required — Before writing any changes to skill files, you MUST:

1. Present proposed changes — Show the user a clear summary of what will be modified:
   • Which skill file(s) will be updated
   • What content will be added, modified, or removed
   • The rationale behind each change (source episode, pattern, confidence level)
2. Wait for explicit approval — Do NOT proceed until the user confirms. Acceptable confirmations include explicit affirmative responses (e.g., "确认", "好的", "proceed", "yes").
3. Apply changes only after approval — Once confirmed, apply the changes with evolution markers for traceability.

If the user rejects or requests modifications, adjust the proposed changes accordingly and re-present for confirmation.

Proposed Change Summary Format:

## Proposed Skill Update

**Target**: `{skill-file-path}`
**Action**: {Add new pattern | Correct existing guidance | Update checklist}
**Source**: {episode_id or trigger}
**Confidence**: {X.XX}

### Changes Preview
{Show the exact content that will be added/modified, using diff-style or before/after format}

### Rationale
{Why this change is recommended}

---
Confirm this update? (yes/no/modify)

Once confirmed, update skill files with evolution markers for traceability:

<!-- Evolution: 2025-01-12 | source: ep-2025-01-12-001 | task: debugging -->

## Pattern Added (2025-01-12)

**Pattern**: Always verify callbacks are not empty functions

**Source**: Episode ep-2025-01-12-001

**Confidence**: 0.95

### Updated Checklist
- [ ] Verify all callbacks have implementations
- [ ] Test callback execution paths

Correction Markers (when fixing wrong guidance):

<!-- Correction: 2025-01-12 | was: "Use callback chain" | reason: caused stale state -->

## Corrected Guidance

Use direct state monitoring instead of callback chains for reactive updates.

Use the templates in templates/ for consistent formatting. See references/appendix.md for the full template structures.

Phase 4: Memory Consolidation

1. Update semantic memory — add or update patterns in memory/self-improving/semantic/patterns.json

2. Store episodic memory — write episode to memory/self-improving/episodic/YYYY/YYYY-MM-DD-{task}.json

3. Update pattern confidence — increase confidence for patterns that were successfully applied, decrease for those that led to errors

4. Prune outdated patterns — lower confidence for patterns with no recent applications; archive patterns below 0.3 confidence

5. Supplement Agent memory — propose additions to Agent's own memory files. User confirmation is REQUIRED before any write to MEMORY.md or memory/YYYY-MM-DD.md. Follow the same confirmation protocol as Phase 3:

   What to propose:

   • High-confidence patterns (>= 0.9) as concise entries → MEMORY.md
   • Today's session summary and key learnings → memory/YYYY-MM-DD.md

   Confirmation format:

   ## Proposed Agent Memory Update
   
   ### → MEMORY.md (append)
   {Exact content to be appended, preview here}
   
   ### → memory/YYYY-MM-DD.md (append)
   {Exact content to be appended, preview here}
   
   **Source patterns**: {pattern IDs and confidence levels}
   
   ---
   Confirm this memory update? (yes/no/modify)

   After approval:

   • Append confirmed content with <!-- Source: self-improving-agent | date: YYYY-MM-DD --> markers for traceability
   • Do NOT overwrite existing content — always append at the end