Copilot Log Analysis

Copilot Log Analysis Skill

This skill documents the methods and approaches used by the AI Engineering Fluency extension to analyze Copilot session log files. These files contain chat sessions, token usage, and model information.

Overview

The extension analyzes two types of log files:

• .json files: Standard VS Code Copilot Chat session files
• .jsonl files: Copilot CLI/Agent mode sessions, JetBrains IDE Copilot Chat sessions, Claude Code sessions, Gemini CLI sessions, Antigravity sessions, and other ecosystem adapters (one JSON event per line)

Session File Discovery

Key Method: getCopilotSessionFiles()

Location: src/extension.ts (via SessionDiscovery) Helper Methods: getVSCodeUserPaths(), scanDirectoryForSessionFiles()

This method discovers session files across all VS Code variants and locations:

Supported VS Code Variants:

• VS Code (Stable)
• VS Code Insiders
• VS Code Exploration
• VSCodium
• Cursor
• VS Code Server/Remote

File Locations Checked:

1. Workspace Storage: {VSCode User Path}/workspaceStorage/{workspace-id}/chatSessions/*.json
2. Global Storage (Legacy): {VSCode User Path}/globalStorage/emptyWindowChatSessions/*.json
3. Copilot Chat Extension Storage: {VSCode User Path}/globalStorage/github.copilot-chat/**/*.json
4. Copilot CLI Sessions: ~/.copilot/session-state/*.jsonl
5. JetBrains IDE Copilot Sessions: ~/.copilot/jb/{conversationId}/partition-{n}.jsonl
6. Gemini CLI Sessions: ~/.gemini/tmp/{project}/chats/session-*.jsonl
7. Antigravity Sessions: ~/.gemini/antigravity/brain/{session-uuid}/.system_generated/logs/transcript.jsonl

Platform-Specific Paths:

• Windows: %APPDATA%/{Variant}/User
• macOS: ~/Library/Application Support/{Variant}/User
• Linux: ~/.config/{Variant}/User (respects XDG_CONFIG_HOME)
• Remote/Server: ~/.vscode-server/data/User, ~/.vscode-server-insiders/data/User

Helper Method: getVSCodeUserPaths()

Location: src/extension.ts

Returns all possible VS Code user data paths for different variants and platforms.

Helper Method: scanDirectoryForSessionFiles()

Location: src/extension.ts

Recursively scans directories for .json and .jsonl session files.

Field Extraction Methods

Parsing and Token Accounting: parseSessionFileContent()

Location: src/sessionParser.ts

Purpose: Parses session files and returns tokens, interactions, model usage, and editor type-safe model IDs.

How it works:

1. Accepts raw file content along with callbacks for token estimation and model detection.
2. Supports both .json (Copilot Chat) and .jsonl (CLI/agent) formats, including delta-based JSONL streams.
3. Counts interactions (user messages), input tokens, and output tokens while grouping by model.
4. Uses estimateTokensFromText() (in src/extension.ts) for character-to-token estimation.

Model Detection Logic: getModelFromRequest()

Location: src/extension.ts

• Primary: request.result.metadata.modelId
• Fallback: parses request.result.details for known model patterns
• Detected patterns: GPT-3.5-Turbo, GPT-4 family (4, 4.1, 4o, 4o-mini, 5, o3-mini, o4-mini), Claude Sonnet (3.5, 3.7, 4), Gemini (2.5 Pro, 3 Pro, 3 Pro Preview); defaults to gpt-4
• Display name mapping in getModelDisplayName() adds variants such as GPT-5 family, Claude Haiku, Claude Opus, Gemini 3 Flash, Grok, and Raptor when present in metadata.modelId.

Editor Type Detection: getEditorTypeFromPath()

Location: src/extension.ts

Purpose: Determines which VS Code variant created the session file.

Detection patterns:

• Contains /.copilot/jb/ → 'JetBrains' (must be checked BEFORE the Copilot CLI rule below since both live under ~/.copilot/)
• Contains /.copilot/session-state/ → 'Copilot CLI'
• Contains /code - insiders/ → 'VS Code Insiders'
• Contains /code - exploration/ → 'VS Code Exploration'
• Contains /vscodium/ → 'VSCodium'
• Contains /cursor/ → 'Cursor'
• Contains .vscode-server-insiders/ → 'VS Code Server (Insiders)'
• Contains .vscode-server/ → 'VS Code Server'
• Contains /code/ → 'VS Code'
• Default → 'Unknown'

Token Estimation Algorithm

Character-to-Token Conversion: estimateTokensFromText()

Location: src/extension.ts

Approach: Uses model-specific character-to-token ratios

• Default ratio: 0.25 (4 characters per token)
• Model-specific ratios loaded from src/tokenEstimators.json
• Formula: Math.ceil(text.length * tokensPerChar)

Model matching:

• Checks if model name includes the key from tokenEstimators
• Example: gpt-4o matches key gpt-4o

Caching Strategy

Cache Structure: SessionFileCache

Location: src/extension.ts

Stores pre-calculated tokens, interactions, model usage, and file mtime to avoid re-processing unchanged files.

Cache Methods:

• isCacheValid(): Validates cached entry by mtime
• getCachedSessionData(): Retrieves cached data
• setCachedSessionData(): Stores data with FIFO eviction after 1000 files
• clearExpiredCache(): Drops cache entries for missing files
• getSessionFileDataCached(): Reads session content, parses via parseSessionFileContent(), and caches results

Schema Documentation

Schema Files Location

Directory: docs/logFilesSchema/

Key files:

1. session-file-schema.json: Manual curated schema with descriptions
2. session-file-schema-analysis.json: Auto-generated field discovery (generated by PowerShell script)
3. README.md: Complete guide for schema analysis
4. SCHEMA-ANALYSIS.md: Quick reference guide
5. VSCODE-VARIANTS.md: VS Code variant detection documentation

Note: The analysis JSON file is auto-generated and may not exist in fresh clones. It is created by running the schema analysis script documented below.

Schema Analysis

See the Executable Scripts section for available utilities:

1. get-session-files.js - Quick session file discovery
2. diagnose-session-files.js - Detailed diagnostics
3. analyze-session-schema.ps1 - PowerShell schema analysis

JSON File Structure (VS Code Sessions)

Primary fields used by extension:

{
  "requests": [
    {
      "message": {
        "parts": [
          { "text": "user message content" }
        ]
      },
      "response": [
        { "value": "assistant response content" }
      ],
      "result": {
        "metadata": {
          "modelId": "gpt-4o"
        },
        "details": "Used GPT-4o model"
      }
    }
  ]
}

Key paths:

• Input tokens: requests[].message.parts[].text
• Output tokens: requests[].response[].value
• Model ID: requests[].result.metadata.modelId
• Model details: requests[].result.details
• Interaction count: requests.length

JSONL File Structure (Copilot CLI)

Event types:

{"type": "user.message", "data": {"content": "..."}, "model": "gpt-4o"}
{"type": "assistant.message", "data": {"content": "..."}}
{"type": "tool.execution_start", "data": {"toolCallId": "...", "toolName": "...", "arguments": {}}}
{"type": "tool.execution_complete", "data": {"toolCallId": "...", "success": true, "result": {"content": "...", "detailedContent": "..."}}}
{"type": "session.shutdown", "data": {"modelMetrics": {"claude-opus-4.6": {"usage": {"inputTokens": 0, "outputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0}}}}}

Key fields:

• Event type: type
• User input: data.content (when type: 'user.message')
• Assistant output: data.content (when type: 'assistant.message')
• Tool output: data.result.content or data.result.detailedContent (when type: 'tool.execution_complete')
• Tool name for counting: data.toolName (when type: 'tool.execution_start')
• API-accurate token counts: data.modelMetrics[model].usage (when type: 'session.shutdown') — always prefer these over estimates when available
• Model: model (optional, defaults to gpt-4o)

JSONL File Structure (JetBrains IDE)

Location: ~/.copilot/jb/{conversationId}/partition-{n}.jsonl — one UUID-named directory per conversation, one or more partition files per conversation. Empty partition files are skipped.

Full schema documentation: docs/logFilesSchema/jetbrains-session-schema.json

Common envelope — every line is { type, data, id, timestamp, parentId }.

Event types:

{"type":"partition.created","data":{"conversationId":"...","partitionId":1,"source":"panel","createdAt":1777552130660}}
{"type":"user.message","data":{"content":"...","turnId":"..."}}
{"type":"user.message_rendered","data":{"turnId":"...","renderedMessage":"<context>...</context><reminderInstructions>You are an agent...</reminderInstructions><userRequest>...</userRequest>"}}
{"type":"assistant.turn_start","data":{"turnId":"..."}}
{"type":"assistant.message","data":{"text":"...","thinking":{"text":"..."},"iterationNumber":1,"messageId":"..."}}
{"type":"tool.execution_start","data":{"toolCallId":"toolu_bdrk_...","toolName":"read_file","arguments":{...}}}
{"type":"tool.execution_complete","data":{"toolCallId":"...","success":true,"result":{"result":[{"type":"text","value":"..."}]}}}
{"type":"assistant.turn_end","data":{"turnId":"...","status":"success"}}

Key extraction rules (also implemented by parseJetBrainsPartition in src/jetbrains.ts):

• Interactions: count of user.message events
• Input tokens: estimated from user.message_rendered.data.renderedMessage (falls back to user.message.data.content)
• Output tokens: estimated from assistant.message.data.text
• Thinking tokens: estimated from assistant.message.data.thinking.text
• Actual tokens: always 0 — JetBrains does not record API-side usage counts in the session file
• Mode: presence of any tool.execution_start event ⇒ agent, otherwise ask (no edit/plan/customAgent)
• Model: not in the file. Best-effort heuristic from toolCallId prefix (toolu_* ⇒ Anthropic Claude, call_* ⇒ OpenAI), otherwise unknown
• First/last interaction: timestamps of the first user.message and the last assistant.turn_end (or assistant.message) event

JSONL File Structure (Antigravity)

What: Google's closed-source successor to Gemini CLI, released May 2026. An Electron-based desktop IDE backed by cloudcode-pa.googleapis.com.

Location: %USERPROFILE%\.gemini\antigravity\brain\{session-uuid}\.system_generated\logs\transcript.jsonl

Full schema documentation: docs/logFilesSchema/antigravity-session-format.md

Entry types (each line is one entry):

{"step_index":0,"source":"USER_EXPLICIT","type":"USER_INPUT","status":"DONE","created_at":"2026-05-22T21:48:22Z","content":"<USER_REQUEST>\nuser message here\n</USER_REQUEST>\n<ADDITIONAL_METADATA>...</ADDITIONAL_METADATA>"}
{"step_index":1,"source":"SYSTEM","type":"CONVERSATION_HISTORY","status":"DONE","created_at":"2026-05-22T21:48:22Z"}
{"step_index":2,"source":"MODEL","type":"PLANNER_RESPONSE","status":"DONE","created_at":"2026-05-22T21:48:22Z","tool_calls":[{"name":"search_web","args":{"query":"..."}}]}
{"step_index":3,"source":"MODEL","type":"SEARCH_WEB","status":"DONE","created_at":"2026-05-22T21:48:23Z","content":"Search result text..."}
{"step_index":11,"source":"MODEL","type":"PLANNER_RESPONSE","status":"DONE","created_at":"2026-05-22T21:48:44Z","content":"Final answer text...","thinking":"Chain of thought..."}

Key extraction rules (implemented by AntigravityDataAccess in src/antigravity.ts):

• Interactions: count of USER_INPUT entries (source: "USER_EXPLICIT")
• Session title: content of first USER_INPUT, strip <USER_REQUEST> XML wrapper
• User message: strip the <USER_REQUEST>...</USER_REQUEST> wrapper; discard everything after </USER_REQUEST> (metadata blocks)
• Model response: content field of the last PLANNER_RESPONSE with non-empty content
• Thinking: thinking field of any PLANNER_RESPONSE
• Tool calls: tool_calls[].name on PLANNER_RESPONSE entries (e.g. search_web)
• Tool results: content of SEARCH_WEB entries (and future tool result entry types)
• Actual tokens: always 0 — no token counts in Antigravity transcripts
• Model: not available — not stored in the transcript format
• Session ID: the UUID directory name under brain/
• Timestamps: created_at of first and last entries

Pricing Data

Location: src/modelPricing.json

Contains per-million-token costs for input and output:

{
  "pricing": {
    "gpt-4o": {
      "inputCostPerMillion": 1.75,
      "outputCostPerMillion": 14.0,
      "category": "gpt-4"
    }
  }
}