Debug Diff

Debugging Unexpected Diff Output in diffsitter

When diffsitter produces wrong or surprising results, trace the issue through the pipeline. The diff pipeline has 7 stages, and problems can originate at any of them.

Ask the user for the following information if not already provided:

• The two files being compared (or representative snippets)
• The language/file extension
• Any custom config being used
• What output they expected vs what they got

Pipeline Overview

The full pipeline (see src/bin/diffsitter.rs, function run_diff):

1. Language detection
2. Tree-sitter parse (file -> AST Tree)
3. AST leaf extraction (Tree -> Vec<VectorLeaf>)
4. Node filtering (exclude_kinds / include_kinds)
5. Grapheme splitting + whitespace stripping (VectorLeaf -> Vec<Entry>)
6. Myers diff (Vec<Entry> x 2 -> Vec<EditType>)
7. Hunk assembly (Vec<EditType> -> RichHunks)
8. Rendering (RichHunks -> terminal output)

Stage-by-Stage Debugging

Stage 1: Language Detection

File: src/parse.rs, function resolve_language_str

The language is resolved from the file extension via the FILE_EXTS phf_map, with optional user overrides from GrammarConfig.file_associations.

Common issues:

• Extension not mapped (e.g., .jsx maps to "tsx", .h maps to "c" not "cpp")
• User override in config shadowing the default

Debug: Check what language is resolved:

# Enable debug logging to see language resolution
RUST_LOG=debug diffsitter old_file new_file 2>&1 | grep -i "deduced language"

Or read FILE_EXTS in src/parse.rs to verify the extension mapping.

Stage 2: Tree-sitter Parse

File: src/parse.rs, function parse_file

Creates a Parser, sets the language, reads the file to a string, and calls parser.parse(&text, None).

Common issues:

• Grammar ABI version mismatch (AbiOutOfRange error) -- the grammar was compiled against an incompatible tree-sitter version.
• Parse failure returns None from parser.parse(), resulting in a TSParseFailure error.
• File encoding issues (tree-sitter expects UTF-8).

Debug: If parsing succeeds but the tree looks wrong, the grammar itself may have a bug for that language construct. Check tree-sitter's own playground or CLI to inspect the AST.

Stage 3: AST Leaf Extraction

File: src/input_processing.rs, function build

Recursively walks the tree via node.children(&mut cursor). Collects leaf nodes (nodes with child_count() == 0) OR nodes whose kind() matches a pseudo_leaf_types entry.

Key behaviors:

• Empty byte ranges are skipped (node.byte_range().is_empty())
• Nodes that are pure newlines (after removing \n, \r, \r\n) are skipped -- this is a workaround for the Go parser
• Pseudo-leaf types (configured in input_processing.pseudo_leaf_types) treat certain non-leaf nodes as leaves. Default: {"markdown": {"inline"}}. This is critical for text-heavy documents.

Common issues:

• Missing diffs in markdown/prose: Check if pseudo_leaf_types includes the right node types for that language. Without "inline" for markdown, large text blocks are treated as single atoms.
• Unexpected nodes included: The grammar may expose more leaf nodes than expected (e.g., punctuation, delimiters).

Stage 4: Node Filtering

File: src/input_processing.rs, method TreeSitterProcessor::should_include_node

Filters nodes based on exclude_kinds and include_kinds from the config:

• exclude_kinds takes precedence: if a node's kind() is in this set, it is excluded.
• include_kinds: if set, only nodes whose kind() is in this set are included (unless also excluded).
• If neither is set, all nodes pass through.

Common issues:

• User config has exclude_kinds or include_kinds that filters out relevant nodes.
• The node kind() string doesn't match what the user expects (tree-sitter kind names are grammar-specific).

Debug: Check what node kinds exist for a language using tree-sitter's node types. The kind() strings come from the grammar definition.

Stage 5: Grapheme Splitting and Whitespace Stripping

File: src/input_processing.rs, method VectorLeaf::split_on_graphemes

If split_graphemes is enabled (default: true), each leaf's text is split into individual Unicode graphemes, each becoming its own Entry with precise row/column positions. If strip_whitespace is enabled (default: true), whitespace-only graphemes are skipped.

Common issues:

• Whitespace-only diffs not showing: strip_whitespace: true (the default) means pure whitespace/indentation changes are invisible. This is by design for AST-based diffing. If the user wants to see whitespace changes, they need "strip-whitespace": false in their config.
• Performance with large files: split_graphemes: true generates many entries for large text nodes. Setting "split-graphemes": false trades granularity for speed.
• Line position bugs: The grapheme splitter tracks row/column positions. If line_offset == 0, it offsets from the node's start_position().column. Otherwise it resets the column to idx (the byte offset within the line). Bugs here manifest as incorrect column highlighting in the rendered output.