Splitting Oversized Modules

Splitting oversized modules

A module nobody wants to open taxes every change in it, and agents pay that tax on every task because they do not carry knowledge between them. A 3000-line logic.py beside a 3000-line test_logic.py is roughly 55k input tokens of reading before a line gets written. Splitting one measured case cut the read-set 81 to 97% depending on the concern touched, while total lines grew about 5% from the repeated import headers.

So the goal is a small read-set for a typical change, not tidiness.

Is it worth doing here?

All three must hold: over roughly a thousand lines, several concerns that change independently, and something still actively changing in it. A big cohesive frozen file buys nothing. Skip generated files, and skip a file several people are mid-change in (git log --oneline -20 -- <file>).

A complexity warning on one function is a symptom of this, not a separate job. Extracting that function into helpers leaves every helper in the same file, so the read-set is unchanged and the file gets longer. Measure the file first (wc -l), and when it clears the gate, propose the split instead of the in-place extraction.

Splitting is a separate PR from whatever you came to do. Land the move as its own base PR and branch your work on top — see /stacking-prs. Never bundle it into a feature diff, and say what you are doing in one line before you start: which file, and that the split lands separately. Nobody minds the base PR; they mind finding it inside a feature diff.

If a human has not asked for the split, propose it and let them decide.

Method

Commands assume the repo root. S=.agents/skills/splitting-oversized-modules/scripts.

1. Map the symbols and assign them to concerns

uv run --no-project python $S/split_module.py <module> --skeleton > layout.json

Edit layout.json into modules named after concerns (baselines, quarantine, ci_status), not layers (helpers, utils, core). Put module-level state every module needs its own copy of — a logger, a compiled regex — under "__shared__".

Two rules while assigning:

Keep the dependency graph acyclic. If two modules need each other, the seam is wrong. Separating reads from writes fixes most cycles: a lifecycle module and a verification module that both need the same lookups should share a third leaf module holding those lookups.

Do not name a module after a common local variable. A module called artifacts or runs gets shadowed by artifacts = [...] inside a function, and Python binds the whole scope, so a call above the assignment fails too. ruff catches the reachable cases (F811, F823) but not one where the local is assigned before any module use. Pick a non-colliding name, or split finer until the name is specific.

2. Move the code

uv run --no-project python $S/split_module.py <module> layout.json
ruff check <package>/ --fix && ruff format <package>/

Hand-editing a 3000-line move loses code and silently rewrites it. The script copies each symbol verbatim by AST line range, refuses to run unless the layout covers every symbol exactly once, and requalifies cross-module references with tokenize so it never rewrites a name inside a docstring or string literal.

Cross-module calls come out as from . import baselines plus baselines.foo(...), never from .baselines import foo. One binding per symbol means a mock patch on the definition reaches every caller, and from . import x also survives an accidental cycle by falling back to sys.modules.

ruff --fix leaves an emptied if TYPE_CHECKING: pass behind. Delete those, then re-run ruff check --fix so the orphaned TYPE_CHECKING import goes too. The # --- Section --- dividers from the monolith are usually redundant once the module name says it: rg -n '^#\s*-{2,}' <package>/.

3. Retarget callers and mock patch targets

Every logic.foo() becomes <module>.foo(), and every patch target moves to the module that defines the symbol, since that is the binding its callers resolve:

patch("products.foo.backend.logic._post_commit_status")   # -> ...logic.ci_status._post_commit_status
patch.object(logic, "_post_commit_status")                # -> patch.object(ci_status, "_post_commit_status")

Sweep conftest.py too, not just test_*.py — fixtures patch, and a pass matched on test files leaves conftest pointing at paths that no longer exist. Enumerate candidates with rg -n 'patch\(|patch\.object\(' <tests>/. Watch for forms a naive sweep misses: a combined from x import a, logic, a symbol import (from ..logic import SomeError), and a name the old module merely re-exported, which should now come from its real source.