Polylith Migrate Split Big Component

Skill: polylith-migrate-split-big-component

📐 Scope vs sibling skills. This skill operates within one project and turns one component into many components. Don't confuse with:

• polylith-migrate-extract-standalone-modules — same scope (one project) but pulls foundational modules (consts.py, exceptions.py) out of the residual big component into their own standalone components.
• polylith-migrate-split-component-internals — operates inside one already-extracted component, splitting its core.py into multiple files. No new components are created.
• polylith-migrate-isolate-shared-and-project-logic — cross-project scope; identifies shared-vs-project-specific logic when migrating a 2nd+ project.
• polylith-migrate-dedupe — opportunistic deduplication; this skill includes a dedup-analysis subsection so for a single project you usually don't need polylith-migrate-dedupe separately.

Goal

Split the big component (components/<top_ns>/<INITIAL_BASE_NAME>/) into multiple focused components to improve maintainability, clarity, and reusability.

Inputs

From migration/<PROJECT>/state.md:

• TARGET_TOP_NS
• INITIAL_BASE_NAME
• Verification commands.

From migration/<PROJECT>/manifest.md:

• Module map of the big component.

All inputs from state.md are assumed to satisfy the validation rules in polylith-migrate-discover (### Validation rules). Validate before proceeding.

Steps

Phase 1: Plan the Split

1. Review the Big Component: Use directory_tree and grep to analyze the big component's structure and identify natural slices.
2. Define Component Names: Name components after the domain or functionality they represent (e.g., domain_a_serializer, data_transformations). Avoid generic names like utils or helpers.
3. Create a Split Plan: Record the following in migration/<PROJECT>/split_plan.md:
   • Brick name for each new component.
   • Files/modules to move into each component.
   • Public API (key functions/classes to export).
   • Bricks that will import from the new component.

When NOT to Extract:

• The module is tightly coupled to other modules in the component.
• The module is very small (< 20 lines) and extraction adds more indirection than value.

Extraction Order:

• Extract modules with zero internal dependencies first (e.g., exceptions.py, consts.py).
• Extract modules that depend on already-extracted modules next (e.g., models.py that imports exceptions and consts).

Examples of Component Naming

Name components after the domain or functionality they represent:

Avoiding circular imports when extracting modules

Extracting a module into a separate component can create circular imports if the new component imports from the parent component and the parent still imports from the new component. This commonly happens when a component's __init__.py eagerly imports from many submodules.

Diagnosis: The cycle typically looks like:

new_component.core → parent.__init__ → parent.submodule → new_component

Python triggers parent.__init__ whenever any submodule of parent is imported (e.g. from parent.consts import X loads parent/__init__.py first).

Resolution strategies (in order of preference):

1. Extract the circular part into its own component. A circular dependency often signals that the code involved is isolated enough to be its own component. Extract the module that causes the cycle into a standalone component — this breaks the cycle structurally. A component doesn't have to be a "feature"; it can be a utility, a data definition, a pure technical module, or a single ORM model. If the code has a clear responsibility and can be imported without pulling in the rest of the parent, it belongs in its own brick.

   # Before (cycle): new_component → parent.consts → parent.__init__ → parent.handlers → new_component
   # After (no cycle): new_component → consts_component (standalone, no __init__ chain)

2. Trim __init__.py exports. Remove the problematic import from the parent's __init__.py and have callers import the submodule directly. This makes the dependency graph explicit and often eliminates the cycle without creating a new brick.

   # Before: parent/__init__.py imports everything eagerly
   from parent.command_handler import CommandHandler  # triggers handler → new_component cycle
   
   # After: remove from __init__.py, callers import directly
   from parent.command_handler import CommandHandler  # in the base that needs it

3. Standalone component instead of submodule. If the new component would be a submodule of an existing package (e.g. myns.models.example_transaction), importing it triggers the parent package's __init__.py and all its eager imports. Make it a standalone component at the namespace level instead (e.g. myns.example_transaction).

4. Deferred import (last resort). Move the import inside the function that uses it. This works but hides the dependency and makes the code harder to reason about. Prefer strategies 1–3 first.

Pre-flight check: Before extracting a module, trace the import chain:

1. The new component imports parent.submodule_X → Python loads parent.__init__
2. Does parent.__init__ (directly or transitively) import from the new component?
3. If yes → apply one of the strategies above (extract, trim, or restructure) before proceeding.

Refactoring shared infrastructure components

When a second project needs a component that already exists (e.g. myns.logging, myns.kafka), compare the implementations closely. Common refactoring patterns:

Pattern: Parameterize the shared component. When two implementations are 80%+ identical with project-specific extras, refactor the shared component to accept optional parameters rather than duplicating code.

Example — logging with project-specific loggers:

# Shared component: myns.logging
def init(config, *, extra_loggers=None, cache_logger_on_first_use=False):
    loggers = {**_BASE_LOGGERS}
    loggers.update(_verbosity_overrides(config.LOG_VERBOSITY_LEVEL))
    if extra_loggers:
        loggers.update(extra_loggers)
    ...

# Project A base:
init(config, extra_loggers={"httpx": {...}, "backoff": {...}},
     cache_logger_on_first_use=config.LOG_CACHE_LOGGER_ON_FIRST_USE)

# Project B base:
init(config, extra_loggers={"confluent_kafka_helpers": {...}})

When to parameterize vs. keep separate:

• Parameterize when the core logic is identical and only data/config differs.
• Keep separate when the control flow or structure diverges (different frameworks, different patterns).
• Extract shared base + project-specific wrappers when there's a significant shared core but non-trivial project-specific logic around it.

Splitting Component Internals

Components with generic names like models, schemas, exceptions, or consts may start with a single core.py file. As the workspace grows, these components can accumulate code from different domains. Splitting core.py into multiple domain-focused modules inside the component can improve maintainability and clarity.

When to Split:

• If the core.py file contains definitions from multiple distinct domains (e.g., domain_a and domain_b).
• If the file contains helper/utility functions alongside class definitions.
• If preparing for a second project migration that will contribute to the same component.

Approach:

• Group definitions by the domain concept they serve.
• Name each module after the domain or functionality it represents (e.g., domain_a.py, domain_b.py).
• Ensure the public API remains unchanged to avoid breaking existing imports.