Notebooks

Notebooks

A single skill for authoring, validating, and delivering reproducible analysis notebooks. Marimo is the default format; Jupyter is supported for existing .ipynb files and when a downstream tool requires JSON. Conversion between the two formats is part of this skill.

A notebook is not "done" until it has been executed end-to-end on a fresh kernel and every figure is embedded in the delivered file.

Instructions

1. Pick the format.

   • New notebook: write a marimo .py notebook. Use the canonical cell layout (one concept per cell, final expression renders, no if guards around outputs, no try/except for control flow).
   • Existing .ipynb to extend or polish: keep it as Jupyter unless the user asks to convert.
   • Conversion: see "Convert between marimo and Jupyter" below.

2. Outline before coding. Write the notebook plan (purpose, data sources, analysis steps, expected outputs/plots) as the first markdown cell, then implement against that plan.

3. Keep marimo cells clean. These are hard rules for every .py notebook:

   • Markdown cells use one plain triple-quoted string: mo.md(r"""...""") or mo.md(f"""...""") only when interpolation is required. Put the prose directly inside the string; never paste quoted string fragments such as " ... " lines inside the markdown body.
   • Do not leave empty generated cells, whitespace-only cells, or @app.cell def _(): return placeholders. Remove them before final verification.
   • Do not accept a marimo "fix" prompt blindly. If one is accepted during interactive editing, inspect the diff immediately and remove unintended PEP 723/header/cell churn.

4. Set up the kernel and dependencies.

   • Marimo. Pin dependencies in the PEP 723 script header at the top of the .py file:

     # /// script
     # requires-python = ">=3.12"
     # dependencies = [
     #     "marimo",
     #     "polars",
     #     "duckdb",
     #     "matplotlib",
     #     # ... add every import used in the notebook
     # ]
     # ///

     Run with uv run marimo run <notebook.py> (uv reads the header and resolves the env automatically) or with marimo edit --sandbox <notebook.py> for interactive work.
   • Jupyter. Register a named ipykernel for the project's pixi env before the first execution and pin the kernel in the notebook metadata. The kernel name is mandatory — the generic python3 kernel leaks the system interpreter:

     pixi run python -m ipykernel install --user --name <project> --display-name "<project> (pixi)"

     Then in <notebook>.ipynb confirm:

     "kernelspec": {"name": "<project>", "display_name": "<project> (pixi)"}

     Add every import used in the notebook to pixi.toml (or the project's requirements.txt / environment.yml) so the kernel can resolve it from a clean install.

5. Load data with project-relative paths. Prefer DuckDB for TSV/Parquet (duckdb.read_csv, duckdb.read_parquet). Avoid absolute paths and ~. Avoid hidden state from the runtime working directory.

6. Run checks and all cells to generate plots. Execute the notebook headlessly on a fresh kernel before delivery:

   • Marimo: run uvx marimo check <notebook.py> before export and fix every reported issue or warning, including empty-cells and markdown formatting. Then run uv run marimo export ipynb <notebook.py> -o <notebook.executed.ipynb> or uv run marimo run <notebook.py> for a non-interactive smoke run; for a deterministic HTML artifact, uv run marimo export html <notebook.py> -o <notebook.html>. Run uvx marimo check <notebook.py> again after the final edit/export cycle.
   • Jupyter: python scripts/execute_notebook.py <notebook.ipynb> (writes <notebook>.executed.ipynb) or pixi run jupyter nbconvert --to notebook --execute --inplace <notebook.ipynb>.

7. Evaluate the plots, then refine. This step is required, not optional. After the run-all execution:

   • Open the executed notebook (or exported HTML) and visually inspect every figure.
   • Check for: empty axes, mis-scaled axes (log when linear was intended or vice versa), missing labels/legends, overlapping ticks, illegible font sizes at target output size, ambiguous palettes, colorbars without units, NaN-driven gaps, axis ranges clipping data, broken layouts.
   • If a figure is wrong or unclear, edit the source cell and re-run end-to-end. Repeat until each figure communicates what the surrounding markdown says it communicates.
   • Record what changed between revisions in a brief "Figure revision log" markdown cell or in the run log.

8. Deliver pre-executed notebooks. The artifact handed back to the user must:

   • Have every cell executed against the registered kernel.
   • Embed every figure (PNG / SVG cell outputs) directly in the .ipynb (or in the marimo HTML export).
   • Be reproducible from a clean clone: a new environment built from the PEP 723 header (marimo) or pixi install + jupyter nbconvert --to notebook --execute (Jupyter) must reproduce the same notebook end-to-end.

9. Convert between marimo and Jupyter when the user asks for it:

   • .ipynb → marimo .py: uvx marimo convert <notebook.ipynb> -o <notebook.py>, then uvx marimo check <notebook.py>, then clean up Jupyter artifacts (display() calls, %magics, indented final expressions, ipywidget usage). See references/widgets.md and references/latex.md for ipywidget→marimo and MathJax→KaTeX mappings.
   • marimo .py → .ipynb: uvx marimo export ipynb <notebook.py> -o <notebook.ipynb>.
   • After conversion, re-run step 6 (check/execute), step 7 (inspect plots), and step 8 (deliver pre-executed).