Flowfile Docs And Writing

Flowfile docs & writing conventions

When NOT to use this skill

• Editorial standard for docs pages (house style/voice rules, the persona nav map, fact-checking a docs claim against code, the tested-examples contract and add-an-example recipe) → flowfile-docs-review (this skill owns how the site builds; that one owns what good pages say and how to verify it).
• CI gates that happen to touch docs (check-formula-docs, check-stubs jobs, version-sync) and release mechanics → flowfile-change-control (this skill explains what the formula-docs generator does and the doc-authoring traps; that skill owns why the CI job exists and what blocks a merge).
• flowfile_frame public API design, .pyi stub authoring, expression codegen → flowfile-frame-and-codegen.
• Writing/reviewing a custom node's NodeSettings docstrings for the Node Designer UI → flowfile-node-development.
• Vue/TS component conventions, ESLint/Prettier setup → flowfile-frontend-conventions.
• AI prompt-string authoring (system prompts, tool descriptions) → flowfile-ai-subsystem (this skill only tells you prompt strings are exempt from the comment doctrine, not how to write them).
• Env var / feature-flag reference tables → flowfile-config-and-flags.
• Chasing a specific historical bug for its own sake → flowfile-failure-archaeology.

1. Docs inventory — what exists, and where a new page goes

Two documentation surfaces exist; don't confuse them:

The docs/ tree structure mirrors mkdocs.yml's nav: block almost exactly (verify by diffing them — see §3.3 for the one place they currently disagree). Top-level layout:

Decision rule: if the reader is using Flowfile (drag-drop or flowfile_frame), write under users/. If the reader is extending or debugging Flowfile, write under for-developers/. If in doubt, look at which register the neighboring pages use (§4) — that tells you the audience faster than the folder name does.

Every new page must be added to mkdocs.yml's nav: block by hand — MkDocs does not auto-discover nav entries from the docs/ tree (§3.3 has a live example of what happens if you forget).

2. How the docs site builds and deploys

• Engine: MkDocs 1.6.1 + Material theme (deps come from the root Poetry dev group — pyproject.toml's mkdocs, mkdocs-material, mkdocstrings, mkdocstrings-python, griffe, griffe-pydantic.).
• Plugins: search; mkdocstrings (Python handler, griffe_pydantic extension with schema: true — this is what renders Pydantic "Fields"/"Validators" sections on for-developers/python-api-reference.md, which documents flowfile_core classes like FlowGraph/FlowNode/FlowDataEngine and the Pydantic schemas).
• Markdown extensions: attr_list, md_in_html, admonition, pymdownx.details (the <details markdown="1"> collapsibles you'll see all over quickstart.md), pymdownx.superfences with a custom mermaid fence, footnotes.
• CI: .github/workflows/documentation.yml, triggered on push/PR to main touching docs/**, mkdocs.yml, flowfile_frame/**/*.py, or tools/generate_formula_docs.py (plus workflow_dispatch).
  • build job: Python 3.11 → poetry install --with dev → make check_formula_docs (drift gate, see §3.4) → poetry run mkdocs build → uploads site/ as an artifact.
  • deploy job (push-to-main only): poetry run mkdocs gh-deploy --force → publishes to the gh-pages branch → GitHub Pages. There is no staging environment; a merge to main that touches docs paths goes live on the next deploy run.
• Build locally: poetry run mkdocs build — this is exactly what CI runs and reproduces every warning CI would see. Live-reload with poetry run mkdocs serve.

Gotcha — local build touches your live catalog DB. for-developers/python-api-reference.md uses mkdocstrings to introspect flowfile_core classes, which means mkdocs build imports flowfile_core as a side effect of building the docs. Importing flowfile_core runs validate_setup() + init_db() at import time, including the Alembic startup migration, against whatever DB get_database_url() resolves to — your real local flowfile_catalog.db if you haven't set anything. Set FLOWFILE_SKIP_STARTUP_MIGRATION=1 before running mkdocs build/mkdocs serve locally, same as any other diagnostic import of flowfile_core.

Gotcha — the build does not fail on warnings. documentation.yml runs plain poetry run mkdocs build, with no --strict flag. Orphaned nav pages, broken in-page anchors, and griffe docstring-parsing warnings all print as INFO/WARNING lines and the job still exits 0. A green Documentation workflow run is not proof the docs are internally consistent — read the log, don't just trust the check mark.

3. Traps — read before touching docs/

3.1 The site home is docs/index.html, not index.md

mkdocs.yml's nav: starts with - Home: index.html. The home page is a hand-authored HTML file (raw <a href> tags, inline CSS using Material's CSS custom properties) — MkDocs supports mixing static HTML pages into the nav alongside Markdown, and this repo uses that for the landing page. If you "fix" this by adding a docs/index.md and repointing the nav, you change the entire site's home page and lose all the hand-tuned hero/feature-grid markup in one edit. If the home page needs a content change, edit docs/index.html directly.

3.2 use_directory_urls: false — know which links need .html

mkdocs.yml sets use_directory_urls: false, so every non-index page publishes as a flat <name>.html file, not a directory with an index.html inside (e.g. quickstart.md → quickstart.html, reachable at .../quickstart.html, not .../quickstart/). Two different rules apply depending on where the link lives:

• Inside a .md page, write relative links the normal way, ending in .md (e.g. [Provider Setup](providers.md), [Visual Editor](visual-editor/index.md)) — MkDocs' Markdown processor rewrites these to the correct published extension at build time regardless of the use_directory_urls setting. This is why the existing pages are full of .md-suffixed relative links and they all work.
• Inside docs/index.html (raw HTML, not run through the Markdown link-rewriter) or any hardcoded absolute URL outside the docs build (README.md, CONTRIBUTING.md, a package README linking to the hosted site), you must write the actual published path by hand: flat pages need the literal .html suffix (href="quickstart.html", not href="quickstart/" — the latter 404s because no such directory exists), while section index pages can use a trailing-slash directory form (href="users/", href="for-developers/") because index.html is always served as the directory default regardless of use_directory_urls. docs/index.html itself mixes both forms correctly today — use it as the reference. CONTRIBUTING.md links to https://edwardvaneechoud.github.io/Flowfile/community.html for the same reason.

3.3 Forgetting the nav: entry makes a page invisible, not broken

MkDocs builds every .md file under docs/ whether or not it's listed in mkdocs.yml's nav: — a missing nav entry does not fail the build (§2's "no --strict" gotcha) and does not stop the page from existing at its URL if something else links to it directly. It just means the page is absent from the sidebar and site search weighting. The build prints an INFO line ("The following pages exist in the docs directory, but are not included in the 'nav' configuration") and still exits 0. If you add a new page and it doesn't show up in the sidebar after a build, this is almost certainly why — check mkdocs.yml's nav: block before assuming something is actually broken. (The long-standing live example, an orphaned for-developers/docker-deployment.md, was merged into users/deployment/docker.md in the 2026-07 docs rebuild, with a redirects-plugin mapping preserving its URL — as of that rebuild the nav is orphan-free, and the redirect map in mkdocs.yml is where moved pages get recorded.)

3.4 The formula reference is generated — never hand-edit it

docs/users/formulas/functions.md starts with <!-- AUTO-GENERATED by tools/generate_formula_docs.py — do not edit. Run 'make formula_docs'. --> and documents all 95 functions of the Flowfile formula language, sourced from polars-expr-transformer's own docstrings (the library is upstream of this repo; bumping its pin is what usually triggers drift here).

make formula_docs         # regenerate docs/users/formulas/functions.md from the current polars-expr-transformer pin
make check_formula_docs   # formula_docs + `git diff --exit-code` on that file — what CI runs

If you bump the polars-expr-transformer version pin (or edit tools/generate_formula_docs.py) and forget to run make formula_docs, documentation.yml's check-formula-docs gate fails on a diff in a file you never touched by hand. This is CI-gate mechanics — for how the gate fits into the broader change-control picture (required checks, what blocks a merge), see flowfile-change-control.

4. House style for docs pages

Since the 2026-07 docs rebuild there is one register: tight declarative prose, Material admonitions (!!! info, !!! tip, !!! warning, with a quoted title), comparison tables, exact file paths and symbol names, explicit invariant statements, mermaid diagrams where a picture beats prose. The old marketing register (emoji headers, gradient <div> cards, clamp() inline CSS, hype prose) was eliminated page-by-page — do not reintroduce it. The full editorial standard — voice rules (including the ban on generic "Tips for Success" filler), the persona nav map, the claim→source verification index, and the tested-examples contract — lives in flowfile-docs-review; read that skill before writing or reviewing any page.

Admonition syntax (register 2's signature move):

!!! note "Registered Flows Required"
    `publish_global` requires the flow to be registered in the catalog. It is not available in interactive (cell) mode.

!!! warning "Docker Socket"
    The kernel manager needs access to the Docker socket to spawn containers.

Package-level READMEs (flowfile_core/README.md, flowfile_worker/README.md, flowfile_frontend/README.md, flowfile_wasm/README.md) use their own feature-list-with-emoji register — that's a separate, narrower convention scoped to "here's what this package does," not the MkDocs site's house style. Don't import docs-site register 2 conventions into a package README wholesale; keep it short.