Flowfile Build and Environment Setup

Flowfile: build & environment recreation

This skill tells you how to stand up every flavor of the Flowfile dev/build environment and what each Makefile target actually does under the hood (as opposed to what its name implies). It does not cover how to write or test code inside an already-running environment.

When NOT to use this skill

• You already have a working dev environment and just want to start/stop services, seed demo data, or drive the app for a manual check — see flowfile-run-and-operate.
• You're picking pytest markers, writing new tests, or setting up Docker-backed test fixtures (postgres/mysql/s3/kafka) — see flowfile-testing-and-validation.
• You need to know what an env var or feature flag does at runtime (FLOWFILE_MODE, FEATURE_FLAG_AI, offload flags) rather than how to set it up for a build — see flowfile-config-and-flags.
• You're debugging a broken flow/AI/node at runtime, not a broken build — see flowfile-debugging-playbook or flowfile-ai-subsystem.
• You need the core/worker/kernel contract or subsystem architecture, not build mechanics — see flowfile-architecture-contract.

1. Ground-truth version pins (as of 2026-07-03, v0.12.7)

Verify your machine matches:

python3 --version && poetry --version && node --version && npm --version && rustc --version

2. From-scratch recipes

(a) Backend dev (core + worker, no frontend)

poetry install                    # main + dev groups (pytest, ruff, mkdocs, testcontainers...)
poetry run flowfile_core          # FastAPI on :63578
poetry run flowfile_worker        # separate terminal, :63579

Notes:

• Importing flowfile_core has side effects: flowfile_core/flowfile_core/__init__.py calls validate_setup() then init_db() at module import time, which runs Alembic migrations against the live catalog SQLite DB. For any diagnostic script that just imports the package without wanting to touch the DB, set FLOWFILE_SKIP_STARTUP_MIGRATION=1 first.
• FLOWFILE_MODE defaults to electron when unset. The embedded scheduler only starts if FLOWFILE_SCHEDULER_ENABLED is truthy.
• Local data lands under ~/.flowfile (internal storage) and your home dir (user data); override with FLOWFILE_STORAGE_DIR / FLOWFILE_USER_DATA_DIR.
• poetry install --with build is only needed when you're going to run PyInstaller (§4) — skip it for plain backend dev, it just costs install time.

(b) Frontend web dev (no Tauri/Rust needed)

poetry run flowfile_core          # start this FIRST — Vite proxies /api → :63578
cd flowfile_frontend
npm install
npm run dev:web                   # Vite dev server at http://localhost:8080

Notes (flowfile_frontend/vite.config.mjs):

• root is src/renderer/; dev server listens on port 8080 with strictPort: true — it will refuse to start (not fall back to 8081/8082) if the port is taken, because Tauri's devUrl is hard-coded to http://localhost:8080 and a silent port-jump would break npm run dev.
• The /api proxy strips the /api prefix when forwarding to core, and also rewrites FastAPI's absolute 307 trailing-slash redirects back under /api so client requests stay same-origin — a bug here manifests as requests that work in curl but 404 in the browser.
• optimizeDeps.force: true means every cold start re-bundles dependencies (5–15s) — this is intentional, not a bug, to avoid a stale "504 Outdated Optimize Dep" error.
• There is no dev server on port 5173. The only 5173 reference in the repo is a leftover entry in core's CORS allowlist; if you see a tutorial mention 5173, it does not apply here.

(c) Full Tauri desktop build

Prereqs: Python 3.10–3.13 + Poetry 1.8.x + Node 20+ + a Rust toolchain (rustc must be on PATH, even just to stage sidecars).

make all

make all is exactly install_python_deps build_python_services rename_sidecars sign_sidecars build_tauri_app generate_key, run in that fixed order:

1. poetry install --with build (self-heals a stale lock first — see §5 gotcha 1)
2. poetry run build_backends → PyInstaller onedir builds of core + worker into services_dist/ (§4)
3. poetry run python tools/rename_sidecar.py → stages services_dist/ into flowfile_frontend/src-tauri/binaries/<name>-<host-triple> (§4b)
4. sign_sidecars → no-op unless macOS and APPLE_SIGNING_IDENTITY is set (§5)
5. cd flowfile_frontend && npm install && npm run build → npm run build = npm run lint && vue-tsc --noEmit && tauri build
6. generate_key → writes repo-root master_key.txt only if it doesn't already exist (§4c — note this file is dev convenience, not what the running app reads)

Day-to-day dev-mode desktop shell (skip the full release pipeline):

make services                     # = build_python_services + rename_sidecars — MUST run before `npm run dev`
cd flowfile_frontend && npm run dev    # tauri dev; beforeDevCommand runs `npm run dev:web` for you

If you skip make services, the Tauri shell still launches but with no backend and no error at build time — sidecar resolution (flowfile_frontend/src-tauri/src/sidecar/mod.rs) just fails to find the binary at runtime. If the desktop app opens to a blank/non-functional UI, re-run make services.

Whenever you re-stage sidecars (re-run rename_sidecars after rebuilding core/worker), re-run npm run dev / rebuild — a stale cargo build can keep pointing at a binary that no longer matches.

Platform-specific one-shot builds (make build_tauri_win|_mac|_mac_arm|_mac_intel|_linux) call npx tauri build --target <triple> directly — unlike build_tauri_app they skip npm run lint and vue-tsc --noEmit and don't run npm install for you. Run npm install yourself first if node_modules isn't already present.

tauri.conf.json key facts: frontendDist: "../build/renderer", devUrl: "http://localhost:8080", beforeBuildCommand: "npm run build:renderer-only" (note: no lint/typecheck in this hook — that check happens earlier in npm run build), bundle targets: ["app","dmg","nsis","deb"], createUpdaterArtifacts: true. Sidecars ship via bundle.resources: ["binaries/**/*"], not Tauri's externalBin mechanism.

(d) Docker full stack

cp .env.example .env
# set FLOWFILE_MASTER_KEY, JWT_SECRET_KEY, FLOWFILE_INTERNAL_TOKEN, FLOWFILE_ADMIN_USER/PASSWORD
docker compose up -d
# frontend: http://localhost:8080 (nginx), core: :63578, worker: :63579

Compose starts core, worker, and frontend only — kernel images are build-only profiles, not part of the default up:

docker compose --profile kernel build flowfile-kernel        # -> flowfile-kernel-base:local
docker compose --profile kernel build flowfile-kernel-ml     # EXTRAS=ml build arg
docker compose --profile kernel build flowfile-kernel-lite   # SLIM_CONSTRAINTS=true build arg

Then point core at your local image (unset/empty env vars fall through to the published defaults pinned in flowfile_core/flowfile_core/kernel/images.py):

# in .env or the shell running `docker compose up`
FLOWFILE_KERNEL_IMAGE=flowfile-kernel-base:local
# or per-flavour: FLOWFILE_KERNEL_IMAGE_BASE / _ML / _LITE / _NOTEBOOK

Other things worth knowing before you run this:

• Core mounts the host Docker socket (/var/run/docker.sock) to manage kernel containers — this is a documented security tradeoff in docker-compose.yml, not an accident.
• Core + worker share a named volume flowfile-internal-storage; user data bind-mounts to ./flowfile_data and ./saved_flows.
• The compose network is a fixed name (flowfile-network) so kernel containers spun up dynamically via the Docker API can join it.
• Defaults are insecure on purpose for local dev (admin/changeme, dev JWT/internal tokens) — override in .env for anything shared.
• There is no secrets: block in compose. FLOWFILE_MASTER_KEY unset ⇒ the UI shows a first-run setup screen that prompts for one; env var always wins over a manually-added Docker secret file.
• Core and worker images are python:3.12-slim; the Dockerfile installs psycopg2-binary/pymysql explicitly at image-build time even though those are dev-group-only in pyproject.toml — a plain poetry install --only main locally will NOT have Postgres/MySQL drivers.

(e) WASM dev (fully standalone — no backend)

cd flowfile_wasm
npm install
npm run dev                       # http://localhost:5174

• WASM has its own package.json/lockfile, independent of flowfile_frontend. It tracks React 19 and the same @kanaries/graphic-walker major as the main frontend; a plain npm install and a clean npm ci both resolve.
• On a peer-dep error, add the missing package to flowfile_wasm/package.json explicitly rather than reaching for --legacy-peer-deps — that flag stops npm auto-installing peers, which has already bitten codemirror and styled-components once.
• Pyodide is loaded from a CDN at runtime, pinned to v0.27.7 (the last release with Polars support) — it is not an npm dependency, so npm install won't show it. The pin also appears in the pyodide-smoke CI job and flowfile_wasm/tests/python/requirements.txt (which separately pins polars 1.18.0, pydantic 2.10.5, polars-expr-transformer 0.6.0 for the CPython-side test suite) — bump all of these together if you touch one.
• Dev server headers include COOP/COEP (Cross-Origin-Opener-Policy: same-origin, Cross-Origin-Embedder-Policy: require-corp) — required for SharedArrayBuffer; anyone embedding the built package on another host page must set the same headers.
• Other scripts: npm run build (app, vue-tsc --noEmit && vite build), npm run build:lib (BUILD_MODE=lib → dist/flowfile-editor.js, the published npm package), npm run test:run (Vitest one-shot, happy-dom env).

(f) pip-installed unified mode

pip install flowfile
flowfile run ui              # serves bundled web UI + core + worker on ONE port

Importing the flowfile package sets FLOWFILE_WORKER_PORT=63578 and FLOWFILE_SINGLE_FILE_MODE=1 as a side effect at import time (flowfile/flowfile/__init__.py) — do not import flowfile in the same process as a normally-configured core/worker pair, it will silently rewire the worker port out from under you. Other CLI verbs: flowfile run {ui|core|worker|flow <path>} (run flow forces local, non-offloaded execution and strips UI-only nodes), flowfile project {init|open|save}, flowfile seed-demo / remove-demo.

3. Makefile target reference (what each target truly does, not what its name implies)