Flowfile Config and Flags Catalog

3.17 Test-infrastructure env (not runtime config, but easy to confuse with the above)

• TEST_MODE — flowfile_worker/configs.py:18: presence-based ("TEST_MODE" in os.environ) — TEST_MODE=0 still enables it! Effect: flowfile_worker/secrets.py:128 returns a static test master key instead of real key resolution. CI sets it in .github/workflows/test-kafka-integration.yml:115 and test-kernel-integration.yml:72.
• TESTING='True' — set by flowfile_core/tests/conftest.py:23; see §3.4 for the shared-DB hazard.
• FLOWFILE_WORKER_HOST (conftest.py:53, default 0.0.0.0), FLOWFILE_STARTUP_TIMEOUT (:56, 30s), FLOWFILE_SHUTDOWN_TIMEOUT (:58, 15s), SKIP_WORKER_TESTS (:226, == "1"), FLOWFILE_TEST_REUSE_WORKER (conftest.py::_claim_worker_port, == "1": reuse a worker already on the default 63579 instead of moving the session to a free port; an explicit FLOWFILE_WORKER_PORT is always used as given) — test-only. FLOWFILE_WORKER_HOST is NOT read by package source (which uses WORKER_HOST) — near-identical name, different variable, easy to set the wrong one.
• test_utils/ Docker fixtures: TEST_POSTGRES_{HOST,PORT,USER,PASSWORD,DB,SCHEMA,IMAGE,CONTAINER,STARTUP_TIMEOUT,SHUTDOWN_TIMEOUT}, TEST_MYSQL_* (same shape + ROOT_PASSWORD), TEST_MINIO_{HOST,PORT,CONSOLE_PORT,ACCESS_KEY,SECRET_KEY,CONTAINER}, TEST_GCS_{HOST,PORT,CONTAINER}, TEST_AZURITE_{HOST,BLOB_PORT,CONTAINER}, TEST_REDPANDA_{HOST,PORT,IMAGE,CONTAINER}, KEEP_{MINIO,GCS,AZURITE,REDPANDA}_RUNNING, CI, GITHUB_ACTIONS.
• Tests also export AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_ENDPOINT_URL/AWS_REGION/AWS_ALLOW_HTTP/AWS_SESSION_TOKEN for object-store SDKs. Package source never reads AWS_* itself, but the ambient AWS environment is still live runtime config: a cloud node with No connection, a saved aws-cli connection with a blank aws_profile, and any env_vars connection resolve credentials through boto3's default chain and object_store's own env lookup, so AWS_* in the core/worker environment (AWS_PROFILE, AWS_SHARED_CREDENTIALS_FILE, AWS_CONFIG_FILE, AWS_ENDPOINT_URL, AWS_ALLOW_HTTP, …) decides where such a node reads and writes (a saved connection's own endpoint_url still wins). Docker mode (auth/sharing.py, read live): the no-connection fallback is refused for every user, admins included (ambient_credentials_allowed() → flow_graph.get_cloud_connection_settings raises ValueError("Select a cloud storage connection; …")); server-identity connections (SERVER_IDENTITY_AUTH_METHODS = aws-cli, env_vars, iam_role, managed_identity; uses_server_identity()) are admin-only — a non-admin create/update is a 422 in routes/cloud_connections.py, and db_connections.get_cloud_connection_schema raises CloudConnectionNotAllowedError for any such row whose owner is not an admin (nodes, storage browser + cloud Delta routes as 400 CONNECTION_NOT_ALLOWED, catalog storage), so admin-owned ones keep working, group-shared included; cloud node paths must be cloud URIs (shared/cloud_storage/utils.py::validate_cloud_resource_path: empty/relative refused in every mode, absolute local paths refused in docker). Suites that exercise it pin a temp static-key profile plus AWS_ENDPOINT_URL=http://localhost:9000, AWS_ALLOW_HTTP=true and AWS_EC2_METADATA_DISABLED=true, so nothing can reach real AWS.

3.18 CI-only runtime env (across all 12 workflows — only 3 hits)

TEST_MODE: "1" (kafka + kernel integration workflows); FLOWFILE_INTERNAL_TOKEN: ${{ github.run_id }}-test-token (test-kernel-integration.yml:74); FLOWFILE_KERNEL_IMAGE: flowfile-kernel-base:test (test-kernel-integration.yml:76).

3.19 Interpreter hygiene (Dockerfiles, not app config)

PYTHONPATH=/app, PYTHONDONTWRITEBYTECODE=1, PYTHONUNBUFFERED=1 in flowfile_core/Dockerfile:69-72, flowfile_worker/Dockerfile:61-64, kernel_runtime/Dockerfile:80-84 (+ docker-compose.yml:46-47,75-76).

4. MutableBool — flipping a feature flag without restart

flowfile_core/flowfile_core/configs/utils.py defines MutableBool: a dataclass wrapping value: bool with __bool__, &/| combinators, int/float coercion, and .set(value). settings.py exports six instances: SINGLE_FILE_MODE, OFFLOAD_TO_WORKER, FEATURE_FLAG_AI, FLOWFILE_LSP_ENABLED, FLOWFILE_AI_LOG_PROMPTS, FLOWFILE_AI_LOG_PROMPTS_SCRUB. Every call site does bool(_settings.FEATURE_FLAG_AI) at call time, so .set() takes effect immediately, process-wide, with no reload — but only in that process (see caveat below).

FEATURE_FLAG_AI live flip:

• Gate: flowfile_core/flowfile_core/ai/feature_flag.py — is_ai_enabled() reads the MutableBool per call; require_ai_enabled() is a router-level Depends raising 503 with "AI features are disabled. Set FEATURE_FLAG_AI=true to enable.".
• Flip: POST /system/feature_flags/ai (ai/admin_routes.py:58-78, admin-only, mounted under /system — deliberately outside the gated /ai router so it stays callable while the gate is off). Sets both settings.FEATURE_FLAG_AI.set(enabled) and os.environ["FEATURE_FLAG_AI"] (admin_routes.py:76-77). Response always has persisted: false — surviving a restart is the operator's .env's job, not this endpoint's. GET /system/feature_flags/ai reads current state (also admin-only).

FLOWFILE_LSP_ENABLED live flip: identical pattern in flowfile_core/flowfile_core/lsp/admin_routes.py — POST/GET /system/feature_flags/lsp (admin_routes.py:49,60). Difference: /lsp/* never 503s when off — it degrades to empty-200 so the editor falls back to client-side completion. Caveat in the module docstring: open editor sessions cache /lsp/capabilities per session, so a live flip only reaches them on reload.

Other live-mutable state (not admin-endpoint-backed):

• OFFLOAD_TO_WORKER.set(False) is forced by python -m flowfile_core.main --run-flow (per flowfile_core/CLAUDE.md), and flowfile/web/__init__.py:157 sets OFFLOAD_TO_WORKER.value = True in unified (pip) mode.
• import flowfile mutates os.environ unconditionally: flowfile/flowfile/__init__.py:17-18 sets FLOWFILE_WORKER_PORT=63578 and FLOWFILE_SINGLE_FILE_MODE=1 at import time, before any of your code runs. This is how the pip-installed unified mode co-hosts the worker on the core port. Consequence: any process that does import flowfile (even just to reach flowfile_frame) silently flips core into single-file mode if flowfile_core.configs.settings is imported afterward. Import order matters.

Multi-process caveat: in a multi-worker deployment (e.g. gunicorn -w N), a flip via HTTP hits one worker process only (inferred from the per-process nature of MutableBool; .env.example documents the identical caveat for AI rate limits — §3.6, per-process in-memory deques).

5. Parsing quirks — truthy rules are NOT uniform

This is the single most common source of "I set the var and nothing changed" bugs. There are five different truthy rules in play:

Corollary gotchas:

• _envvar_or_default (kernel image vars, manager.py:48-55) treats an empty string as unset, deliberately, because Compose's ${VAR:-} interpolation writes "" for an unset host var. Don't "fix" this to treat "" as a real override.
• AVAILABLE_RAM, DEBUG go through Starlette's cast=bool/cast=int, not the hand-rolled truthy checks above — different failure mode (raises on unparseable input rather than silently falling to default).

6. Drift report (both directions)

Documented but missing or stale

1. .env.example:57 calls the /project router "admin-only" in docker mode — unverified nuance (the router-level permission dependency itself is out of this skill's scope); treat as a flag, not a confirmed fact.

Read in code but dead

5. TEMP_DIR env var — settings.get_temp_dir() (settings.py:86-92) has zero callers repo-wide.
6. FLOWFILE_SHARED_PATH — kernel_runtime/flowfile_client.py:221 assigns _SHARED_PATH; never referenced again.
7. DEBUG and FILE_LOCATION Starlette-Config keys — read into module constants with zero consumers.

Read in code but absent from BOTH .env.example and root CLAUDE.md

FLOWFILE_LSP_ENABLED (+ its admin flip endpoint) · FLOWFILE_API_RUN_TIMEOUT_SECONDS · FLOWFILE_API_MAX_CONCURRENT_RUNS · FLOWFILE_ARTIFACT_STORAGE / FLOWFILE_S3_BUCKET / _PREFIX / _REGION / _ENDPOINT_URL · FLOWFILE_LOCAL_MODEL_CTX · FLOWFILE_DB_READ_HEDGE_DELAY · FLOWFILE_DOCKER_NETWORK · TESTING / FLOWFILE_SKIP_STARTUP_MIGRATION · FLOWFILE_HOST / FLOWFILE_PORT / FLOWFILE_MODULE_NAME / FORCE_POETRY / POETRY_PATH · SECURE_STORAGE_PATH · GOOGLE_OAUTH_CLIENT_ID / _CLIENT_SECRET / _REDIRECT_URI · FLOWFILE_SECRET_* placeholder prefix · AVAILABLE_RAM (Starlette key, live) · FLOWFILE_INTERNAL_SERVICE_USER_ID (documented only in docs/for-developers/kernel-architecture.md) · most of the kernel-container contract vars in §3.9 (PERSISTENCE_*, RECOVERY_MODE, KERNEL_ID, MAX_NAMESPACES, MAX_DISPLAY_OUTPUTS, KERNEL_PACKAGES, KERNEL_CONSTRAINTS_FILE, the FLOWFILE_HOST_*/FLOWFILE_KERNEL_* path-translation quartet — some appear in docs/for-developers/kernel-architecture.md, most don't anywhere).

.env.example ↔ docker-compose.yml divergences (trap-shaped, not bugs)

• .env.example has no FLOWFILE_SCHEDULER_ENABLED line at all; compose hard-codes true; docs/users/deployment/docker.md says default is false. Net effect: local/pip runs never start the scheduler unless you set it yourself; a fresh .env copied from .env.example for a non-compose docker run also won't start it.
• FLOWFILE_ENABLE_PROJECTS: env-level code default is off (settings.py:50), but compose defaults it on (compose:34) and .env.example:62 ships true. Read the compose file, not just the code default, before assuming projects are off.
• Compose passes empty strings for unset kernel-image vars (${VAR:-}). This is safe only because _envvar_or_default treats empty as unset (§5) — copying that ${VAR:-} pattern for a var that does NOT special-case empty (e.g. FLOWFILE_MASTER_KEY, compose:31) yields an empty-string env that the consuming code must handle explicitly (it does, via a falsy check in secrets.py, but this is not free — verify before reusing the pattern for a new var).

7. Add-a-config-axis checklist

When you add a new environment-variable-driven behavior:

1. Read it in exactly one place if possible. If it must be read per-call (like sharing_enabled() or the catalog-storage getters), say so in a comment — the import-time-vs-per-call split is the #1 source of "I set it but nothing changed" bugs in this codebase (§2, §6).
2. Pick a truthy rule deliberately and be consistent with a sibling flag, not with "whatever felt natural." Prefer the true/1/yes/on rule used by the MutableBool flags (§3.3) unless you have a specific reason (e.g. FLOWFILE_SINGLE_FILE_MODE's exact-"1" rule exists to avoid ambiguity in a mode-switch, not by accident). Document the rule inline — this table exists because five different rules already crept in un-intentionally (§5).
3. Give it a real default in code, and decide whether an empty string counts as "set" (compose's ${VAR:-} idiom means it usually shouldn't — mirror _envvar_or_default, manager.py:48-55, if the var can be Compose-templated).
4. Add a line to .env.example (repo root, the only one in the repo) with a one-line comment; if the semantics are non-obvious (like FLOWFILE_CATALOG_STORAGE_URI's creation-time-only caveat), write the caveat there, not just in code.
5. Add it to root CLAUDE.md's Environment Variables table if it's operator-facing; if it's a container-internal contract var like the ones in §3.9, a one-line mention in docs/for-developers/kernel-architecture.md is enough — don't bloat the root table with vars nobody sets by hand.
6. Wire it into docker-compose.yml if docker deployments need a non-default value, and make sure the compose default doesn't silently diverge from the code default (§6 has three live examples of exactly this drift — don't add a fourth).
7. Add or extend a test that asserts the default behavior AND at least one non-default value takes effect. If the var is read in the worker or a kernel container, a subprocess/container-level test (not just a unit test of the getter) is the only way to catch the "set at core, but the string mismatches at the read site" class of bug.
8. If it's a live-flippable flag, follow the MutableBool pattern (§4): mount the admin endpoint under /system, not the feature's own gated router, set both the MutableBool and os.environ, and return persisted: false honestly — do not silently write back to a .env file.
9. Decide whether the frontend needs to know (maintainer guidance, 2026-07-03: many config axes must be frontend-accessible — plan the exposure path when you add the var, not after). The renderer cannot read backend env: in Docker it is static files behind nginx, and import.meta.env is baked at build time — fine for build concerns, wrong for anything an operator sets at deploy time. If the var changes UI-visible behavior, expose it through a backend response and make the UI react. The two existing patterns to copy: the mode field the backend returns from FLOWFILE_MODE (consumed in flowfile_frontend/src/renderer/app/services/auth.service.ts), and AI availability signaled by the /ai/* 503 detail string (AI_DISABLED_DETAIL check in app/api/ai.api.ts). Never mint a VITE_* var for deployment config.

8. Quick reference: minimal env per deployment

• Desktop (Tauri): nothing — env.rs supplies everything (mode, ports, storage, supervisor PID).
• Local dev (poetry run flowfile_core + poetry run flowfile_worker): nothing required; defaults to electron mode, ~/.flowfile, ports 63578/63579. Optional: FLOWFILE_SCHEDULER_ENABLED=1 (note: the =on spelling silently does nothing, §5).
• pip unified (flowfile run ui): nothing; import flowfile's side effect sets single-file mode; serves the UI at :63578/ui (host/port hard-checked to 127.0.0.1:63578, flowfile/web/__init__.py:154-157).
• Docker: required JWT_SECRET_KEY, FLOWFILE_INTERNAL_TOKEN, FLOWFILE_MASTER_KEY (or the Docker secret file); recommended FLOWFILE_ADMIN_USER/_PASSWORD; compose wires WORKER_HOST, CORE_HOST, storage dirs, scheduler, projects, and kernel-image overrides for you — check compose defaults against §6 before assuming .env.example alone tells the whole story.

Classification rule (maintainer, 2026-07-03): split every var into installation-specific (paths, hosts, ports — defaulted per machine or wired by env.rs/compose) vs deployment config (secrets, tokens, mode, admin bootstrap, feature gates). Everything in the second group is required-or-relevant for hosting: a server deployment must consciously set them even though local/desktop runs never touch them. .env.example is the hosting operator's checklist — which is why the undocumented-var drift in §6 is a hosting hazard, not a docs nit.

Provenance and maintenance

All facts above were spot-verified by reading the cited files at their cited lines against the repo as of 2026-07-03 (v0.12.7), on a clean checkout. Re-run these before trusting any file:line in this document after a refactor:

# Re-derive the full env-var read list (definitive checklist; compare unique names against §3)
grep -rhoE 'environ(\.get)?\[?\(? ?"[A-Z_0-9]+"|getenv\("[A-Z_0-9]+"' . --include="*.py" -r \
  | grep -oE '"[A-Z_0-9]+"' | tr -d '"' | sort -u

# Re-check every occurrence with file:line (use this to re-verify a specific var's line numbers)
grep -rn --include="*.py" -E "os\.environ|os\.getenv|environ\.get|getenv\(" . \
  --exclude-dir=node_modules --exclude-dir=target --exclude-dir=dist --exclude-dir=.venv --exclude-dir=.git

# TS/JS/Vue env reads (frontend + wasm)
grep -rn --include="*.ts" --include="*.js" --include="*.vue" --include="*.mjs" \
  -E "import\.meta\.env|process\.env" flowfile_frontend flowfile_wasm | grep -v node_modules

# Rust env reads (Tauri sidecar)
grep -rn -E "env::|std::env|option_env!|env!\(" flowfile_frontend/src-tauri/src

# FLOWFILE_MODE default-stamping sites (should be exactly 4 Python + 1 Rust + Dockerfiles/compose)
grep -rn 'FLOWFILE_MODE.*=.*"electron"\|FLOWFILE_MODE.*not in os.environ' \
  flowfile_core flowfile flowfile_frontend/src-tauri/src --include="*.py" --include="*.rs"

# MutableBool truthy-parse sites (confirms §3.3 / §5 truthy rules haven't drifted)
grep -n "MutableBool\|strip().lower() in" flowfile_core/flowfile_core/configs/settings.py

# Admin flip endpoints still mounted under /system, not under the gated router
grep -n "feature_flags" flowfile_core/flowfile_core/ai/admin_routes.py flowfile_core/flowfile_core/lsp/admin_routes.py

# Kernel image defaults + empty-string-is-unset guard
grep -n "_KERNEL_IMAGE.*DEFAULT\|_envvar_or_default" flowfile_core/flowfile_core/kernel/images.py

# Compose vs .env.example divergence re-check (scheduler / projects)
grep -n "FLOWFILE_SCHEDULER_ENABLED\|FLOWFILE_ENABLE_PROJECTS" docker-compose.yml .env.example

# Kernel image version doc-drift re-check
grep -n "flowfile-kernel-base:0\." flowfile_core/flowfile_core/kernel/images.py

If a grep above turns up a line-number shift but the same variable name and semantics, just fix the file:line in this document — don't re-litigate the fact. If a variable disappears entirely, move its row to a "removed" note rather than silently deleting history from this table (this is the config record for the whole monorepo; treat entries as append-mostly).