Wxo Builder

watsonx Orchestrate (wxO): Build · Test · Debug · Publish

Last verified: ADK v2.15.x (Aug 2026). Version-specific items below (server extras, premier-model default, default LLM, UI render flags) are point-in-time; re-verify if orchestrate --version differs. Golden rule: the ADK moves fast. Always verify uncertain flags with orchestrate <group> --help; never rely on memory. Upgrade: pip install -U ibm-watsonx-orchestrate (Python ≥3.11, <3.15). Pre-flight: activate venv (source venv/bin/activate) · always import + test after code generation.

1. Mental Model

2. Setup & Environment

source venv/bin/activate && orchestrate --version

Finding the venv (multi-project workspaces): check ./venv/ first (project root), then ../venv/ (repo root, shared across projects). In scripts, resolve paths relative to the script's directory for portability.

orchestrate targets the active environment (orchestrate env list). Confirm before importing.

# SaaS — use API service URL (contains /instances/<id>), not the console URL
orchestrate env add -n my-saas -u https://api.<region>.watson-orchestrate.cloud.ibm.com/instances/<ID>
orchestrate env activate my-saas --api-key "$IBM_CLOUD_API_KEY"   # auth type auto-inferred (ibm_iam)
orchestrate agents list   # confirm connected
# Auth type override: --type [ibm_iam|mcsp|mcsp_v2|cpd]

# Local Developer Edition (Docker, 16 GB RAM / 8 cores / 25 GB disk, entitlement key in .env)
orchestrate server start -e .env --accept-terms-and-conditions && orchestrate env activate local
# ⚠ Document processing nodes (docproc/docext/docclassifier) require the WDU service — add -d:
# orchestrate server start -d -e .env --accept-terms-and-conditions
# On-prem CPD setup + all env/server flags (--with-voice/-langflow/-ai-builder, extras) → references/cli-reference.md §1, §8

orchestrate env list · orchestrate env activate <name> · orchestrate env remove --name <name>

Keep secrets in a gitignored .env; pass via "$VAR" to stay out of shell history.

3. Canonical Lifecycle

write tools + connections/models/KB → write agent YAML
  → scripts/import-all.sh (connections → models → KB → tools/toolkits → agent)
  → test gate (§7) → debug + re-import → deploy to production (§11)

**Default build assumptions (unless the user says otherwise):**
- If the request says to use a knowledge base, create an actual wxO knowledge base spec plus source documents, import it, wait/check until it is ready, and reference it from the agent YAML with `knowledge_base:`. Do **not** substitute a plain `.txt` file plus custom file-reading tool for a requested knowledge base.
- If the user names a specific model/LLM, use that exact model. If the user does **not** specify one, first inspect the workspace for an existing model convention in nearby agent YAML files or project docs; if none exists, use the skill's documented default LLM. Never pick an arbitrary alternative model when the request is silent.

Project scaffold:

my_agent/
├── agents/         *.yaml
├── tools/          *.py  (one @flow per file; @tool files self-contained)
├── connections/    *.yaml
├── knowledge_base/ *.yaml + source docs
├── models/         *.yaml  (custom models only)
├── tests/
│   └── test_<flow_name>.py   ← one test file per @flow (ONLY create when the project contains a @flow)
│   └── TEST_REPORT.md
├── generated/         ← auto-created by test script; stores compiled flow JSON specs
├── scripts/
│   └── import-all.sh   dependency-ordered imports
│   └── delete-all.sh
└── .env            secrets (gitignored)

tests/test_<flow_name>.py (one per flow):

# tests/test_<flow_name>.py
import asyncio
from pathlib import Path
from tools.<flow_module> import build_<flow_name>   # adjust import to match your flow file

async def main():
    # compile_deploy ONCE — the CompiledFlow object is reusable across invocations
    fdef = await build_<flow_name>().compile_deploy()
    generated_folder = Path(__file__).resolve().parent.parent / "generated"
    generated_folder.mkdir(exist_ok=True)
    fdef.dump_spec(str(generated_folder / "<flow_name>.json"))   # saves compiled spec for inspection

    # For run_flow() implementation and why fdef.invoke()/fdef.flow_run() don't work,
    # see references/testing-debugging.md §4
    result = await fdef.flow_run({"<input_field>": "<test_value>"}, debug=True)
    print("Output:", result.output)
    assert result.output.get("<expected_key>") == "<expected_value>", result.output

if __name__ == "__main__":
    asyncio.run(main())

Rules:

• Always create one tests/test_<flow_name>.py per @flow — required deliverable, not optional. Skip only for tool-only or agent-only projects (no @flow present). ⚠ Exception: flows that use docproc/docext/docclassifier nodes require a live WDU service — skip those flows only.
• Call compile_deploy() once at the top of main(); calling it again raises ValueError: Flow has already been compiled.
• Use the run_flow() helper from references/testing-debugging.md §4; fdef.invoke() and fdef.flow_run() don't work reliably (see reference for why).
• ⚠ Platform pre-processes inputs: the wxO LLM layer silently normalises invalid inputs. Assert against business-state outcomes (mock data that always returns fixed status), not input-validation rejections.
• Run with python -m pytest tests/ -v (set PYTHONPATH=. first).
• The generated/ folder is gitignored.

4. Python Tools (@tool)

from ibm_watsonx_orchestrate.agent_builder.tools import tool, ToolPermission
from pydantic import BaseModel

class WeatherInfo(BaseModel):
    city: str
    temp_c: float

@tool(permission=ToolPermission.READ_ONLY)
def get_weather(city: str) -> WeatherInfo:
    """Get current weather for a city.

    Args:
        city (str): Name of the city.
    Returns:
        WeatherInfo: Temperature in Celsius for the city.
    """
    return WeatherInfo(city=city, temp_c=21.1)

Must-haves: @tool on every callable · Google-style docstring (summary → Args: → Returns:, no blank line between them) · type hints on all params and return · self-contained file (no cross-file local imports) · Pydantic models as explicit classes · never add ibm-watsonx-orchestrate to requirements.txt. ToolPermission valid values: READ_ONLY · WRITE_ONLY · READ_WRITE · ADMIN. WRITE does not exist; use WRITE_ONLY for any tool that mutates state. Docstring/type-hint import warning (Pydantic/dict/list returns): known ADK false positive; imports and runs fine. A real problem only if (1) a blank line sits between Args: and Returns:, or (2) a param lacks a type hint. Full note → references/agents-tools-schemas.md. Credentials — never pass as parameters; declare in expected_credentials and fetch at runtime:

from ibm_watsonx_orchestrate.agent_builder.connections import ConnectionType, ExpectedCredentials
from ibm_watsonx_orchestrate.run import connections

APP_ID = "my_api"

@tool(permission=ToolPermission.READ_ONLY,
      expected_credentials=[ExpectedCredentials(app_id=APP_ID, type=ConnectionType.API_KEY_AUTH)])
def call_api(query: str) -> dict:
    """Call the API.

    Args:
        query (str): Search text.
    Returns:
        dict: API response.
    """
    conn = connections.api_key_auth(APP_ID)   # .api_key · .token · .username/.password · .access_token
    headers = {"Authorization": f"Bearer {conn.api_key}"}

orchestrate tools import -k python -f tools/api_tool.py --app-id my_api

Full decorator signature, ConnectionType values, and Pydantic patterns → references/agents-tools-schemas.md §2.

5. Flows (@flow)

Flow vs. Agent: use @flow for deterministic sequences (BPMN/SOP), agent for LLM-decided order. If you can draw a flowchart → @flow.

Signal	Use @flow	Use agent
Step sequence	Fixed, deterministic	Dynamic (LLM decides)
Source	BPMN diagram / SOP	Open-ended conversation
Data	Structured, typed (Pydantic)	Unstructured / natural language

from pydantic import BaseModel
from ibm_watsonx_orchestrate.flow_builder.flows import Flow, flow, START, END

class MyInput(BaseModel):
    city: str

@flow(name="weather_flow", display_name="Weather Flow",
      description="Fetch and summarise weather", input_schema=MyInput)
def build_weather_flow(aflow: Flow) -> Flow:
    fetch = aflow.tool(get_weather)
    summarize = aflow.prompt(
        name="summarize",
        system_prompt="You format weather data for users.",   # REQUIRED
        user_prompt=["Summarize: {weather}"],
        llm="groq/openai/gpt-oss-120b",
    )
    aflow.sequence(START, fetch, summarize, END)
    return aflow

Must-haves: signature exactly def build_<name>(aflow: Flow) -> Flow: · one flow per file · system_prompt required on aflow.prompt(...) · map_input/map_output single-line Python only · wire with aflow.sequence(START, …, END) or aflow.edge(a, b).

@flow decorator options: name, display_name, description, input_schema, output_schema, schedulable, suppress_agent_summarization (surface the last node's output verbatim; full note in §10 and references/agents-tools-schemas.md §3).

Default LLM: groq/openai/gpt-oss-120b · Node builders: aflow.tool · aflow.prompt · aflow.agent · aflow.script · aflow.foreach · aflow.conditions · aflow.parallel_conditions · aflow.docext · aflow.docclassifier · aflow.docproc · aflow.userflow

Programmatic test (run after every flow change): await build_<flow_name>().compile_deploy() then .invoke(<input_dict>, debug=True). Verify the response is not {}, check every mapped output field, and confirm no An error has occurred in the agent chat response.

Full flow node API → references/agents-tools-schemas.md §3–4.

6. Agent YAML

spec_version: v1                # REQUIRED
kind: native                    # REQUIRED
name: weather_agent             # snake_case, no spaces
description: Returns weather information for a location.   # REQUIRED — drives routing
instructions: |
  You are a helpful weather assistant. When the user asks about weather,
  call get_weather with the city name and present the result clearly.
llm: groq/openai/gpt-oss-120b
style: react_core
tools:
  - get_weather
starter_prompts:                # include 2–4; set `is_default_prompts: false`, each prompt needs `subtitle` & `state: active`
  is_default_prompts: false
  prompts:
    - id: default0
      title: Check weather
      subtitle: ''
      prompt: "What's the weather in Boston?"
      state: active
welcome_content:
  welcome_message: Welcome to the Weather Agent
  description: Ask me about the weather in any city.
  is_default_message: false     # REQUIRED for custom welcome_content to render
# Production extras:
# compaction_settings: {context_compaction_enabled: true, context_compaction_threshold: 20000, compaction_sliding_window: 10}
# llm_config: {temperature: 0, max_tokens: 2048}
# is_schedulable: true   # ⚠ must be enabled at tenant level first

Key constraints: spec_version: v1 + kind: native mandatory · resources listed by name (imported first) · toolkits only for experimental_customer_care · snake_case name.

Multi-agent (collaborators):

style: react_core        # experimental_customer_care does NOT support collaborators
collaborators:           # import/deploy collaborators FIRST
  - dr_wilson
  - dr_cuddy

wxO auto-generates chat_with_collaborator_<name> per collaborator. Routing driven by collaborator's description, so make it distinct. Agent cannot list itself.

Full schema (external/assistant kinds, guidelines, structured_output, chat_with_docs, memory_enabled) → references/agents-tools-schemas.md §1.

7. Import (dependency-ordered)

Import rule (Follow this priority order):

1. import-all.sh exists → run chmod +x import-all.sh && ./import-all.sh.
2. Missing → create it first (connections → models → KB → tools → agent), then run it immediately via execute_command.
3. MCP import tools (import_tool, import_agent, …) → last resort only, when running a script isn't possible. ⚠️ If you just wrote import-all.sh this turn, run it next; do NOT switch to MCP import tools. ⚠️ Keep in sync: any time you add/rename/remove a resource file, update both import-all.sh and delete-all.sh in the same turn before running. Never import a single file manually outside the script.

# import-all.sh template
source venv/bin/activate
orchestrate connections import   -f connections/my_api.yaml
orchestrate models import        -f models/granite.yaml --app-id watsonx_credentials
orchestrate knowledge-bases import -f knowledge_base/kb.yaml
orchestrate tools import -k python -f tools/weather.py
orchestrate tools import -k python -f tools/api_tool.py --app-id my_api
orchestrate tools import -k flow   -f tools/weather_flow.py
orchestrate agents import        -f agents/weather_agent.yaml

-k values: python|openapi|flow|langflow. Use --safe to prompt before overwriting. MCP toolkit: orchestrate toolkits add -k mcp -n <name> --description "…" --package-root ./mcp_server --language node --command '["node","dist/index.js"]' --tools "*" Skills: orchestrate skills import -f SKILL.md --upsert · orchestrate skills import -d skills/ --recursive --upsert · orchestrate skills remove --skill-name <name> (uses --skill-name, not --name)

For cleanup: use orchestrate <resource> remove in reverse import order (agents → tools → kb → models → connections). Always append || true for idempotency:

orchestrate agents remove -n weather_agent --kind native || true
orchestrate tools remove -n weather_flow || true
orchestrate knowledge-bases remove -n my_kb || true
orchestrate models remove -n my_model || true
orchestrate connections remove -a my_api || true

8. Test Gate (verify before handover)

Deployed ≠ verified. Never declare "done" until tested, or the human explicitly declines.

After ./import-all.sh:

Step 1 — Unit-test flows: create one tests/test_<flow_name>.py per flow (stub the SDK, test tool logic, Pydantic schemas, and flow wiring). Execute pytest now — do not proceed to Step 2 until all tests pass:

source venv/bin/activate && PYTHONPATH=. python -m pytest tests/ -v

If it fails: fix flow → ./import-all.sh → re-run until all pass. Step 2 is blocked until this is green.

Step 2 — Agent smoke-test: Ask:

"<agent> is deployed to <env>. Want me to smoke-test it? I'll run 1 single-turn + 1 multi-turn — read-only prompts." — Yes / No

Execute (preferred): watsonx-orchestrate-adk:chat_with_agent; Turn 1 with include_reasoning=True, save thread_id, Turn 2 with same thread_id. CLI fallback: orchestrate chat ask -n <agent> "<prompt>" -r (⚠ can hang on SaaS; use runtime REST API from references/runtime-api.md instead).

Pass criteria: no error · correct output · expected tool fired (check reasoning) · turn 2 uses context from turn 1.

Fix loop (any failed test): read root cause from reasoning → fix .py/.yaml → update import-all.sh/delete-all.sh if files changed → ./import-all.sh → re-run with chat_with_agent. Repeat until all pass.

Emit TEST_REPORT.md: "deployed and tested (2/2)" · "deployed; test N failed — <reason>" · "deployed; not tested at your request."

After testing, always tell the user how to test manually:

• UI — open the agent in the wxO web UI, use starter prompts or type directly.
• CLI — orchestrate chat ask -n <agent_name> "<prompt>" -r (-r = reasoning trace; ⚠ can hang on SaaS — use chat_with_agent MCP tool instead).
• Bob — "Chat with <agent_name>: <prompt>" — Bob calls chat_with_agent with include_reasoning=True.

Provide 3–5 sample prompts covering: happy path · edge/unknown input · multi-turn. For each, state the expected output so the user knows what a pass looks like.

Full gate procedure + report template + pre-publish checklist → references/testing-debugging.md.

9. Connections, Models, Knowledge Bases

Connection YAML:

spec_version: v1
kind: connection       # singular — NOT 'connections'
app_id: my_api
environments:
  draft:
    security_scheme: api_key_auth   # NOT 'kind:' — must be 'security_scheme:'
    type: team                      # team (shared) | member (per-user)
    server_url: https://api.example.com

security_scheme values: basic_auth · bearer_token · api_key_auth · oauth2 · key_value_creds. OAuth2: use oauth2_auth_code, not authorization_code. YAML defines structure only; never hardcode secrets.

orchestrate connections import -f connections/my_api.yaml
orchestrate connections configure -a my_api --kind api_key --type team --env draft
orchestrate connections set-credentials -a my_api --env draft --api-key "$MY_API_KEY"

Models: orchestrate models list to see available IDs. orchestrate models list --all to show all including disabled. Premier models disabled by default in 2.13+; check with orchestrate models config are-premier-models-enabled. Custom watsonx.ai model: create a watsonx_credentials key-value connection + kind: model YAML → orchestrate models import --app-id watsonx_credentials.

LLM selection rule: if the user specifies a model, use it. If not, inspect existing project agent YAML files / docs for the established model first; only fall back to the skill default when no project convention exists.

Knowledge bases: built-in Milvus is the default (no infra); external AstraDB / Milvus / Elasticsearch use provider blocks, anything else (Pinecone etc.) a custom @tool. If the request says to use a knowledge base, create the KB resource itself (kind: knowledge_base) and attach source documents to it; a local .txt file read by a tool is not an acceptable substitute for a requested KB. Decision tree + provider schemas → references/connections-models-kb.md §3.

orchestrate knowledge-bases import -f kb.yaml
orchestrate knowledge-bases status -n product_docs   # watch indexing

Reference in agent YAML: knowledge_base: [product_docs]

Full schemas → references/connections-models-kb.md.