Agentic Workflow Advisor

Agentic Workflow Advisor

Purpose

This skill defines the process for analyzing IBM watsonx Orchestrate agentic workflow artefacts (JSON or Python @flow) and generating a structured, prioritised set of architecture recommendations that help builders identify and fix design issues before they reach production.

The skill performs static analysis only — it does not invoke, modify, or connect to any running workflow or platform API. It works entirely from the artefact the builder provides.

Use this skill when you need:

• A prioritised list of architecture issues before deploying a workflow to production
• Performance diagnosis for flows with slow response times or high latency
• A repeatable design review that does not require engineering escalation
• Recommendations grounded in real platform behaviour and measured latency figures

Objective

Analyze an agentic workflow artefact and produce a structured recommendation report that:

• Identifies architecture patterns known to cause performance problems, routing failures, or maintainability risks
• Prioritises findings by impact — High, Medium, and Low
• Provides a specific, actionable recommended action for each finding
• Is specific to the artefact provided — not generic advice

Scope

This skill applies to:

• Any workflow JSON artefact exported from the IBM watsonx Orchestrate Flow Builder UI
• Any workflow JSON artefact authored via the ADK pro-code path
• Any Python flow file authored via the ADK pro-code path using the @flow decorator
• Agent YAML definitions (optional — used for Check 3 agent node analysis)

In scope:

• All watsonx Orchestrate flows — any JSON with "spec": { "kind": "flow", ... }, or any Python file with a @flow-decorated function, regardless of whether they contain an Agent node
• All supported node types: tool, agent, script, branch, parallel, foreach, loop, prompt, docproc, docext, decisions, user, user_flow
• Top-level flow structure and the internals of container nodes (parallel, foreach, loop, user_flow)

Python @flow artefacts: When the artefact is a Python file using the ADK @flow decorator, the same 6 checks apply. Run scripts/extract_flow_info.py <file.py> to pre-extract the structural summary before analysis (see Step 1).

Out of scope:

• Agent definitions beyond their YAML (model selection, RAG configuration, knowledge base content)
• Python toolkit source code beyond identifying whether a tool makes external calls
• Automated remediation — this skill identifies and explains issues; it does not rewrite the workflow
• Performance benchmarking or load testing guidance

Analysis Principles

IMPORTANT: Generate findings based ONLY on what is present in the provided artefact.

Rules:

• Reference actual display_name values from the artefact — never use internal node IDs like tool_562086
• Do not flag issues you cannot point to directly in the JSON
• Do not fabricate recommendations to appear thorough — a clean result is a valid and useful output
• Use measured latency figures from the platform: explicit mapping <10ms, auto-mapping 500–3000ms, context compression 1000–5000ms (triggers at >6500 tokens), per-task transition ~35ms, agent context retrieval ~500ms, Generative Prompt 500–3000ms, agent ReACT loop 2–10s
• Distinguish between definite findings and candidates (e.g. Check 5 is a candidate based on description, not a certainty)
• If an artefact is partially incomplete (metadata.is_under_specified: true), note this and scope findings accordingly
• Output only what the 6 checks produce. Do not add freeform observations, "worth noting" sections, commentary, or supplementary advice outside the check findings. If the checks produce no findings, the output is the clean result message — nothing else.

Platform-managed fields — do NOT flag or comment on:

• assignees — this field on user_flow nodes is populated automatically by the platform (defaulting to the flow initiator). It does not require an explicit input_map. Never suggest mapping assignees to a user list.
• Any other field whose absence of an input_map is intentional platform behaviour — if in doubt, do not flag it.

Excluded from this assessment:

• Non-functional requirements beyond what is detectable statically (throughput, concurrency, load behaviour)
• Business logic correctness — the skill assesses architecture patterns, not whether the flow does the right thing
• Security review of credentials, API keys, or data handling beyond what is visible in the JSON structure
• Completeness gaps on fields not covered by the 6 checks — missing input_map on a user_flow node is not an architecture issue detectable by this skill

Core Principle

Evaluate the workflow artefact for what it will do at runtime — not for how it looks on the canvas. A flow that appears clean may have sequential tool nodes with no dependencies, auto-mapped fields invoking an LLM on every call, or an agent used purely to route a result that is already deterministic. These patterns are invisible in the UI and only surface under load or in production. This skill makes them visible before deployment.

Workflow JSON Structure Reference

Before running analysis, understand the structure of a watsonx Orchestrate workflow JSON export:

{
  "spec": { "kind": "flow", "name": "...", "display_name": "...", "private_schema": { ... } },
  "nodes": {
    "<node_name>": {
      "spec": {
        "kind": "<node_kind>",
        "name": "...",
        "display_name": "...",
        "input_schema": { "type": "object", "required": [...], "properties": { ... } },
        "output_schema": { ... }
      },
      "input_map": {
        "spec": {
          "fallback_strategy": "ask_the_user" | "no_value",
          "maps": [
            {
              "metadata": { "assignmentType": "variable" | "literal" | "pyExpression" | "automap" },
              "target_variable": "self.input.<field>",
              "value_expression": "...",
              "has_no_value": false | true
            }
          ]
        }
      },
      "edges": [...],
      "nodes": { ... }
    }
  },
  "edges": [
    { "id": "...", "start": "<node_name>", "end": "<node_name>", "display_name": "..." }
  ],
  "metadata": { "source_kind": "ui" | "adk", "is_under_specified": true | false }
}

Key structural facts:

• Mappings live at node.input_map.spec.maps[] — not at data_map
• fallback_strategy lives at node.input_map.spec.fallback_strategy
• A required field is auto-resolved by the runtime if it has no entry in maps[] — detection is purely structural (gap between input_schema.required[] and maps[].target_variable)
• Empty maps: [] means all required fields fall through to fallback_strategy
• script nodes ("kind": "script") ARE Logic Code Blocks — inline fn expression, no external calls
• Container nodes (parallel, foreach, loop, user_flow) have their own inner edges and nodes
• Nodes inside a parallel container are already concurrent — do not flag them for sequential execution

"kind": "tool" means exactly that — a tool node. Do NOT infer what lies behind it (MCP, OpenAPI, Python, sub-flow, or anything else) from the node's name, display_name, or description. Treat every tool node uniformly for all detection checks.

Supported node kinds: tool | agent | script | branch | parallel | foreach | loop | prompt | docproc | docext | decisions | user | user_flow | start | end

Analysis Workflow

Step 1 — Collect the Artefact

If the user has not yet provided a workflow artefact in this message, reply with exactly this:

"Please paste your workflow JSON or provide the file path to analyze."

Then stop. Do nothing else.

STOP. The following are strictly forbidden before the user provides content in this message:

• Do NOT explore, list, or search the workspace or any directory
• Do NOT read any file from disk — not .bob/tmp/, not resource/, not anywhere
• Do NOT use any previously seen files from earlier conversations or sessions
• Do NOT assume any file in the workspace is the intended target
• The ONLY valid input is content the user explicitly provides in the current message: either pasted JSON/Python, or a file path they typed in this message

The artefact must come from the user in this conversation turn. Nothing else is acceptable.

Valid inputs (once the user provides one):

1. A file path the user typed (e.g. resource/MyFlow/MyFlow.json) — fastest, Bob runs the script directly on it
2. Pasted raw JSON or Python @flow content — Bob saves to temp file and runs the script
3. Agent YAML — optional, used for Check 3 agent node analysis

If only partial artefacts are provided, proceed with what is available and clearly state at the end of the review what could not be assessed.

Step 2 — Pre-flight: Determine Analysis Path

Do NOT ask the user any question.

Path A — User provided a file path (e.g. resource/MyFlow/MyFlow.json):

Run exactly this one command and nothing else:

python3 .bob/skills/agentic-workflow-advisor/scripts/extract_flow_info.py <file_path>

Use python3. If not found, try python. Use the full path to the script relative to workspace root.

This single command produces all data needed for all 6 checks. Run it once. That is all.

STRICTLY FORBIDDEN for Path A — these are never acceptable under any circumstances:

• Do NOT run python3 -c "..." inline scripts to read or inspect the JSON file
• Do NOT run multiple commands to gather evidence for individual checks
• Do NOT read the JSON file directly with read_file or any other tool
• Do NOT run any additional commands after the script — the script output is complete and sufficient
• Do NOT re-run the script for individual checks

If the script output seems incomplete, proceed with what it produced — do not supplement with ad-hoc commands.

After the script runs once, map its output keys to checks using the table at the end of this step, then proceed to Step 3.

Path B — User pasted raw JSON content:

Do NOT write the JSON to any file. Do NOT use write_file. Do NOT run execute_command. Do NOT use a heredoc. Do NOT re-emit the JSON content in any tool call. Re-emitting thousands of lines of JSON through any tool takes 10+ minutes. It must never happen.

Apply the extraction algorithm (E0–E3) below against the JSON in context. Then go directly to Step 4 output.

OUTPUT RULE — strictly enforced: The first token Bob outputs to the user must be ## (the report header: ## Agentic Workflow Advisor — [flow name]). Nothing else comes before it — no sentence, no label, no acknowledgement. Do not say "Running…", "Analysing…", "Working through…", "E0–E3…", "silently…", "against the pasted JSON", or any other intro phrase. The report header is the very first thing the user sees.

Everything from the report header onward is shown normally: flow name, execution path, findings, review scope line.

Extraction Algorithm (E0–E3 — internal only, never output)

E0 — Resolve schemas

• Top-level schema: data.spec.input_schema or data.spec.private_schema
• If that value is {"$ref": "#/schemas/SomeName"}, resolve it: top_input_schema = data.schemas["SomeName"]
• Otherwise use the value inline
• top_properties = keys of top_input_schema.properties (or [])
• top_required = top_input_schema.required (or [])
• flow_name = data.spec.display_name or data.spec.name

E1 — Walk all nodes Iterate data.nodes (top-level dict, keys are node IDs). For each node:

• kind = node.spec.kind
• display_name = node.spec.display_name or node_id
• description = node.spec.description (or "")
• input_schema_required = node.spec.input_schema.required (or [])
• input_schema_properties = keys of node.spec.input_schema.properties (or [])
• explicitly_mapped_fields = all target_variable values in node.input_map.spec.maps[] that start with "self.input.", with the "self.input." prefix stripped
• unmapped_required_fields = fields in input_schema_required NOT in explicitly_mapped_fields
• phantom_mapped_fields = fields in explicitly_mapped_fields NOT in input_schema_properties
• fallback_strategy = node.input_map.spec.fallback_strategy (or "")
• If kind == "agent": also capture message = node.spec.message, thread_control_policy = node.spec.thread_control_policy, output_schema_properties = keys of node.spec.output_schema.properties
• If kind == "branch": also capture branch_conditions = node.spec.evaluator.conditions[] → each as {expression, default, edge_id}
• If kind == "parallel": also capture parallel_children = entries in node.nodes{} where kind is NOT "start" or "end" → each as {id, kind, display_name}; parallel_is_conditional = any condition in node.spec.evaluator.conditions[] where default != true
• If kind is "start" or "end": skip (internal plumbing)
• For nodes with sub-nodes (e.g. parallel): also walk node.nodes{} with the same rules above

E2 — Build edge maps From data.edges[]:

• successors[start_id] = list of end_id values
• predecessors[end_id] = list of start_id values

E3 — Derive check inputs (populate these named results for Step 3):

check_1_sequential_tool_pairs: For each node where kind == "tool", for each successor where kind == "tool":

• Look at successor's node.input_map.spec.maps[] → collect all value_expression strings
• If any value_expression contains the current node's display_name, name (node.spec.name), or node_id → has dependency → skip
• Otherwise → no dependency → add {node_a: display_name, node_b: successor_display_name}

check_1_parallel_containers: All nodes where kind == "parallel" → {display_name, children: parallel_children, is_conditional: parallel_is_conditional}

check_2_unmapped_required_fields: All nodes where unmapped_required_fields is non-empty AND kind is NOT one of start, end, branch, parallel, foreach, loop, user_flow → {display_name, kind, unmapped_required_fields, fallback_strategy}

check_2_phantom_mappings: All nodes where phantom_mapped_fields is non-empty → {display_name, kind, phantom_mapped_fields}

check_3_branch_nodes: All nodes where kind == "branch" → {display_name, conditions: branch_conditions}

check_3_6_agent_nodes: All nodes where kind == "agent" → {display_name, message, thread_control_policy, output_schema_properties, predecessors: [{display_name, kind}] from predecessors map}

check_4_input_schema_field_count: len(top_properties)

check_5_tool_nodes: All nodes where kind == "tool" → {display_name, description}

node_kinds_present: sorted unique set of all kind values across all nodes

is_under_specified: data.metadata.is_under_specified (default false)

Script output keys (Path A — map script output directly to checks):

• check_1_sequential_tool_pairs → Check 1 candidates
• check_1_parallel_containers → Check 1 exemptions (children already concurrent)
• check_2_unmapped_required_fields → Check 2 unmapped field findings
• check_2_phantom_mappings → Check 2 phantom mapping findings
• check_3_branch_nodes → Check 3 branch condition evidence
• check_3_6_agent_nodes → Check 3 (predecessors) and Check 6 (message, tools, output_schema)
• check_4_input_schema_field_count → Check 4 field count
• check_5_tool_nodes → Check 5 tool descriptions
• node_kinds_present → which checks will produce findings
• is_under_specified: true → note at top that flow is incomplete, findings may be partial

Step 3 — Run All 6 Detection Checks

Work through every check below. Collect all findings before composing the output — do not output findings one by one.

Check 1 — Sequential Tool Execution 🔴 High Impact

What to look for: Two or more "kind": "tool" nodes connected sequentially in the top-level edges array with no data dependency between them.

How to detect:

1. Build the execution sequence from the top-level edges array
2. Find consecutive runs of tool nodes (skip user_flow, script, branch, prompt, parallel, etc.)
3. For each consecutive tool pair (A → B), check node_B.input_map.spec.maps:
   • If any map entry has a value_expression referencing node A's output (e.g. contains flow["<node_A_display_name>"] or flow.<node_A_name>), there IS a dependency — do not flag
   • If no map entry references node A, there is NO dependency — flag this pair
4. Skip nodes already inside a parallel container — they are intentionally concurrent

Flag when: 2 or more consecutive top-level tool nodes share no data dependency.

Recommendation:

Sequential tool execution detected: [list node display_names]. These tool nodes execute one after the other but share no data dependency. Wrap them in a Parallel Node — in the Flow Builder, add a Parallel Node container and move these tool nodes inside it. The Parallel Node runs all contained nodes at the same time, so the total wait time becomes the duration of the slowest node rather than the sum of all nodes. Typical saving: 2–4s for tools averaging 1–2s each, plus 35ms orchestration overhead per eliminated sequential step.

Check 2 — Unmapped Required Input Field 🟡 Medium Impact

What to look for: Any node where input_schema.required[] lists fields that have no explicit entry in input_map.spec.maps — meaning the runtime must resolve the value another way. Also covers the flow-level output_map if output fields are defined but not explicitly mapped. Applies to all node kinds that have an input_schema, including tool, agent, prompt, docproc, and docext.

Context — automap is a supported feature with a known performance cost: Auto-mapping is valid and intentional — the runtime uses an LLM to resolve unmapped required fields from available context. However, it always costs 500–3000ms per invocation regardless of whether it was chosen deliberately or left as a default. This check always reports unmapped required fields with those numbers so the builder can make an informed decision. If explicit mapping is not feasible or the latency is acceptable for the use case, there is no need to act.

How to detect:

1. For each node that has both input_schema.required and input_map:
   • Collect required fields from input_schema.required[]
   • Collect explicitly mapped targets from input_map.spec.maps[].target_variable (extract field name after self.input.)
   • Any required field not covered by an explicit map entry will be resolved by the runtime at execution time
2. Also check the flow-level output_map.spec.maps[] — only if output_map.spec.maps is non-empty (i.e. the builder has defined at least one output field). If any output field has no explicit mapping source, flag it. If output_map.spec.maps is empty or absent, skip — there are no output fields to evaluate.
3. Phantom mapping check — for each input_map.spec.maps entry, extract the field name from target_variable (after self.input.) and verify it exists as a key in input_schema.properties. If it does NOT exist in input_schema.properties, it is a phantom mapping: the value is assigned to a field the node does not declare, so it is silently discarded at runtime. This is a copy-paste error signal — flag it regardless of performance intent. Common pattern: a field is renamed in the node spec but an old mapping with the previous name was not removed, resulting in both a phantom entry (old name) and a potentially missing entry (new name, which may or may not have a separate correct mapping).

Flag when: Any required field on a node has no explicit mapping (performance-mode recommendation), OR any input_map.spec.maps entry targets a field name not present in input_schema.properties (phantom mapping — always flag).

Recommendation (node input — unmapped required field):

Unmapped required field on [display_name]: required field(s) [list field names] have no explicit mapping in input_map. The runtime will use an LLM to resolve each unmapped field at execution time — this costs 500–3000ms per invocation regardless of how many fields are unmapped. Explicit mapping costs <10ms. If this latency is acceptable for the use case, no action is needed; otherwise add explicit mappings under input_map.spec.maps for each field.

Recommendation (flow output_map — unmapped output field):

Unmapped flow output field: output field(s) [list field names] in output_map have no explicit mapping source. The runtime will use an LLM to resolve these at execution time — this costs 500–3000ms per invocation. If this latency is acceptable, no action is needed; otherwise add an explicit data mapping pointing to the node output that produces each value.

Recommendation (phantom mapping):

Phantom mapping on [display_name]: input_map entry targeting self.input.[field_name] does not match any field in input_schema.properties. This value is silently discarded at runtime — the node never receives it. This is typically a copy-paste error where a field was renamed in the node spec but the old mapping was not removed. Remove the phantom entry and verify the intended field has a correctly named mapping. Cross-check input_schema.required to confirm no required field is inadvertently left unmapped as a result.

Check 3 — LLM-Based Routing over Deterministic Result 🔴 High Impact

What to look for: An agent node that exists solely to route based on an already-deterministic result — where a branch node or tool could make the same decision without an LLM call.

How to detect:

1. For each agent node, check what node directly precedes it in the top-level edges
2. If the preceding node is decisions, docproc, docext, or a tool that produces a categorical/classification output (e.g. feature_match_tool returning is_match + feature_match), inspect the agent's message field for routing language ("route", "call", "invoke", "initiate", "based on", "depending on")
3. Check what follows the agent — if it goes directly to branch, __end__, or another flow invocation, this is a routing-only agent
4. An agent node in a flow whose sole output feeds into a branch evaluator is a strong signal
5. Also applicable at the top-level agent YAML level: if the agent's instructions describe routing exclusively via a variable like {llm_instruct} or {feature_match} that was already computed by an upstream tool, this is LLM-based routing over a deterministic result — the agent adds latency without adding intelligence

Do NOT flag:

• Agent nodes doing genuine NLP work — semantic understanding, free-text matching from scratch, knowledge base search (RAG), or multi-turn reasoning. If the agent is performing the matching/classification itself (not receiving a pre-computed result), it is doing legitimate LLM work.
• An agent that receives a free-text user query and must determine the intent from scratch is NOT routing a deterministic result — it IS the decision-maker.

Signal that separates the two:

• 🔴 Flag: agent receives a pre-computed category string (e.g. feature_match = "Block_Card_Flow") and passes it to another flow. The decision was already made.
• ✅ Do not flag: agent receives a raw user query and must determine intent, then act. The decision has NOT been made upstream.

Flag when: An agent exists solely to route based on a deterministic upstream result (the decision was already made before the agent was called).

Recommendation:

LLM-based routing detected: agent [display_name] appears to route based on the output of [upstream display_name], which already produces a deterministic result. Passing this through an agent ReACT loop adds 2–10s per turn and introduces routing failures due to LLM variability. A Branch node evaluates the condition in <10ms with no LLM call. Replace with a Branch node — set the condition to evaluate the upstream result directly (e.g. flow["<upstream>"].output.value == "category_x"), eliminating the agent call entirely.