Add Subworkflow

Add Subworkflow

Scaffold a Bactopia subworkflow that orchestrates one or more existing modules. Subworkflows are glue -- they wire modules together, aggregate results, and provide a clean interface for workflows.

Prerequisites

• The module(s) this subworkflow will use must already exist under modules/
• Read .agents/docs/standards/04-subworkflow-documentation.md for documentation standards

Interactive Questioning

This skill is interactive -- ask the user early and often, especially before creating files.

• Multiple questions at once: Use AskUserQuestion popups (up to 4 questions per batch). Mark the recommended option with "(Recommended)" at the end of its label and place it first.
• Single simple question: Just ask in chat, no popup needed.
• When in doubt: Ask. It's cheaper to clarify upfront than to regenerate files.

What a Subworkflow Contains

A subworkflow is a single main.nf file plus tests:

subworkflows/{tool}/
    main.nf              # Workflow definition with GroovyDoc
    tests/
        main.nf.test     # nf-test specification
        main.nf.test.snap # Snapshot (generated by nf-test)
        nextflow.config  # Includes ALL module.configs used by this subworkflow
        nf-test.config   # Standard nf-test config
        .nftignore       # Exclude unstable files from snapshots

No module.config or schema.json -- those belong to modules only.

Phased Workflow

Phase 1: Gather Information

Goal: Determine which modules to orchestrate and how, using interactive prompts.

Important: Use the AskUserQuestion tool for structured choices throughout this phase. Present up to 4 questions per batch. Mark the recommended option with "(Recommended)" at the end of its label and place it first in the options list.

1. Ask the user which modules this subworkflow uses. Get the module paths (e.g., modules/nohuman/run, modules/mlst).

   • Read each module's main.nf to understand its inputs and outputs.
   • Does it call other subworkflows? If so, use @subworkflows tag (not @modules) for those includes.

2. Run the lookup command if package info is needed:

   bash .agents/skills/add-bactopia-tool/scripts/run-bactopia-scaffold.sh lookup {package_name} --bactopia-path . --pretty

3. Determine the input type from the primary module's inputs, then run test-data discovery:

   bash .agents/skills/add-bactopia-tool/scripts/run-bactopia-scaffold.sh test-data --input-type {input_type} --bactopia-path . --pretty

   This returns species/accession combinations already used by similar modules, with pre-computed test_data_path, test_uncompressed_path, test_species, and test_sample_id values. Use the returned paths directly in the scaffold config (Phase 2) -- do NOT construct paths manually. Subworkflow tests use the compressed path (test_data_path).

4. Batch 1: Design choices (AskUserQuestion, up to 3 questions)

   Question 1 -- Aggregation strategy:

   • CSVTK_CONCAT -- concatenate per-sample tabular output (most common) (Recommended)
   • Dedicated summary module -- tool has its own aggregation command (rare)
   • No aggregation -- tool doesn't produce per-sample tabular output

   Question 2 -- Test data species: Present top 3-4 species from the test-data discovery results. Recommend species that exercise the tool's functionality. Include the accession in each option's description.

   Question 3 -- Aggregation field (if CSVTK_CONCAT or dedicated_summary selected): Which output field should be aggregated? (e.g., tsv, report, csv) What format? (tsv or csv)

5. Present the following for confirmation (derive from module main.nf and lookup):

   • Tool identity: name (snake_case), display name, one-sentence description
   • Outputs: from the primary module's output block (for @output GroovyDoc tags)
   • Citation key and keywords for GroovyDoc

6. Final confirmation (AskUserQuestion, 1 question)

   After presenting the summary, ask:

   • Looks good, proceed to file generation
   • I need to make changes (user provides details via "Other" or notes)

Phase 2: File Generation

Goal: Generate the 5 subworkflow files using bactopia-scaffold.

1. Construct the JSON config from the design decisions. Write it to /tmp/scaffold-config.json:

   {
       "tool": "{tool_name}",
       "display_name": "{DisplayName}",
       "description": "{One-sentence description}",
       "process_name": "{TOOL_NAME}",
       "package": "{package_name}",
       "version": "{version}",
       "build": "{build}",
       "home_url": "{github_url}",
       "input_type": "assembly",
       "has_database": false,
       "handles_gz": false,
       "layout": "flat",
       "resource_label": "process_low",
       "version_command": "{version_command}",
       "citation_key": "{citation_key}",
       "keywords": ["{keyword1}", "{keyword2}"],
       "aggregation": {
           "strategy": "csvtk_concat",
           "field": "{output_field}",
           "format": "{tsv|csv}"
       },
       "outputs": [
           {"name": "{field}", "extension": "{ext}", "description": "{desc}"}
       ],
       "parameters": [],
       "container_refs": {
           "toolName": "{from lookup}",
           "docker": "{from lookup}",
           "image": "{from lookup}"
       },
       "test_species": "{species}",
       "test_sample_id": "{sample_id}",
       "test_data_path": "{compressed_path}",
       "test_uncompressed_path": "{uncompressed_path}"
   }

   For database-dependent subworkflows, also include:

   {
       "database": {
           "param_name": "{tool}_db",
           "test_path": "datasets/{tool}/{db_file}"
       }
   }

2. Run the scaffold command:

   bash .agents/skills/add-bactopia-tool/scripts/run-bactopia-scaffold.sh subworkflow --config /tmp/scaffold-config.json --bactopia-path . --pretty

3. The command creates 5 files:

   • subworkflows/{tool}/main.nf
   • subworkflows/{tool}/tests/main.nf.test
   • subworkflows/{tool}/tests/nextflow.config
   • subworkflows/{tool}/tests/nf-test.config
   • subworkflows/{tool}/tests/.nftignore