Agent skill

Configuring Airflow Language Sdks

by astronomer in astronomer/agents

Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options.

Apache-2.0Auto-check: notesData & Analytics

Install Configuring Airflow Language Sdks

skills CLI
$ npx skills add astronomer/agents --skill configuring-airflow-language-sdks -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install astronomer/agents configuring-airflow-language-sdks --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/astronomer/agents.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/configuring-airflow-language-sdks .claude/skills/configuring-airflow-language-sdks && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
configuring-airflow-language-sdks
GitHub stars
451
Token cost
~2.3k tokens
SKILL.md length
972 words
Files
1
Skills in repo
34
Repo updated
First seen
Licence
Apache-2.0

At a glance

Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options.

  • Works in 2 steps: coordinators — a JSON object mapping a… → queue_to_coordinator — a JSON object…
  • The user wants Airflow to route a queue to a native-language coordinator
  • SKILL.md covers Prerequisites on the worker, The two settings, Per-coordinator options and Verifying the configuration, plus 1 more section
  • Calls java and docker

What it does

Configuring Airflow Language Sdks is an agent skill from astronomer/agents. Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options. Use when the user wants Airflow to route a queue to a native-language coordinator, asks about the [sdk] coordinators/queuetocoordinator settings, AIRFLOWSDKCOORDINATORS, jarsroot, executablesroot or other coordinator kwargs, taskstartuptimeout, or why their native tasks aren't being picked up. Covers the shared routing…

Its SKILL.md is about 2.3k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Data & Analytics, covering Data pipelines and ETL. It works with Apache Airflow and Java. The repository describes itself as: AI agent tooling for data engineering workflows. The licence is Apache-2.0.

When your agent uses it

  • The user wants Airflow to route a queue to a native-language coordinator
  • Asks about the [sdk] coordinators/queuetocoordinator settings
  • AIRFLOWSDKCOORDINATORS
  • Executablesroot

Example prompts

  • “Use the configuring-airflow-language-sdks skill to configure Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a…”
  • “/configuring-airflow-language-sdks”

Requirements

  • Python 3
  • Docker

Workflow steps

2 steps, taken from the first numbered list in SKILL.md.

  1. coordinators — a JSON object mapping a coordinator name you choose to its implementation (classpath) and constructor kwargs.
  2. queue_to_coordinator — a JSON object mapping a task queue to a coordinator name.

What it can do on your machine

Read from SKILL.md and the folder at commit 486ee63. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • java
    • docker

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use docker, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Configuring Airflow Language Sdks loads about 2.3k tokens when it runs. Until then it costs about 165 tokens; SKILL.md has 972 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~165
When it runs · the whole SKILL.md, loaded when a task matches
~2.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check: notes

The automated check noted patterns worth knowing about, such as sudo or a known installer.

  • NoteMentions a .env fileSKILL.md:52
    This form is convenient for containers, `.env` files, Docker Compose, and Helm.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from astronomer/agents at commit 486ee63, republished under its Apache-2.0 licence (© astronomer). 972 words, ~2,345 tokens.

Download SKILL.mdSave it as .claude/skills/configuring-airflow-language-sdks/SKILL.md (or your agent's skills folder).
name
configuring-airflow-language-sdks
description
Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options. Use when the user wants Airflow to route a queue to a native-language coordinator, asks about the `[sdk]` `coordinators`/`queue_to_coordinator` settings, `AIRFLOW__SDK__COORDINATORS`, `jars_root`, `executables_root` or other coordinator `kwargs`, `task_startup_timeout`, or why their native tasks aren't being picked up. Covers the shared routing mechanism plus per-coordinator options (e.g. JavaCoordinator, ExecutableCoordinator).

Configuring Airflow for Language SDKs

To run language SDK tasks, Airflow needs to know two things: which coordinator launches the native subprocess, and which queue routes to that coordinator. The mechanism is identical across every language SDK — only each coordinator's classpath and kwargs differ. This skill documents the shared wiring once, then the per-coordinator options. It is platform-neutral: the same settings apply on open-source Airflow and on managed platforms like Astro.

Experimental. The language SDKs are in preview; configuration keys may change.

For the task code, see authoring-language-sdk-tasks (and the per-language authoring skill, e.g. authoring-java-sdk-tasks, authoring-go-sdk-tasks). For building and shipping the artifact, see the per-language deploy skill (e.g. deploying-java-sdk-bundles, deploying-go-sdk-bundles).


Prerequisites on the worker

  • The runtime or artifact the SDK needs must be present on the worker nodes, because the coordinator spawns a native subprocess per task instance. The exact requirement is per-SDK — see Per-coordinator options (the Java SDK needs a JRE 17+; the Go SDK needs no language runtime — the bundle is a self-contained native executable, but it must be built for the worker's OS/arch).
  • The compiled/native artifact(s) must be reachable on the worker. See the per-language deploy skill.
  • The coordinators ship with the Airflow Task SDK (apache-airflow-task-sdk, installed with Airflow). No extra Python package is required.

The two settings

Both live in the [sdk] configuration section and apply to every language SDK:

  1. coordinators — a JSON object mapping a coordinator name you choose to its implementation (classpath) and constructor kwargs.
  2. queue_to_coordinator — a JSON object mapping a task queue to a coordinator name.

A task whose stub sets queue="..." is handed to the named coordinator, which launches the native subprocess. The coordinator name is arbitrary — it just has to be the same string in both settings. The queue name must match the queue= set on the Python @task.stub.

Option A: airflow.cfg
ini
[sdk]
coordinators = {
  "java-jdk17": {
    "classpath": "airflow.sdk.coordinators.java.JavaCoordinator",
    "kwargs": {"jars_root": ["/opt/airflow/jars"]}
  },
  "go": {
    "classpath": "airflow.sdk.coordinators.executable.ExecutableCoordinator",
    "kwargs": {"executables_root": ["/opt/airflow/executable-bundles"]}
  }
}
queue_to_coordinator = {"java": "java-jdk17", "golang": "go"}
Option B: environment variables

Each value must be valid one-line JSON. This form is convenient for containers, .env files, Docker Compose, and Helm.

bash
export AIRFLOW__SDK__COORDINATORS='{"java-jdk17": {"classpath": "airflow.sdk.coordinators.java.JavaCoordinator", "kwargs": {"jars_root": ["/opt/airflow/jars"]}}, "go": {"classpath": "airflow.sdk.coordinators.executable.ExecutableCoordinator", "kwargs": {"executables_root": ["/opt/airflow/executable-bundles"]}}}'
export AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"java": "java-jdk17", "golang": "go"}'

The examples above register multiple coordinators at once (one per language) and map a different queue to each — register only the ones you use.


Per-coordinator options

The classpath and kwargs are specific to each coordinator. Add a subsection here as new language SDKs land.

JavaCoordinator
  • classpath: airflow.sdk.coordinators.java.JavaCoordinator
  • Worker runtime: JRE 17+ (java on PATH, or set java_executable).
ParameterDefaultDescription
jars_root(required)One or more directories scanned recursively for .jar files. Accepts a string or a list of strings/paths. The classpath is assembled automatically.
java_executable"java"Path to the java binary. Defaults to java on $PATH.
jvm_args[]Extra JVM arguments, e.g. ["-Xmx1g", "-Dsome.property=value"].
main_class(auto-detect)Explicit entry-point class. If omitted, the coordinator scans jars_root for a JAR whose manifest declares Main-Class. Set this explicitly if multiple executable JARs are present — otherwise the choice is non-deterministic.
task_startup_timeout10.0Seconds to wait for the subprocess to connect after launch. Increase it if JVM startup is slow (constrained hardware, large classpath, first cold start).

Java logging via java.util.logging. Of the SDK logging integrations, only JPL and SLF4J are zero-config build dependencies; Log4j 2 and JUL need extra setup — see the logging integration section in deploying-java-sdk-bundles. JUL's documented alternative to calling AirflowJulHandler.setup() in main() is a logging.properties file, wired through jvm_args:

ini
[sdk]
coordinators = {
  "java-jdk17": {
    "classpath": "airflow.sdk.coordinators.java.JavaCoordinator",
    "kwargs": {
      "jars_root": ["/opt/airflow/jars"],
      "jvm_args": ["-Djava.util.logging.config.file=/opt/airflow/logging.properties"]
    }
  }
}
Show full SKILL.md (439 more words)Show less
ExecutableCoordinator (Go and other self-contained-executable SDKs)
  • classpath: airflow.sdk.coordinators.executable.ExecutableCoordinator
  • Worker runtime: none beyond the bundle itself. The bundle is a self-contained native executable (AFBNDL01), so it needs no language runtime, but it must be built for the worker's OS/arch (a mismatch fails with exec format error).
ParameterDefaultDescription
executables_root(required)One or more directories scanned recursively for executable bundles (AFBNDL01-trailered native binaries). Accepts a string or a list of strings/paths. Bundles are identified by the trailer magic, not by filename. The coordinator matches an incoming dag_id against each bundle's embedded manifest and verifies its integrity hash before launching.
task_startup_timeout10.0Seconds to wait for the subprocess to connect after launch. Increase it if bundle startup is slow (constrained hardware, first cold start).

(Future coordinators — for other languages — will list their own classpath, runtime, and kwargs here.)


Verifying the configuration

  1. Confirm the runtime/artifact is usable where workers run — for the Java SDK, java -version via astro dev bash or docker compose exec ...; for the Go SDK, the packed bundle exists and matches the worker's OS/arch.
  2. Confirm the artifact directory referenced in kwargs (e.g. jars_root, executables_root) actually contains your artifact on the worker filesystem.
  3. Trigger the DAG and open the native task's logs — you should see the subprocess start and your task output.
Troubleshooting
SymptomLikely cause / fix
Task fails immediately mentioning coordinator or queuecoordinators / queue_to_coordinator not valid one-line JSON, or the queue name doesn't match the stub's queue=. Fix the JSON and restart.
Runtime not found (e.g. java: command not found)The language runtime isn't on the worker, or the executable path kwarg is wrong. Install the runtime and verify its version.
"No artifact found" / "no DAGs" / "no bundle contains dag_id"The artifact-directory kwarg points at the wrong place, the artifact isn't there yet, or its dag_id doesn't match the stub. Confirm the path and the IDs.
Wrong/ambiguous entry point (Java)Multiple executable JARs under jars_root. Set main_class explicitly.
Go bundle is skipped silentlyNot a valid AFBNDL01 bundle, or its integrity hash failed (re-pack after any strip/sign/rebuild).
exec format error on the Go bundleBuilt for a different OS/arch than the worker. Cross-compile with --goos/--goarch (see deploying-go-sdk-bundles).
DAG run hangs at the native taskRaise task_startup_timeout (e.g. 30.0); first-run subprocess startup can be slow.

  • authoring-language-sdk-tasks: The shared Python-stub pattern and conceptual model.
  • authoring-java-sdk-tasks: Java task code and matching Python stubs.
  • deploying-java-sdk-bundles: Build the bundle and put the artifact where the coordinator scans.
  • authoring-go-sdk-tasks: Go task code and matching Python stubs.
  • deploying-go-sdk-bundles: Build/pack the Go bundle and place it where the coordinator scans.
  • deploying-airflow: General deployment of Airflow on Astro, Docker Compose, or Kubernetes.

© astronomer, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/configuring-airflow-language-sdks of astronomer/agents.

Open the folder on GitHubat commit 486ee63

Compare with similar skills

Configuring Airflow Language Sdks next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Configuring Airflow Language Sdks compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Configuring Airflow Language Sdks this skillastronomer/agents451—~2.3kAutomated safety check: NotesApache-2.0
Chart Testsastronomer/airflow-chart297—~2.8kAutomated safety check: PassCustom licence
Functional Testsastronomer/airflow-chart297—~2.2kAutomated safety check: PassCustom licence
Create Examplegodatadriven/whirl205—~1.1kAutomated safety check: PassApache-2.0
Senior Data Engineerbenchflow-ai/skillsbench1.8k—~5.9kAutomated safety check: PassMIT
Airflow DAG Patternswshobson/agents40k9 repos~784Automated safety check: PassMIT

Similar skills

  • Chart Tests

    astronomer/airflow-chart

    A skill your agent uses when writing, editing, reviewing, or running Helm chart tests for the Astronomer airflow-chart repository.

    297 GitHub stars~2.8k tokensUpdated 2 days ago
    Data & AnalyticsAuto-check passed
  • Functional Tests

    astronomer/airflow-chart

    A skill your agent uses when writing, editing, reviewing, or running functional (end-to-end) tests for the Astronomer airflow-chart repository.

    297 GitHub stars~2.2k tokensUpdated 2 days ago
    Data & AnalyticsAuto-check passed
  • Create Example

    godatadriven/whirl

    Create a new Whirl example project in the examples/ directory.

    205 GitHub stars~1.1k tokensUpdated 8 days ago
    Data & AnalyticsAuto-check passed
  • Senior Data Engineer

    benchflow-ai/skillsbench

    World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, real-time streaming, and data infrastructure.

    1.8k GitHub stars~5.9k tokensUpdated 2 mo ago
    Data & AnalyticsAuto-check passed
  • Airflow DAG Patterns

    wshobson/agents

    Patterns for writing production-ready Apache Airflow DAGs: task dependencies, custom operators and sensors, local testing, and rules for what to avoid.

    40k GitHub starsUsed in 9 repos~784 tokens
    Data & AnalyticsAuto-check passed
  • Version Bumper

    godatadriven/whirl

    Bump the Airflow or Python version across all project files.

    205 GitHub stars~1.2k tokensUpdated 8 days ago
    Data & AnalyticsAuto-check passed

More from astronomer/agents

All 34 skills in this repo
  • Analyzing Data

    astronomer/agents

    Queries the data warehouse with SQL and answers business questions about data.

    451 GitHub stars~1.3k tokensUpdated 2 days ago
    Auto-check passed
  • Airflow

    astronomer/agents

    Queries, manages, and troubleshoots Apache Airflow using the af CLI.

    451 GitHub starsUsed in 1 repo~3.8k tokens
    Auto-check passed
  • Guide for migrating Dagster projects to Apache Airflow 3 on Astro.

    451 GitHub stars~3.8k tokensUpdated 2 days ago
    Auto-check passed
  • Authoring Dags

    astronomer/agents

    Workflow and best practices for writing Apache Airflow DAGs.

    451 GitHub starsUsed in 1 repo~1.8k tokens
    Auto-check passed
  • Deploying Airflow

    astronomer/agents

    Deploys Airflow DAGs and projects. An agent skill from astronomer/agents.

    451 GitHub starsUsed in 1 repo~2.8k tokens
    Auto-check passed
  • Airflow Hitl

    astronomer/agents

    Builds human-in-the-loop (HITL) Airflow workflows - approval gates, form input, and human-driven branching.

    451 GitHub stars~1.8k tokensUpdated 2 days ago
    Auto-check passed

Questions about Configuring Airflow Language Sdks

What does Configuring Airflow Language Sdks do?

Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options. Configuring Airflow Language Sdks is an agent skill from astronomer/agents. Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options.

When should I use Configuring Airflow Language Sdks?

Configuring Airflow Language Sdks fits situations like: the user wants Airflow to route a queue to a native-language coordinator; asks about the [sdk] coordinators/queuetocoordinator settings; AIRFLOWSDKCOORDINATORS; executablesroot.

How do I install Configuring Airflow Language Sdks in Claude Code?

Run `npx skills add astronomer/agents --skill configuring-airflow-language-sdks -a claude-code`. Or copy the skill folder (skills/configuring-airflow-language-sdks in astronomer/agents) into .claude/skills/configuring-airflow-language-sdks in your project. Claude Code loads it when a task matches its description.

How do I install Configuring Airflow Language Sdks in Codex?

Run `npx skills add astronomer/agents --skill configuring-airflow-language-sdks -a codex`. Or copy the skill folder (skills/configuring-airflow-language-sdks in astronomer/agents) into .agents/skills/configuring-airflow-language-sdks in your project. Codex loads it when a task matches its description.

Can I use Configuring Airflow Language Sdks in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add astronomer/agents --skill configuring-airflow-language-sdks -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/configuring-airflow-language-sdks, .gemini/skills/configuring-airflow-language-sdks, .github/skills/configuring-airflow-language-sdks and .opencode/skills/configuring-airflow-language-sdks in your project.

What does Configuring Airflow Language Sdks need to run?

Going by SKILL.md and its folder, Configuring Airflow Language Sdks needs the command-line tools its instructions call (java and docker). Our summary lists: Python 3; Docker.

Does Configuring Airflow Language Sdks access the network?

SKILL.md contains no URLs. Its commands use docker, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Configuring Airflow Language Sdks safe to install?

Our automated static check of SKILL.md found notes only (mentions a .env file), nothing it rates as a warning. It is not a guarantee. Review the folder before installing.

What licence does Configuring Airflow Language Sdks use?

Configuring Airflow Language Sdks is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Configuring Airflow Language Sdks use?

About 2.3k tokens (SKILL.md is roughly 9.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Configuring Airflow Language Sdks?

Skills that share tags, products or a category with Configuring Airflow Language Sdks: Chart Tests (astronomer/airflow-chart, 297 stars), Functional Tests (astronomer/airflow-chart, 297 stars), Create Example (godatadriven/whirl, 205 stars) and Senior Data Engineer (benchflow-ai/skillsbench, 1.8k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Configuring Airflow Language Sdks?

astronomer (a GitHub organization) maintains it in astronomer/agents, which has 451 GitHub stars. The repository holds 34 skills in this directory. The repository was last updated on October 7, 2026.

Source: astronomer/agents on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.