Best Data pipelines and ETL skills, page 5

Guide for migrating Apache Airflow 2.x projects to Airflow 3.x.

astronomer/agents

Deep-dive data profiling for a specific table. An agent skill from astronomer/agents.

astronomer/agents

Initialize and configure Astro/Airflow projects. An agent skill from astronomer/agents.

astronomer/agents

Complex DAG testing workflows with debugging and fixing cycles.

astronomer/agents

Initialize warehouse schema discovery. An agent skill from astronomer/agents.

astronomer/agents

Multi-agent pipeline orchestration skill. An agent skill from autopus-ai/autopus-adk.

autopus-ai/autopus-adk

Process multiple sequence files in batch using Biopython. An agent skill from FreedomIntelligence/OpenClaw-Medical-Skills.

FreedomIntelligence/OpenClaw-Medical-Skills

Index of the engineering-team skills bundle for Claude Code, Codex, Gemini CLI, Cursor, OpenClaw, and 6 more tools.

alirezarezvani/claude-skills

Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks.

databricks/databricks-agent-skills

Python data pipelines with modular architecture. An agent skill from jamditis/claude-skills-journalism.

jamditis/claude-skills-journalism

Populate person or group properties from a data warehouse table or materialized view, so warehouse columns become properties usable in feature flags, cohorts, and insights.

PostHog/posthog

Define implementation contracts (APIs and data models) that developers will build against during PRD v0.6 Architecture.

mattgierhart/PRD-driven-context-engineering

Transform v0.6 specifications into context-window-sized work packages (EPICs) during PRD v0.7 Build Execution.

mattgierhart/PRD-driven-context-engineering

Define test cases BEFORE implementation, ensuring every API, business rule, and user journey has verifiable acceptance criteria during PRD v0.7 Build Execution.

mattgierhart/PRD-driven-context-engineering

Build event streaming and real-time data pipelines with Kafka, Pulsar, Redpanda, Flink, and Spark.

ancoleman/ai-design-components

Airflow adapter pattern for v2/v3 API compatibility. An agent skill from astronomer/agents.

astronomer/agents

Builds Airflow 3.1+ plugins that embed FastAPI apps, custom UI pages, React components, middleware, macros, and operator links directly into the Airflow UI.

astronomer/agents

Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (taskstatestore, assetstatestore) and the crash-safe ResumableJobMixin.

astronomer/agents

Define reusable Airflow task group templates with Pydantic validation and compose DAGs from YAML.

astronomer/agents

Check ETL step files for outdated coding patterns and offer to fix them.

owid/etl

Split text into specified-sized chunks while maintaining semantic integrity.

VanGong1999/screen-creative-skills

[omh] Data pipeline work -- an ETL or streaming job, a backfill or replay, duplicate events, a schema change downstream, a lineage question, a data-quality regression: make every rerun idempotent…

rlaope/oh-my-hermes

Run the PAIDF Orchestration Image Attribute Augmentation DAG on Kubernetes - person-crop clothing augmentation, attribute search, and augmented dataset generation.

NVIDIA/skills

Process many sequence files in batch (count, merge, split, convert, summarize) with memory-safe streaming and on-disk indexing using Biopython, pysam, or pyfastx.

GPTomics/bioSkills

A skill your agent uses when building batch jobs, ETL pipelines, scheduled imports/exports, or any chunk-oriented bulk processing with Spring Batch.

rrezartprebreza/spring-boot-skills

A skill your agent uses when building batch jobs, ETL pipelines, scheduled imports/exports, or any chunk-oriented bulk processing with Spring Batch.

rrezartprebreza/spring-boot-skills

Tests Amazon MWAA workflow execution end-to-end: trigger a run and monitor it to completion for Provisioned (Python DAG, via Airflow REST API) and Serverless (YAML workflow, via StartWorkflowRun).

aws/agent-toolkit-for-aws

Process automate data cleaning, transformation, and validation for ML tasks.

jeremylongshore/tons-of-skills-marketplace

A skill your agent uses when reviewing, designing, or modifying Java enterprise systems that expose, exchange, store, process, export, or port data across users, businesses, connected products…

jabrena/plinth

Comprehensive knowledge system for addiction recovery environments, supporting both residential and outpatient (IOP/PHP) patients.

curiositech/some_claude_skills

Data engineering knowledge reference covering Airflow, Dagster, Kafka Streams, Flink, dbt, and data quality patterns.

telagod/code-abyss

A skill your agent uses when building Query Reports, Script Reports, or configuring Report Builder, including chart data integration.

Impertio-Studio/Frappe_Claude_Skill_Package

Turns a dbt Core project into an Airflow DAG/TaskGroup using Astronomer Cosmos.

astronomer/agents

Execute arbitrary Python or PySpark code on Fabric Spark compute without creating a notebook artifact; ephemeral Livy sessions with full Delta table access.

data-goblin/power-bi-agentic-development

A skill your agent uses when operating PAIDF Curation and Retrieval or NVIDIA Cosmos Curator pipelines (split, filter, caption, embed, dedup, shard, image annotate) or PAIDF Data Mining…

NVIDIA/skills

Retrieves and searches dbt documentation pages in LLM-friendly markdown format.

Kilo-Org/kilo-marketplace

Produces distinctive, production-grade UI for pages, components, visual interfaces, typography, and screenshot-driven polish.

ninehills/skills

Signals scout for PostHog data pipelines — CDP destinations and transformations, batch exports, and hog flows.

PostHog/posthog

Optimizes Snowflake SQL query performance from provided query text.

AltimateAI/data-engineering-skills

Cross-domain loop orchestration for the chip design pipeline.

hdl-tools/digital-chip-design-agents

Troubleshoot and resolve issues with Azure Messaging SDKs for Event Hubs and Service Bus.

microsoft/GitHub-Copilot-for-Azure

Guard production Databricks data pipelines — Delta Lake, Liquid Clustering, Structured Streaming, Auto Loader, and DLT — against the twelve foot-guns that fire at scale: OPTIMIZE/auto-compaction…

jeremylongshore/tons-of-skills-marketplace

Manage Firebase/Firestore operations including CRUD, queries, batch processing, and index/rule guidance.

jeremylongshore/tons-of-skills-marketplace

Programmatic page generation at scale using template-based SEO and data pipelines: keyword pattern mining, template architecture, and indexation for 100-100K+ pages.

borghei/Claude-Skills

Data engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka.

borghei/Claude-Skills

Modern Python project architecture guide for 2025. An agent skill from majiayu000/spellbook.

majiayu000/spellbook

Document column-level mappings between source and target schemas.

nimrodfisher/data-analytics-skills

Build structured semantic layer documentation for metrics, dimensions, and entities.

nimrodfisher/data-analytics-skills