Topic · Data & Analytics
Best Data pipelines and ETL skills, page 3
Data pipelines and ETL skills, ranked
Ranked by score. Sort bymost stars,trending,newest,recently updated
| # | Skill | Repository | Stars | Used in | Tokens | Auto-check | Licence | Updated |
|---|---|---|---|---|---|---|---|---|
| 97 | Build or refresh an OWID static visualization end to end — resolve what data it needs from an old static viz image, an indicator, or a grapher chart; check both the ETL catalog and the producer's… | owid/ | 158 | — | ~8.3k | Automated safety check: Pass | MIT | today |
| 98 | Describe images (charts, diagrams, tables, screenshots) using Vision AI. | Wide-Moat/ | 126 | — | ~732 | Automated safety check: Pass | Unknown | 2 days ago |
| 99 | A skill your agent uses for any task involving Dagster, the dg CLI, assets, materialization, components, data tools or data pipelines, matching the situations the Dagster skill handles. | dagster-io/ | 211 | — | ~321 | Automated safety check: Pass | Apache-2.0 | today |
| 100 | Guides technology selection and implementation of AI and ML features in .NET 8+ applications using ML.NET, Microsoft.Extensions.AI (MEAI), Microsoft Agent Framework (MAF), GitHub Copilot SDK, ONNX… | dotnet/ | 5.6k | 2 repos | ~2.1k | Automated safety check: Pass | MIT | today |
| 101 | Guidance for running the Tinybird tb CLI: choosing local, branch or cloud workflows, building and deploying, loading data, handling tokens and secrets, and CI/CD. | TryGhost/ | 55k | — | ~388 | Automated safety check: Pass | MIT | today |
| 102 | Writes model and column descriptions in dbt schema.yml files, matching the project's existing documentation style and recording grain, business rules and caveats. | AltimateAI/ | 128 | — | ~1.2k | Automated safety check: Pass | MIT | 6 days ago |
| 103 | 103.Veomni Develop VeOmni-specific checklist for feature development and refactoring. | ByteDance-Seed/ | 2.2k | — | ~1k | Automated safety check: Pass | Apache-2.0 | today |
| 104 | 104.Data Engineer Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. | davila7/ | 32k | 8 repos | ~2.8k | Automated safety check: Pass | MIT | today |
| 105 | Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. | alirezarezvani/ | 28k | 3 repos | ~1.4k | Automated safety check: Pass | MIT | 1 mo ago |
| 106 | Installs and configures KubeSphere's WizTelemetry Data Pipeline, built on Vector, which collects and routes logs, audit events, events and notifications to OpenSearch. | kubesphere/ | 17k | — | ~2.6k | Automated safety check: Pass | Unknown | 2 mo ago |
| 107 | 107.Cx Data Pipeline A skill your agent uses when the user asks to "set up parsing", "create parsing rule", "extract fields from logs", "regex extraction", "log parsing", "enrich logs", "add context to logs", "custom… | coralogix/ | 121 | — | ~3k | Automated safety check: Pass | Apache-2.0 | yesterday |
| 108 | Build automated AI workflows combining multiple models and services. | NeverSight/ | 216 | 1 repo | ~2.6k | Automated safety check: Pass | No licence | yesterday |
| 109 | Rules for writing Tinybird datasources, pipes, endpoints and materialized views, with SQL constraints, optimization habits and deduplication patterns. | TryGhost/ | 55k | — | ~417 | Automated safety check: Pass | MIT | today |
| 110 | Take (soon-to-sunset) OWID explorers to redirected MDIMs, end to end. | owid/ | 158 | — | ~7.3k | Automated safety check: Notes | MIT | today |
| 111 | Build real-time streaming applications with Azure Event Hubs SDK for Java. | microsoft/ | 3.1k | 5 repos | ~2.8k | Automated safety check: Pass | MIT | 2 days ago |
| 112 | 112.Media Processing Video/audio/image processing with FFmpeg and ImageMagick. An agent skill from nicepkg/ai-workflow. | nicepkg/ | 285 | — | ~2.4k | Automated safety check: Notes | MIT | 8 mo ago |
| 113 | Conventions for writing empirical finance R code with data.table, fixest, arrow, and ggplot2. | aspi6246/ | 158 | — | ~1.3k | Automated safety check: Pass | No licence | 1 mo ago |
| 114 | 114.Airflow Hitl Builds human-in-the-loop (HITL) Airflow workflows - approval gates, form input, and human-driven branching. | astronomer/ | 451 | — | ~1.8k | Automated safety check: Pass | Apache-2.0 | yesterday |
| 115 | Azure Event Hubs SDK for .NET. An agent skill from microsoft/skills. | microsoft/ | 3.1k | 6 repos | ~2.7k | Automated safety check: Pass | MIT | 2 days ago |
| 116 | Automatically merge scattered Excel and CSV files, normalize column names, and extract structured tables from PDF, HTML, TXT, or Markdown documents. | Drchronx/ | 135 | — | ~671 | Automated safety check: Pass | Unknown | 4 mo ago |
| 117 | Converts legacy SQL to modular dbt models. An agent skill from AltimateAI/data-engineering-skills. | AltimateAI/ | 128 | — | ~762 | Automated safety check: Pass | MIT | 6 days ago |
| 118 | Diagnose and accelerate large data movement — ingestion, backfill, export, ETL, warehouse loading, manifest catch-up, and table synchronization — by isolating the true bottleneck, benchmarking… | affaan-m/ | 275k | 1 repo | ~707 | Automated safety check: Pass | MIT | 3 days ago |
| 119 | 119.Clickhouse Io ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads. | affaan-m/ | 275k | 1 repo | ~2.7k | Automated safety check: Pass | MIT | 3 days ago |
| 120 | World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. | davila7/ | 32k | 1 repo | ~1.4k | Automated safety check: Pass | MIT | today |
| 121 | Maps backtest data sources and research data needs to the right provider or tool, with each one's markets, required environment keys and network constraints. | HKUDS/ | 35k | — | ~2.4k | Automated safety check: Pass | MIT | today |
| 122 | 122.Gget Queries 20+ bioinformatics resources through CLI/Python. An agent skill from K-Dense-AI/scientific-agent-skills. | K-Dense-AI/ | 48k | 1 repo | ~2.8k | Automated safety check: Notes | BSD-2-Clause | 3 days ago |
| 123 | 123.Databricks Jobs Develop and deploy Lakeflow Jobs on Databricks via DABs, Python SDK, or the CLI. | Kilo-Org/ | 190 | 1 repo | ~3.1k | Automated safety check: Pass | Unknown | 10 days ago |
| 124 | Data pipelines, feature stores, and embedding generation for AI/ML systems. | ancoleman/ | 526 | 1 repo | ~3.5k | Automated safety check: Pass | MIT | 10 mo ago |
| 125 | Annotate Airflow tasks with data lineage using inlets and outlets. | astronomer/ | 451 | — | ~2.8k | Automated safety check: Pass | Apache-2.0 | yesterday |
| 126 | A skill your agent uses when: comparing SQL Server tables across instances, data migration validation, ETL verification, row mismatch detection, schema drift, reconciliation report, production vs… | github/ | 40k | 1 repo | ~1.4k | Automated safety check: Pass | MIT | today |
| 127 | 127.Cocoindex Comprehensive toolkit for developing with the CocoIndex library. | davila7/ | 32k | 2 repos | ~6.4k | Automated safety check: Notes | MIT | today |
| 128 | Analyze DBSQL queries, including SQL embedded in notebooks (spark.sql(...), %sql cells), for anti-patterns, lint issues, and performance problems, using Databricks-specific dialect and platform… | AltimateAI/ | 128 | — | ~6.7k | Automated safety check: Pass | MIT | 6 days ago |
| 129 | 129.Fp Pipe Ref Quick reference for pipe and flow. An agent skill from sickn33/agentic-awesome-skills. | sickn33/ | 47k | 2 repos | ~478 | Automated safety check: Pass | MIT | yesterday |
| 130 | 130.Rocky Poc Authoring a new POC under examples/playground/pocs/. An agent skill from rocky-data/rocky. | rocky-data/ | 304 | — | ~1.7k | Automated safety check: Pass | Apache-2.0 | today |
| 131 | Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. | maziyarpanahi/ | 5.5k | — | ~2.2k | Automated safety check: Pass | Apache-2.0 | today |
| 132 | 132.Data Engineering Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation | rohitg00/ | 2.7k | — | ~1.7k | Automated safety check: Pass | Apache-2.0 | 4 mo ago |
| 133 | 133.Data Pipeline Data pipeline expert for ETL, Apache Spark, Airflow, dbt, and data quality | RightNow-AI/ | 18k | — | ~847 | Automated safety check: Pass | Apache-2.0 | 3 mo ago |
| 134 | Comprehensive data quality assessment against business rules, schema constraints, and freshness expectations. | nimrodfisher/ | 465 | — | ~685 | Automated safety check: Pass | MIT | 13 days ago |
| 135 | Generates SQL validation notebooks for dbt PR changes with before/after comparison queries. | sickn33/ | 47k | 1 repo | ~1.1k | Automated safety check: Pass | MIT | yesterday |
| 136 | Writes Airflow task logic in Go using the Airflow Go SDK. An agent skill from astronomer/agents. | astronomer/ | 451 | — | ~2.1k | Automated safety check: Pass | Apache-2.0 | yesterday |
| 137 | 137.Prompt Guard Meta's 86M prompt injection and jailbreak detector. An agent skill from Orchestra-Research/AI-Research-SKILLs. | Orchestra-Research/ | 13k | 1 repo | ~2.4k | Automated safety check: Warn | MIT | 3 mo ago |
| 138 | Optimizes Snowflake query performance using query ID from history. | AltimateAI/ | 128 | — | ~919 | Automated safety check: Pass | MIT | 6 days ago |
| 139 | Process and manipulate images using ImageMagick. An agent skill from github/awesome-copilot. | github/ | 40k | 1 repo | ~1.7k | Automated safety check: Notes | MIT | today |
| 140 | Comprehensive Snowflake development assistant covering SQL best practices, data pipeline design (Dynamic Tables, Streams, Tasks, Snowpipe), Cortex AI functions, Cortex Agents, Snowpark Python, dbt… | sickn33/ | 47k | 2 repos | ~2.1k | Automated safety check: Pass | MIT | yesterday |
| 141 | Search and download images via Google Custom Search API with LLM-powered selection. | glebis/ | 389 | — | ~1.4k | Automated safety check: Notes | MIT | 12 days ago |
| 142 | Discover OPC UA servers, connect MATLAB clients, and browse/navigate server nodes using opcuaserverinfo, the Local Discovery Service (LDS), opcua, connect, setSecurityModel, certificate-trust… | matlab/ | 1.1k | — | ~4.4k | Automated safety check: Pass | Unknown | yesterday |
| 143 | Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. | astronomer/ | 451 | — | ~2.6k | Automated safety check: Pass | Apache-2.0 | yesterday |
| 144 | 144.Datavis Comprehensive data visualization toolkit for creating beautiful, mathematically elegant visualizations with D3.js, Chart.js, and custom SVG. | foryourhealth111-pixel/ | 3.6k | — | ~1.1k | Automated safety check: Pass | Apache-2.0 | 1 mo ago |