Topic · Data & Analytics
Best Data pipelines and ETL skills, page 9
Data pipelines and ETL skills, ranked
Ranked by score. Sort bymost stars,trending,newest,recently updated
| # | Skill | Repository | Stars | Used in | Tokens | Auto-check | Licence | Updated |
|---|---|---|---|---|---|---|---|---|
| 385 | Build automated PII detection and redaction pipelines using spaCy NER, Microsoft Presidio, and AWS Macie integration. | mukul975/ | 295 | — | ~5.6k | Automated safety check: Pass | Apache-2.0 | 6 mo ago |
| 386 | 386.Icde Experiments A skill your agent uses when designing or auditing IEEE ICDE experiments for a data-engineering paper: workload realism, tuned baselines, scale curves over single points, throughput and tail-latency… | brycewang-stanford/ | 1.2k | — | ~1k | Automated safety check: Pass | MIT | 11 days ago |
| 387 | A skill your agent uses when strengthening reproducibility evidence for an IEEE ICDE data-engineering paper: pinning hardware, storage devices, software versions, datasets and workload generators… | brycewang-stanford/ | 1.2k | — | ~1k | Automated safety check: Pass | MIT | 11 days ago |
| 388 | 388.Icde Submission A skill your agent uses when auditing an IEEE ICDE research-track submission for round choice, Microsoft CMT setup, conflict declarations, the IEEE 12-page format, single-blind requirements… | brycewang-stanford/ | 1.2k | — | ~1.1k | Automated safety check: Pass | MIT | 11 days ago |
| 389 | 389.Icde Workflow A skill your agent uses when planning an IEEE ICDE project timeline around the two-round-per-edition calendar, choosing between the June and November research deadlines, budgeting for a possible… | brycewang-stanford/ | 1.2k | — | ~1.1k | Automated safety check: Pass | MIT | 11 days ago |
| 390 | A skill your agent uses when revising an IEEE ICDE paper for the first-page data-engineering contract, a reconstructable mechanism description, the running-example device, performance claims scoped… | brycewang-stanford/ | 1.2k | — | ~972 | Automated safety check: Pass | MIT | 11 days ago |
| 391 | Security review checklist for construction software systems. | datadrivenconstruction/ | 344 | — | ~3.3k | Automated safety check: Pass | MIT | 1 mo ago |
| 392 | 392.Data Pipelines A skill your agent uses when building or debugging data pipelines with Airflow or Prefect, writing dbt models or tests, designing incremental loads, implementing idempotent ETL/ELT jobs, validating… | kid-sid/ | 189 | — | ~4.3k | Automated safety check: Pass | MIT | 2 mo ago |
| 393 | Process large Django querysets and write-heavy jobs with memory-safe reads, values/valueslist, iterator chunking, set-based update/delete, bulkcreate, bulkupdate, F expressions, Func expressions… | hashgraph-online/ | 1.2k | — | ~604 | Automated safety check: Pass | MIT | today |
| 394 | Selects, investigates, and compares AWS object, file, and block storage services, and answers cost, performance, configuration, security, and troubleshooting questions about storage services. | aws/ | 2.8k | — | ~5.8k | Automated safety check: Pass | Apache-2.0 | today |
| 395 | Orchestrate AI/ML pipelines for data ingestion, model training, batch inference, and RAG indexing using Prefect, Airflow, or Dagster. | majiayu000/ | 666 | 1 repo | ~2.3k | Automated safety check: Pass | MIT | today |
| 396 | 396.Automatic1111 Feature-rich Stable Diffusion Web UI for image generation. An agent skill from majiayu000/claude-skill-registry. | majiayu000/ | 666 | 1 repo | ~1.6k | Automated safety check: Pass | Apache-2.0 | today |
| 397 | 397.Data Researcher Data discovery and analysis specialist focused on extracting actionable insights from complex datasets, identifying patterns and anomalies, and transforming raw data into strategic intelligence. | majiayu000/ | 666 | 1 repo | ~4.6k | Automated safety check: Pass | MIT | today |
| 398 | High-quality voice synthesis with 18 personas, 32 languages, sound effects, batch processing, and voice design using ElevenLabs API. | majiayu000/ | 666 | 1 repo | ~3k | Automated safety check: Notes | MIT | today |
| 399 | 399.ML Engineer Expert in building scalable ML systems, from data pipelines and model training to production deployment and monitoring. | majiayu000/ | 666 | 1 repo | ~2.8k | Automated safety check: Pass | MIT | today |
| 400 | Marveen flottában új channel plugin (slack-channel, telegram, stb.) telepítésekor a user-szintű ~/.claude/settings.json enabledPlugins minden agent-nek loadolja a plugin server-t, és ha az egy… | Szotasz/ | 115 | — | ~1.3k | Automated safety check: Notes | MIT | today |
| 401 | 401.Ae Community AE community analysis and data reporting for posts, comments, topics, livestreams, customer-service chats, WeCom after-sales groups, and in-game chat. | infometa/ | 344 | — | ~3.8k | Automated safety check: Pass | No licence | today |
| 402 | Answers questions about ThinkingData SDK integration and usage, including the LogBus2 data import tool. | infometa/ | 344 | — | ~3k | Automated safety check: Pass | No licence | today |
| 403 | 403.AI Governance Design and operate an organization's AI governance system: define governance principles, operating models and decision rights, risk frameworks, lifecycle gates, and fairness, transparency, privacy… | magnus919/ | 113 | — | ~3.3k | Automated safety check: Pass | MIT | 2 days ago |
| 404 | Design and implement backend services and APIs — REST, gRPC, GraphQL, event-driven handlers, transaction boundaries, outbox/inbox delivery, migration coexistence, database access, integration, error… | magnus919/ | 113 | — | ~2.6k | Automated safety check: Pass | MIT | 2 days ago |
| 405 | 405.Data Cleaning Clean, profile, validate, reshape, and document messy tabular, text, JSON, and relational data through an evidence-first, reproducible workflow. | magnus919/ | 113 | — | ~2.1k | Automated safety check: Pass | MIT | 2 days ago |
| 406 | Build and maintain web frontends — component architecture, state management, API integration, responsive layout, client-side performance, and frontend testing patterns. | magnus919/ | 113 | — | ~1.5k | Automated safety check: Pass | MIT | 2 days ago |
| 407 | 407.Instagram API An Instagram API alternative on fetcher.sh — pay-per-call in USDC via x402, or prepaid credits with a Bearer key, no login and no session cookies. | aiskillstore/ | 430 | — | ~2.1k | Automated safety check: Pass | No licence | yesterday |
| 408 | 408.Tiktok API A TikTok API alternative on fetcher.sh — pay-per-call in USDC via x402, or prepaid credits with a Bearer key, no login and no app review. | aiskillstore/ | 430 | — | ~2k | Automated safety check: Pass | No licence | yesterday |
| 409 | 409.Twitter API A Twitter API alternative and X API alternative on fetcher.sh — pay-per-call in USDC via x402, or prepaid credits with a Bearer key, no OAuth and no developer application. | aiskillstore/ | 430 | — | ~2.2k | Automated safety check: Pass | No licence | yesterday |
| 410 | Audit, prepare, and deploy PAIDF Orchestration on a Kubernetes GPU cluster - single-GPU H100/L40S hosts, managed Kubernetes, kubeadm, and similar. | NVIDIA/ | 3.5k | — | ~3.8k | Automated safety check: Warn | Apache-2.0 | yesterday |
| 411 | Writes and executes SQL queries against the data warehouse using dbt's Semantic Layer or ad-hoc SQL to answer business questions. | Kilo-Org/ | 190 | — | ~1.9k | Automated safety check: Pass | Apache-2.0 | 9 days ago |
| 412 | 412.Dbt Testing dbt testing strategies using dbtconstraints for database-level enforcement, generic tests, and singular tests. | Kilo-Org/ | 190 | — | ~4k | Automated safety check: Pass | Apache-2.0 | 9 days ago |
| 413 | Formats and executes dbt CLI commands, selects the correct dbt executable, and structures command parameters. | Kilo-Org/ | 190 | — | ~1.5k | Automated safety check: Pass | Apache-2.0 | 9 days ago |
| 414 | 414.Pyspark Etl Best practices for building performant, testable PySpark ETL pipelines with Spark SQL and Apache Iceberg. | Mindrally/ | 268 | — | ~2.4k | Automated safety check: Pass | Apache-2.0 | 1 mo ago |
| 415 | Best practices for Snowflake SQL, semi-structured data, and data pipelines built with Dynamic Tables, Streams, Tasks, and Snowpipe. | Mindrally/ | 268 | — | ~2.3k | Automated safety check: Pass | Apache-2.0 | 1 mo ago |
| 416 | Best practices for Snowpark Python (DataFrames, UDFs, UDTFs, stored procedures) and dbt with the dbt-snowflake adapter. | Mindrally/ | 268 | — | ~2.5k | Automated safety check: Pass | Apache-2.0 | 1 mo ago |
| 417 | Expert knowledge for Azure Data Manager for Agriculture development including limits & quotas, security, configuration, and integrations & coding patterns. | MicrosoftDocs/ | 777 | — | ~1.6k | Automated safety check: Pass | CC-BY-4.0 | 2 days ago |