Agent skill

Data Engineering Data Pipeline

by aiskillstore in aiskillstore/marketplace

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

No licenceAuto-check passedData & Analytics

Install Data Engineering Data Pipeline

skills CLI
$ npx skills add aiskillstore/marketplace --skill data-engineering-data-pipeline -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install aiskillstore/marketplace data-engineering-data-pipeline --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/aiskillstore/marketplace.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/sickn33/data-engineering-data-pipeline .claude/skills/data-engineering-data-pipeline && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
data-engineering-data-pipeline
GitHub stars
430
Used in
8 other repos
Token cost
~1.8k tokens
SKILL.md length
693 words
Files
2
Skills in repo
1,085
Repo updated
First seen
Licence
None found

At a glance

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

  • Works in 12 steps: Architecture Design → Ingestion Implementation → Orchestration → …
  • Tasks that involve Data pipelines and ETL
  • SKILL.md covers Use this skill when, Do not use this skill when, Requirements and Core Capabilities, plus 5 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Data Engineering Data Pipeline is an agent skill from aiskillstore/marketplace. You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

Its SKILL.md is about 1.8k tokens, which your agent loads only when the skill is triggered. The skill folder holds 1 other file (for example `skill-report.json`).

It sits in Data & Analytics, covering Data pipelines and ETL. It works with dbt. The repository describes itself as: Security-audited skills for Claude, Codex & Claude Code. One-click install, quality verified.

When your agent uses it

  • Tasks that involve Data pipelines and ETL

Example prompts

  • “/data-engineering-data-pipeline”

Requirements

  • Python 3
  • Docker

Workflow steps

12 steps, taken from the step headings in SKILL.md.

  1. Architecture Design
  2. Ingestion Implementation
  3. Orchestration
  4. Transformation with dbt
  5. Data Quality Framework
  6. Storage Strategy
  7. Monitoring & Cost Optimization
  8. Architecture Documentation
  9. Implementation Code
  10. Configuration Files
  11. Monitoring & Observability
  12. Operations Guide

What it can do on your machine

Read from SKILL.md and the folder at commit 4ac52da. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md (its code samples are python).

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Data Engineering Data Pipeline loads about 1.8k tokens when it runs. Until then it costs about 46 tokens; SKILL.md has 693 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~46
When it runs · the whole SKILL.md, loaded when a task matches
~1.8k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Without a licence we can't republish the file, so here is its outline and opening line. It has 693 words (~1,821 tokens).

“You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.”

— opening of SKILL.md by aiskillstore
name
data-engineering-data-pipeline
risk
unknown
source
community
date_added
2026-02-27

Read the full SKILL.md on GitHub

Files

SKILL.md and 1 other file in skills/sickn33/data-engineering-data-pipeline of aiskillstore/marketplace.

  • SKILL.md
  • skill-report.json

Open the folder on GitHubat commit 4ac52da

Used in 8 other repositories

We found 18 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 8 other GitHub owners. This page covers the copy in aiskillstore/marketplace, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Data Engineering Data Pipeline next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Data Engineering Data Pipeline compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Data Engineering Data Pipeline this skillaiskillstore/marketplace4308 repos~1.8kAutomated safety check: PassNone
Dbt Databricks PR Readydatabricks/dbt-databricks379—~2.8kAutomated safety check: PassApache-2.0
Mz Dbt ReleaseMaterializeInc/materialize6.4k—~1.2kAutomated safety check: PassCustom licence
Erd Studio Setupliam-machine/erd-studio165—~8.5kAutomated safety check: PassCustom licence
PR Verifydocglow/docglow147—~1.5kAutomated safety check: PassMIT
Migrating Dagster To Airflowastronomer/agents450—~3.8kAutomated safety check: PassApache-2.0

Similar skills

  • Dbt Databricks PR Ready

    databricks/dbt-databricks

    Official

    A skill your agent uses for an open dbt-databricks pull request, including your own PR or a fork PR, to assess merge readiness and optionally repair selected gaps on the PR head branch.

    379 GitHub stars~2.8k tokensUpdated today
    Data & AnalyticsAuto-check passed
  • Mz Dbt Release

    MaterializeInc/materialize

    Cut a dbt-materialize PyPI release: bump the version in version.py and setup.py, date the Unreleased CHANGELOG entry, and open the release PR with a Ship: <url body.

    6.4k GitHub stars~1.2k tokensUpdated today
    Data & AnalyticsAuto-check passed
  • Erd Studio Setup

    liam-machine/erd-studio

    Friendly, step-by-step setup for ERD Studio in an existing dbt project, for people who may be new to dbt or data modelling.

    165 GitHub stars~8.5k tokensUpdated 3 days ago
    Data & AnalyticsAuto-check passed
  • PR Verify

    docglow/docglow

    Verify a Docglow change actually works before submitting or merging a PR.

    147 GitHub stars~1.5k tokensUpdated 10 days ago
    Data & AnalyticsAuto-check passed
  • Guide for migrating Dagster projects to Apache Airflow 3 on Astro.

    450 GitHub stars~3.8k tokensUpdated yesterday
    Data & AnalyticsAuto-check passed
  • Dbt Parser Refresh

    yu-iskw/dbt-artifacts-parser

    Refreshes dbt artifact schemas from dbt-labs/dbt-core and regenerates Pydantic parser classes.

    118 GitHub stars~716 tokensUpdated 13 days ago
    Data & AnalyticsAuto-check passed

More from aiskillstore/marketplace

All 1,085 skills in this repo
  • Code Stats

    aiskillstore/marketplace

    Analyze codebase with tokei (fast line counts by language) and difft (semantic AST-aware diffs).

    430 GitHub starsUsed in 2 repos~697 tokens
    Auto-check: notes
  • Data Processing

    aiskillstore/marketplace

    Process JSON with jq and YAML/TOML with yq. An agent skill from aiskillstore/marketplace.

    430 GitHub starsUsed in 1 repo~720 tokens
    Auto-check: notes
  • Doc Scanner

    aiskillstore/marketplace

    Scans for project documentation files (AGENTS.md, CLAUDE.md, GEMINI.md, COPILOT.md, CURSOR.md, WARP.md, and 15+ other formats) and synthesizes guidance.

    430 GitHub starsUsed in 1 repo~644 tokens
    Auto-check: notes
  • File Search

    aiskillstore/marketplace

    Modern file and content search using fd, ripgrep (rg), and fzf.

    430 GitHub starsUsed in 1 repo~598 tokens
    Auto-check: notes
  • Find Replace

    aiskillstore/marketplace

    Modern find-and-replace using sd (simpler than sed) and batch replacement patterns.

    430 GitHub starsUsed in 1 repo~527 tokens
    Auto-check: notes
  • Investigating Codebases

    aiskillstore/marketplace

    Automatically activated when user asks how something works, wants to understand unfamiliar code, needs to explore a new codebase, or asks questions like "where is X implemented?", "how does Y…

    430 GitHub starsUsed in 1 repo~2.7k tokens
    Auto-check: notes

Works with

Questions about Data Engineering Data Pipeline

What does Data Engineering Data Pipeline do?

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing. Data Engineering Data Pipeline is an agent skill from aiskillstore/marketplace. You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

When should I use Data Engineering Data Pipeline?

Data Engineering Data Pipeline fits situations like: tasks that involve Data pipelines and ETL.

How do I install Data Engineering Data Pipeline in Claude Code?

Run `npx skills add aiskillstore/marketplace --skill data-engineering-data-pipeline -a claude-code`. Or copy the skill folder (skills/sickn33/data-engineering-data-pipeline in aiskillstore/marketplace) into .claude/skills/data-engineering-data-pipeline in your project. Claude Code loads it when a task matches its description.

How do I install Data Engineering Data Pipeline in Codex?

Run `npx skills add aiskillstore/marketplace --skill data-engineering-data-pipeline -a codex`. Or copy the skill folder (skills/sickn33/data-engineering-data-pipeline in aiskillstore/marketplace) into .agents/skills/data-engineering-data-pipeline in your project. Codex loads it when a task matches its description.

Can I use Data Engineering Data Pipeline in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add aiskillstore/marketplace --skill data-engineering-data-pipeline -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/data-engineering-data-pipeline, .gemini/skills/data-engineering-data-pipeline, .github/skills/data-engineering-data-pipeline and .opencode/skills/data-engineering-data-pipeline in your project.

What does Data Engineering Data Pipeline need to run?

SKILL.md names no scripts, command-line tools or credentials: Data Engineering Data Pipeline is instructions for the agent only. Our summary lists: Python 3; Docker.

Does Data Engineering Data Pipeline access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Data Engineering Data Pipeline safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Data Engineering Data Pipeline use?

No licence was found for Data Engineering Data Pipeline or its repository. Without one, default copyright applies: ask the author before reusing or redistributing it.

How many tokens does Data Engineering Data Pipeline use?

About 1.8k tokens (SKILL.md is roughly 7.3k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Data Engineering Data Pipeline?

Skills that share tags, products or a category with Data Engineering Data Pipeline: Dbt Databricks PR Ready (databricks/dbt-databricks, 379 stars), Mz Dbt Release (MaterializeInc/materialize, 6.4k stars), Erd Studio Setup (liam-machine/erd-studio, 165 stars) and PR Verify (docglow/docglow, 147 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Data Engineering Data Pipeline?

aiskillstore (a GitHub organization) maintains it in aiskillstore/marketplace, which has 430 GitHub stars. The repository holds 1,085 skills in this directory. The repository was last updated on October 7, 2026.

Source: aiskillstore/marketplace on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.