New: 97,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Data Quality Frameworks

by wshobson in wshobson/agents

Sets up data quality checks with Great Expectations, dbt tests and data contracts, with checkpoints and pass-fail reports for pipelines.

MITAuto-check passedData & Analytics

Install Data Quality Frameworks

skills CLI
$ npx skills add wshobson/agents --skill data-quality-frameworks -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install wshobson/agents data-quality-frameworks --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/wshobson/agents.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugins/data-engineering/skills/data-quality-frameworks .claude/skills/data-quality-frameworks && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
data-quality-frameworks
GitHub stars
40k
Used in
10 other repos
Token cost
~1.1k tokens
SKILL.md length
205 words
Files
2 (incl. references)
Skills in repo
142
Repo updated
First seen
Licence
MIT

At a glance

Sets up data quality checks with Great Expectations, dbt tests and data contracts, with checkpoints and pass-fail reports for pipelines.

  • Works in 2 steps: Data Quality Dimensions → Testing Pyramid for Data
  • Adding data quality checks to a pipeline
  • SKILL.md covers When to Use This Skill, Core Concepts, Quick Start and Detailed patterns and worked…, plus 1 more section
  • Calls pip

What it does

This skill covers three ways to check data: Great Expectations validation, dbt tests and data contracts between teams. It frames quality through six dimensions, completeness, uniqueness, validity, accuracy, consistency and timeliness, and maps each to an example check such as a not-null or unique-values expectation.

It also describes a testing pyramid for data, with integration tests across tables at the top, and gives a quick start that installs Great Expectations and sets up a daily validation checkpoint in Python. A report-building example summarizes how many tables passed and lists the failed checks per table. Further patterns and worked examples sit in a details reference file.

When your agent uses it

  • Adding data quality checks to a pipeline
  • Setting up Great Expectations validation and checkpoints
  • Building out a dbt test suite for models
  • Agreeing data contracts between producing and consuming teams
  • Running data validation automatically in CI/CD

Example prompts

  • “Add Great Expectations checks for null and duplicate order IDs in the orders table.”
  • “Write dbt tests for the customers model and draft a data contract with the billing team.”
  • “Set up a daily validation checkpoint and fail the CI job when a table fails.”
  • “Generate a Markdown report of which tables passed their expectations.”

Requirements

  • Python with great_expectations installed through pip

Workflow steps

2 steps, taken from the step headings in SKILL.md.

  1. Data Quality Dimensions
  2. Testing Pyramid for Data

What it can do on your machine

Read from SKILL.md and the folder at commit 46891e7. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • pip

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use pip, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Data Quality Frameworks loads about 1.1k tokens when it runs, and up to ~4k if it reads all its reference files. Until then it costs about 55 tokens; SKILL.md has 205 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~55
When it runs · the whole SKILL.md, loaded when a task matches
~1.1k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from wshobson/agents at commit 46891e7, republished under its MIT licence (© wshobson). 205 words, ~1,112 tokens.

Download SKILL.mdSave it as .claude/skills/data-quality-frameworks/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.
name
data-quality-frameworks
description
Implement data quality validation with Great Expectations, dbt tests, and data contracts. Use when building data quality pipelines, implementing validation rules, or establishing data contracts.

Data Quality Frameworks

Production patterns for implementing data quality with Great Expectations, dbt tests, and data contracts to ensure reliable data pipelines.

When to Use This Skill

  • Implementing data quality checks in pipelines
  • Setting up Great Expectations validation
  • Building comprehensive dbt test suites
  • Establishing data contracts between teams
  • Monitoring data quality metrics
  • Automating data validation in CI/CD

Core Concepts

1. Data Quality Dimensions
DimensionDescriptionExample Check
CompletenessNo missing valuesexpect_column_values_to_not_be_null
UniquenessNo duplicatesexpect_column_values_to_be_unique
ValidityValues in expected rangeexpect_column_values_to_be_in_set
AccuracyData matches realityCross-reference validation
ConsistencyNo contradictionsexpect_column_pair_values_A_to_be_greater_than_B
TimelinessData is recentexpect_column_max_to_be_between
2. Testing Pyramid for Data
          /\
         /  \     Integration Tests (cross-table)
        /────\
       /      \   Unit Tests (single column)
      /────────\
     /          \ Schema Tests (structure)
    /────────────\

Quick Start

Great Expectations Setup
bash
# Install
pip install great_expectations

# Initialize project
great_expectations init

# Create datasource
great_expectations datasource new
python
# great_expectations/checkpoints/daily_validation.yml
import great_expectations as gx

# Create context
context = gx.get_context()

# Create expectation suite
suite = context.add_expectation_suite("orders_suite")

# Add expectations
suite.add_expectation(
    gx.expectations.ExpectColumnValuesToNotBeNull(column="order_id")
)
suite.add_expectation(
    gx.expectations.ExpectColumnValuesToBeUnique(column="order_id")
)

# Validate
results = context.run_checkpoint(checkpoint_name="daily_orders")

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.

Summary: {total_passed}/{total_tables} tables passed")

    report.append("")

    for table, result in results.items():
        status = "✅" if result.passed else "❌"
        report.append(f"### {status} {table}")
        report.append(f"- Expectations: {result.total_expectations}")
        report.append(f"- Failed: {result.failed_expectations}")

        if not result.passed:
            report.append("- Failed checks:")
            for detail in result.details:
                if not detail["success"]:
                    report.append(f"  - {detail['expectation']}: {detail['observed_value']}")
        report.append("")

    return "\n".join(report)

Usage

context = gx.get_context() pipeline = DataQualityPipeline(context)

tables_to_validate = { "orders": "orders_suite", "customers": "customers_suite", "products": "products_suite", }

results = pipeline.run_all(tables_to_validate) report = pipeline.generate_report(results)

Fail pipeline if any table failed

if not all(r.passed for r in results.values()): print(report) raise ValueError("Data quality checks failed!")


## Best Practices

### Do's

- **Test early** - Validate source data before transformations
- **Test incrementally** - Add tests as you find issues
- **Document expectations** - Clear descriptions for each test
- **Alert on failures** - Integrate with monitoring
- **Version contracts** - Track schema changes

### Don'ts

- **Don't test everything** - Focus on critical columns
- **Don't ignore warnings** - They often precede failures
- **Don't skip freshness** - Stale data is bad data
- **Don't hardcode thresholds** - Use dynamic baselines
- **Don't test in isolation** - Test relationships too

© wshobson, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 1 other file (references) in plugins/data-engineering/skills/data-quality-frameworks of wshobson/agents.

  • SKILL.md
  • references/details.md

Open the folder on GitHubat commit 46891e7

Used in 10 other repositories

We found 28 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 10 other GitHub owners. This page covers the copy in wshobson/agents, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Data Quality Frameworks next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Data Quality Frameworks compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Data Quality Frameworks this skillwshobson/agents40k10 repos~1.1kAutomated safety check: PassMIT
Credit Risk Data Cleaninggithub/awesome-copilot40k1 repos~1.5kAutomated safety check: PassMIT
Dbt Parser Refreshyu-iskw/dbt-artifacts-parser118—~716Automated safety check: PassApache-2.0
Suggesting Dbt Bouncer Checksgodatadriven/dbt-bouncer135—~1kAutomated safety check: PassMIT
Authoritative Data Harvesteryushui2022/MathModel-Skill4521 repos~1.1kAutomated safety check: PassMIT
Package Version Bumpyu-iskw/dbt-artifacts-parser118—~700Automated safety check: PassApache-2.0

Similar skills

  • Credit Risk Data Cleaning

    github/awesome-copilot

    Official

    Cleans raw credit data and screens variables before loan modeling, dropping unstable, noisy or redundant features and writing an Excel report of every step.

    40k GitHub starsUsed in 1 repo~1.5k tokens
    Data & AnalyticsAuto-check passed

  • Dbt Parser Refresh

    yu-iskw/dbt-artifacts-parser

    Refreshes dbt artifact schemas from dbt-labs/dbt-core and regenerates Pydantic parser classes.

    118 GitHub stars~716 tokensUpdated 13 days ago
    Data & AnalyticsAuto-check passed

  • Suggesting Dbt Bouncer Checks

    godatadriven/dbt-bouncer

    Analyzes a dbt project and suggests dbt-bouncer checks that already pass (for existing projects) or a sensible starter config (for greenfield projects).

    135 GitHub stars~1k tokensUpdated 2 days ago
    Data & AnalyticsAuto-check passed

  • Authoritative Data Harvester

    yushui2022/MathModel-Skill

    Finds authoritative public data sources for modeling tasks, prefers official APIs and bulk downloads, and outputs a reproducible fetch and cleaning plan with citations.

    452 GitHub starsUsed in 1 repo~1.1k tokens
    Data & AnalyticsAuto-check passed

  • Package Version Bump

    yu-iskw/dbt-artifacts-parser

    Bumps the dbt-artifacts-parser package semver, prepares a PyPI release, and aligns GitHub Releases with version.

    118 GitHub stars~700 tokensUpdated 13 days ago
    Data & AnalyticsAuto-check passed

  • Senior Data Engineer

    benchflow-ai/skillsbench

    World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, real-time streaming, and data infrastructure.

    1.8k GitHub stars~5.9k tokensUpdated 2 mo ago
    Data & AnalyticsAuto-check passed

More from wshobson/agents

All 142 skills in this repo

  • Cloud Cost Optimization

    wshobson/agents

    Cuts cloud spend across AWS, Azure, GCP and OCI with cost tagging, rightsizing, commitment and spot pricing models, and architecture changes.

    40k GitHub starsUsed in 13 repos~1.7k tokens
    Auto-check passed

  • Billing Automation

    wshobson/agents

    Covers building subscription billing: billing cycles, subscription states, invoice generation, proration, tax handling and dunning for failed payments.

    40k GitHub starsUsed in 12 repos~473 tokens
    Auto-check passed

  • Python Performance Optimization

    wshobson/agents

    Profiles slow Python code with cProfile and memory profilers, then applies targeted fixes for CPU, memory, I/O and query bottlenecks.

    40k GitHub starsUsed in 12 repos~814 tokens
    Auto-check passed

  • Portfolio Risk Metrics

    wshobson/agents

    Covers portfolio risk measurement with VaR, CVaR, Sharpe, Sortino and drawdown, plus guidance on limits, stress tests and tail risk.

    40k GitHub starsUsed in 12 repos~502 tokens
    Auto-check passed

  • DGX Spark Memory and Thermal Ops

    wshobson/agents

    Plans memory headroom, works through out-of-memory failures and watches temperature and power during long ML training jobs on NVIDIA DGX Spark.

    40k GitHub starsUsed in 1 repo~2k tokens
    Auto-check passed

  • Bats Shell Testing Patterns

    wshobson/agents

    Writes unit tests for shell scripts with Bats: error-condition tests, fixtures and mocks, cross-shell checks, parallel runs, helper files and CI integration.

    40k GitHub starsUsed in 11 repos~1.3k tokens
    Auto-check passed

Works with

Categories

Questions about Data Quality Frameworks

What does Data Quality Frameworks do?

Sets up data quality checks with Great Expectations, dbt tests and data contracts, with checkpoints and pass-fail reports for pipelines. This skill covers three ways to check data: Great Expectations validation, dbt tests and data contracts between teams. It frames quality through six dimensions, completeness, uniqueness, validity, accuracy, consistency and timeliness, and maps each to an example check such as a not-null or unique-values expectation.

When should I use Data Quality Frameworks?

Data Quality Frameworks fits situations like: adding data quality checks to a pipeline; setting up Great Expectations validation and checkpoints; building out a dbt test suite for models; agreeing data contracts between producing and consuming teams.

How do I install Data Quality Frameworks in Claude Code?

Run `npx skills add wshobson/agents --skill data-quality-frameworks -a claude-code`. Or copy the skill folder (plugins/data-engineering/skills/data-quality-frameworks in wshobson/agents) into .claude/skills/data-quality-frameworks in your project. Claude Code loads it when a task matches its description.

How do I install Data Quality Frameworks in Codex?

Run `npx skills add wshobson/agents --skill data-quality-frameworks -a codex`. Or copy the skill folder (plugins/data-engineering/skills/data-quality-frameworks in wshobson/agents) into .agents/skills/data-quality-frameworks in your project. Codex loads it when a task matches its description.

Can I use Data Quality Frameworks in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add wshobson/agents --skill data-quality-frameworks -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/data-quality-frameworks, .gemini/skills/data-quality-frameworks, .github/skills/data-quality-frameworks and .opencode/skills/data-quality-frameworks in your project.

What does Data Quality Frameworks need to run?

Going by SKILL.md and its folder, Data Quality Frameworks needs the command-line tools its instructions call (pip). Our summary lists: Python with great_expectations installed through pip.

Does Data Quality Frameworks access the network?

SKILL.md contains no URLs. Its commands use pip, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Data Quality Frameworks safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Data Quality Frameworks use?

Data Quality Frameworks is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Data Quality Frameworks use?

About 1.1k tokens (SKILL.md is roughly 4.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 2.9k tokens, read only when the agent opens those files.

What are the alternatives to Data Quality Frameworks?

Skills that share tags, products or a category with Data Quality Frameworks: Credit Risk Data Cleaning (github/awesome-copilot, 40k stars), Dbt Parser Refresh (yu-iskw/dbt-artifacts-parser, 118 stars), Suggesting Dbt Bouncer Checks (godatadriven/dbt-bouncer, 135 stars) and Authoritative Data Harvester (yushui2022/MathModel-Skill, 452 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Data Quality Frameworks?

wshobson (a GitHub user) maintains it in wshobson/agents, which has 40,254 GitHub stars. The repository holds 142 skills in this directory. The repository was last updated on October 5, 2026.

Source: wshobson/agents on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.