Agent skill

Validating AI Ethics And Fairness

by jeremylongshore in jeremylongshore/tons-of-skills-marketplace

Validate AI/ML models and datasets for bias, fairness, and ethical concerns.

MITAuto-check passedLegal & Compliance

Install Validating AI Ethics And Fairness

skills CLI
$ npx skills add jeremylongshore/tons-of-skills-marketplace --skill validating-ai-ethics-and-fairness -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install jeremylongshore/tons-of-skills-marketplace validating-ai-ethics-and-fairness --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/jeremylongshore/tons-of-skills-marketplace.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/.curated/validating-ai-ethics-and-fairness .claude/skills/validating-ai-ethics-and-fairness && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
validating-ai-ethics-and-fairness
GitHub stars
2.8k
Token cost
~1.5k tokens
SKILL.md length
662 words
Files
10 (incl. scripts, references, assets)
Skills in repo
3,342
Repo updated
First seen
Licence
MIT

At a glance

Validate AI/ML models and datasets for bias, fairness, and ethical concerns.

  • Works in 10 steps: Load the model predictions and ground… → Define the protected attributes and… → Compute representation statistics: group… → …
  • Auditing AI systems for ethical compliance
  • SKILL.md covers Overview, Prerequisites, Instructions and Output, plus 3 more sections
  • Runs Python scripts from its folder; calls pip

What it does

Validating AI Ethics And Fairness is an agent skill from jeremylongshore/tons-of-skills-marketplace. Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics".

Its SKILL.md is about 1.5k tokens, which your agent loads only when the skill is triggered. The skill folder holds 12 other files, including scripts, reference files and assets (for example `assets/README.md`, `assets/report_template.md` and `references/README.md`). Compatibility notes: Designed for Claude Code

It sits in Legal & Compliance, covering AI governance and Machine learning. The repository describes itself as: Model-agnostic agent-skills platform with a harness-free canonical layer, verified adapters, and the ccpi package manager. Explore at tonsofskills.com. The licence is MIT.

When your agent uses it

  • Auditing AI systems for ethical compliance
  • Fairness assessment
  • With phrases like evaluate model fairness
  • Validate AI ethics

Example prompts

  • “evaluate model fairness”
  • “check for bias”
  • “validate AI ethics”
  • “/validating-ai-ethics-and-fairness”

Requirements

  • Python 3
  • Compatibility (from SKILL.md): Designed for Claude Code
  • Pre-approved tools (allowed-tools): Read, Write, Edit, Grep, Glob, Bash(python:*)

Workflow steps

10 steps, taken from the first numbered list in SKILL.md.

  1. Load the model predictions and ground truth dataset using the Read tool; verify schema includes sensitive attribute columns
  2. Define the protected attributes and privileged/unprivileged group definitions for the fairness analysis
  3. Compute representation statistics: group counts, class label distributions, and feature coverage per demographic segment
  4. Calculate core fairness metrics using Fairlearn or AIF360
  5. Apply four-fifths rule: flag any metric where the ratio falls below 0.80 as potential adverse impact
  6. Classify each finding by severity: low (ratio 0.90-1.0), medium (0.80-0.90), high (0.70-0.80), critical (below 0.70)
  7. Identify proxy variables by computing correlation between non-protected features and sensitive attributes
  8. Generate mitigation recommendations: resampling, reweighting, threshold adjustment, or in-processing constraints (e.g…
  9. Produce a compliance assessment mapping findings to IEEE Ethically Aligned Design, EU Ethics Guidelines for Trustworthy AI, and ACM Code…
  10. Document all ethical decisions, trade-offs, and residual risks in a structured audit report

What it can do on your machine

Read from SKILL.md and the folder at commit cfae287. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves these tools, so the agent can use them without asking each time:

    • Read
    • Write
    • Edit
    • Grep
    • Glob
    • Bash(python:*)

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 4 files in scripts/ (Python), which the agent can run.

    Shell commands in SKILL.md call:

    • pip

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • fairlearn.org
    • pair-code.github.io
    • ethicsinaction.ieee.org

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

  • Compatibility

    Designed for Claude Code

    From compatibility in the SKILL.md frontmatter.

Context cost

Validating AI Ethics And Fairness loads about 1.5k tokens when it runs, and up to ~1.5k if it reads all its reference files. Until then it costs about 75 tokens; SKILL.md has 662 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~75
When it runs · the whole SKILL.md, loaded when a task matches
~1.5k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~1.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from jeremylongshore/tons-of-skills-marketplace at commit cfae287, republished under its MIT licence (© jeremylongshore). 662 words, ~1,516 tokens.

Download SKILL.mdSave it as .claude/skills/validating-ai-ethics-and-fairness/SKILL.md (or your agent's skills folder). This skill also uses 9 other files; get the full folder from GitHub.
name
validating-ai-ethics-and-fairness
description
Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics".
allowed-tools
Read, Write, Edit, Grep, Glob, Bash(python:*)
compatibility
Designed for Claude Code
version
1.24.0
author
Jeremy Longshore <jeremy@intentsolutions.io>
license
MIT
tags
ai, compliance, audit

AI Ethics Validator

Overview

Validate AI/ML models and datasets for bias, fairness, and ethical compliance using quantitative fairness metrics and structured audit workflows.

Prerequisites

  • Python 3.9+ with Fairlearn >= 0.9 (pip install fairlearn)
  • IBM AI Fairness 360 toolkit (pip install aif360) for comprehensive bias analysis
  • pandas, NumPy, and scikit-learn for data manipulation and model evaluation
  • Model predictions (probabilities or binary labels) and corresponding ground truth labels
  • Demographic attribute columns (age, gender, race, etc.) accessible under appropriate data governance
  • Optional: Google What-If Tool for interactive fairness exploration on TensorFlow models

Instructions

  1. Load the model predictions and ground truth dataset using the Read tool; verify schema includes sensitive attribute columns
  2. Define the protected attributes and privileged/unprivileged group definitions for the fairness analysis
  3. Compute representation statistics: group counts, class label distributions, and feature coverage per demographic segment
  4. Calculate core fairness metrics using Fairlearn or AIF360:
    • Demographic parity ratio (selection rate parity across groups)
    • Equalized odds difference (TPR and FPR parity)
    • Equal opportunity difference (TPR parity only)
    • Predictive parity (precision parity across groups)
    • Calibration scores per group (predicted probability vs observed outcome)
  5. Apply four-fifths rule: flag any metric where the ratio falls below 0.80 as potential adverse impact
  6. Classify each finding by severity: low (ratio 0.90-1.0), medium (0.80-0.90), high (0.70-0.80), critical (below 0.70)
  7. Identify proxy variables by computing correlation between non-protected features and sensitive attributes
  8. Generate mitigation recommendations: resampling, reweighting, threshold adjustment, or in-processing constraints (e.g., ExponentiatedGradient from Fairlearn)
  9. Produce a compliance assessment mapping findings to IEEE Ethically Aligned Design, EU Ethics Guidelines for Trustworthy AI, and ACM Code of Ethics
  10. Document all ethical decisions, trade-offs, and residual risks in a structured audit report

Output

  • Fairness metric dashboard: per-group values for demographic parity, equalized odds, equal opportunity, predictive parity, and calibration
  • Severity-classified findings table: metric name, affected groups, ratio value, severity level, recommended action
  • Representation analysis: group sizes, class distributions, feature coverage gaps
  • Proxy variable report: features correlated with protected attributes above threshold (r > 0.3)
  • Mitigation plan: ranked strategies with expected fairness improvement and accuracy trade-off estimates
  • Compliance matrix: pass/fail against IEEE, EU, and ACM ethical guidelines with evidence citations
Show full SKILL.md (307 more words)Show less

Error Handling

ErrorCauseSolution
Insufficient group sample sizeFewer than 30 observations in a demographic groupAggregate related subgroups; use bootstrap confidence intervals; flag metric as unreliable
Missing sensitive attributesProtected attribute columns absent from datasetApply proxy detection via correlated features; request attribute access under data governance approval
Conflicting fairness criteriaDemographic parity and equalized odds contradictDocument the impossibility theorem trade-off; prioritize the metric most aligned with the deployment context
Data quality failuresInconsistent encoding or null values in attribute columnsStandardize categorical encodings; impute or exclude nulls; validate with schema checks before analysis
Model output format mismatchPredictions not in expected probability or binary formatConvert logits to probabilities via sigmoid; binarize at the decision threshold before metric computation

Examples

Scenario 1: Hiring Model Audit -- Validate a resume-screening classifier for gender and age bias. Compute demographic parity across male/female groups and age buckets (18-30, 31-50, 51+). Apply the four-fifths rule. Finding: female selection rate at 0.72 of male rate (critical severity). Recommend reweighting training samples and adjusting the decision threshold.

Scenario 2: Credit Scoring Fairness -- Assess a credit approval model for racial disparate impact. Calculate equalized odds (TPR and FPR) across racial groups. Finding: FPR for Group A is 2.1x Group B (high severity). Recommend in-processing constraint using ExponentiatedGradient with FalsePositiveRateParity.

Scenario 3: Healthcare Risk Prediction -- Evaluate a patient risk model for age and socioeconomic bias. Compute calibration curves per group. Finding: model overestimates risk for low-income patients by 15%. Recommend recalibration using Platt scaling per subgroup with post-deployment monitoring for fairness drift.

Resources

© jeremylongshore, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 9 other files (scripts, references, assets) in skills/.curated/validating-ai-ethics-and-fairness of jeremylongshore/tons-of-skills-marketplace.

  • SKILL.md
  • assets/README.md
  • assets/example_dataset.csv
  • assets/example_model.pkl
  • assets/report_template.md
  • references/README.md
  • scripts/README.md
  • scripts/generate_report.py
  • scripts/validate_dataset.py
  • scripts/validate_model.py

Open the folder on GitHubat commit cfae287

Compare with similar skills

Validating AI Ethics And Fairness next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Validating AI Ethics And Fairness compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Validating AI Ethics And Fairness this skilljeremylongshore/tons-of-skills-marketplace2.8k—~1.5kAutomated safety check: PassMIT
Iso42001Sushegaad/Claude-Skills-Governance-Risk-and-Compliance9461 repos~3.7kAutomated safety check: PassMIT
AI Risk Managementbriiirussell/cybersecurity-skills413—~3.7kAutomated safety check: NotesMIT
EU AI Act System Inventoryanthropics/claude-for-legal9.6k3 repos~2.8kAutomated safety check: PassApache-2.0
Eu AI Act Readinessseb1n/awesome-ai-agent-skills206—~3.3kAutomated safety check: PassMIT
AI GovernanceHack23/cia239—~1.4kAutomated safety check: PassApache-2.0

Similar skills

  • Iso42001

    Sushegaad/Claude-Skills-Governance-Risk-and-Compliance

    Expert ISO 42001 AI Management System (AIMS) compliance advisor.

    946 GitHub starsUsed in 1 repo~3.7k tokens
    Legal & ComplianceAuto-check passed
  • AI Risk Management

    briiirussell/cybersecurity-skills

    Apply the NIST AI Risk Management Framework (AI RMF 1.0) and adjacent guidance to AI / ML systems — model lifecycle governance, fairness and bias evaluation, robustness, transparency…

    413 GitHub stars~3.7k tokensUpdated 4 mo ago
    Legal & ComplianceAuto-check: notes
  • EU AI Act System Inventory

    anthropics/claude-for-legal

    Official

    Maintains a register of AI systems under the EU AI Act, recording each system's role and risk tier separately, because both can differ from one system to the next.

    9.6k GitHub starsUsed in 3 repos~2.8k tokens
    Legal & ComplianceAuto-check passed
  • Eu AI Act Readiness

    seb1n/awesome-ai-agent-skills

    Build a preliminary, evidence-based EU AI Act readiness assessment across AI-system inventory, territorial scope, operator roles, prohibited-practice screening, risk classification, transparency…

    206 GitHub stars~3.3k tokensUpdated 2 mo ago
    Legal & ComplianceAuto-check passed
  • AI Governance

    Hack23/cia

    AI governance, EU AI Act compliance, OWASP LLM security, responsible AI practices for GitHub Copilot agents

    239 GitHub stars~1.4k tokensUpdated today
    Legal & ComplianceAuto-check passed
  • Compliance Testing

    petrkindlmann/qa-skills

    Test for regulatory compliance: GDPR/CMP consent verification, Google Consent Mode v2, Global Privacy Control (GPC), CCPA/US state opt-out, EU AI Act Article 50 transparency, Better Ads Standards…

    170 GitHub stars~4.6k tokensUpdated 4 mo ago
    Legal & ComplianceAuto-check passed

More from jeremylongshore/tons-of-skills-marketplace

All 3,342 skills in this repo
  • Performing Security Code Review

    jeremylongshore/tons-of-skills-marketplace

    Execute this skill enables AI assistant to conduct a security-focused code review using the security-agent plugin.

    2.8k GitHub starsUsed in 2 repos~1.3k tokens
    Auto-check: notes
  • Adapting Transfer Learning Models

    jeremylongshore/tons-of-skills-marketplace

    Build this skill automates the adaptation of pre-trained machine learning models using transfer learning techniques.

    2.8k GitHub stars~1.1k tokensUpdated today
    Auto-check passed
  • Agent Context Loader

    jeremylongshore/tons-of-skills-marketplace

    Execute proactive auto-loading: automatically detects and loads agents.md files.

    2.8k GitHub stars~1.1k tokensUpdated today
    Auto-check passed
  • Aggregating Performance Metrics

    jeremylongshore/tons-of-skills-marketplace

    Aggregate and centralize performance metrics from applications, systems, databases, caches, and services.

    2.8k GitHub stars~1.2k tokensUpdated today
    Auto-check passed
  • Analyzing Capacity Planning

    jeremylongshore/tons-of-skills-marketplace

    Execute this skill enables AI assistant to analyze capacity requirements and plan for future growth.

    2.8k GitHub stars~947 tokensUpdated today
    Auto-check passed
  • Analyzing Database Indexes

    jeremylongshore/tons-of-skills-marketplace

    Process use when you need to work with database indexing. An agent skill from jeremylongshore/tons-of-skills-marketplace.

    2.8k GitHub stars~2k tokensUpdated today
    Auto-check passed

Questions about Validating AI Ethics And Fairness

What does Validating AI Ethics And Fairness do?

Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Validating AI Ethics And Fairness is an agent skill from jeremylongshore/tons-of-skills-marketplace. Validate AI/ML models and datasets for bias, fairness, and ethical concerns.

When should I use Validating AI Ethics And Fairness?

Validating AI Ethics And Fairness fits situations like: auditing AI systems for ethical compliance; fairness assessment; with phrases like evaluate model fairness; validate AI ethics.

How do I install Validating AI Ethics And Fairness in Claude Code?

Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill validating-ai-ethics-and-fairness -a claude-code`. Or copy the skill folder (skills/.curated/validating-ai-ethics-and-fairness in jeremylongshore/tons-of-skills-marketplace) into .claude/skills/validating-ai-ethics-and-fairness in your project. Claude Code loads it when a task matches its description.

How do I install Validating AI Ethics And Fairness in Codex?

Run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill validating-ai-ethics-and-fairness -a codex`. Or copy the skill folder (skills/.curated/validating-ai-ethics-and-fairness in jeremylongshore/tons-of-skills-marketplace) into .agents/skills/validating-ai-ethics-and-fairness in your project. Codex loads it when a task matches its description.

Can I use Validating AI Ethics And Fairness in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add jeremylongshore/tons-of-skills-marketplace --skill validating-ai-ethics-and-fairness -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/validating-ai-ethics-and-fairness, .gemini/skills/validating-ai-ethics-and-fairness, .github/skills/validating-ai-ethics-and-fairness and .opencode/skills/validating-ai-ethics-and-fairness in your project.

What does Validating AI Ethics And Fairness need to run?

Going by SKILL.md and its folder, Validating AI Ethics And Fairness needs Python for the scripts in its folder and the command-line tools its instructions call (pip). Our summary lists: Python 3. Its frontmatter pre-approves these tools: Read, Write, Edit, Grep, Glob, Bash(python:*). Compatibility (from SKILL.md): Designed for Claude Code.

Does Validating AI Ethics And Fairness access the network?

SKILL.md names 3 domains. As links in the text: fairlearn.org, pair-code.github.io and ethicsinaction.ieee.org. This is read from the text; nothing was executed.

Is Validating AI Ethics And Fairness safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Validating AI Ethics And Fairness use?

Validating AI Ethics And Fairness is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Validating AI Ethics And Fairness use?

About 1.5k tokens (SKILL.md is roughly 6.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 16 tokens, read only when the agent opens those files.

What are the alternatives to Validating AI Ethics And Fairness?

Skills that share tags, products or a category with Validating AI Ethics And Fairness: Iso42001 (Sushegaad/Claude-Skills-Governance-Risk-and-Compliance, 946 stars), AI Risk Management (briiirussell/cybersecurity-skills, 413 stars), EU AI Act System Inventory (anthropics/claude-for-legal, 9.6k stars) and Eu AI Act Readiness (seb1n/awesome-ai-agent-skills, 206 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Validating AI Ethics And Fairness?

jeremylongshore (a GitHub user) maintains it in jeremylongshore/tons-of-skills-marketplace, which has 2,827 GitHub stars. The repository holds 3,342 skills in this directory. The repository was last updated on October 10, 2026.

Source: jeremylongshore/tons-of-skills-marketplace on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.