Agent skill

Hunting Leakage And Validating

by flyrank-bih in flyrank-bih/flyrank-ml-internship-starter

Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist.

Custom licenceAuto-check passed

Install Hunting Leakage And Validating

skills CLI
$ npx skills add flyrank-bih/flyrank-ml-internship-starter --skill hunting-leakage-and-validating -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install flyrank-bih/flyrank-ml-internship-starter hunting-leakage-and-validating --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/flyrank-bih/flyrank-ml-internship-starter.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/hunting-leakage-and-validating .claude/skills/hunting-leakage-and-validating && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
hunting-leakage-and-validating
GitHub stars
140
Token cost
~974 tokens
SKILL.md length
551 words
Files
1
Skills in repo
13
Repo updated
First seen
Licence
Custom licence

At a glance

Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist.

  • SKILL.md covers The leakage taxonomy — three…, Honest splits — random is…, Always print the base rate and Sealed holdouts leave receipts, plus 2 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Hunting Leakage And Validating is an agent skill from flyrank-bih/flyrank-ml-internship-starter. Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist. Use before trusting any metric, when a score looks too good, or when features and labels share time windows or origins.

Its SKILL.md is about 970 tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

The repository describes itself as: Starter repo for the FlyRank ML Internship - a runnable ML pipeline on real anonymized Google Search data, with Colab notebooks. Fork it, build your capstone in it.

Example prompts

  • “Use the hunting-leakage-and-validating skill to find label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits…”
  • “/hunting-leakage-and-validating”

What it can do on your machine

Read from SKILL.md and the folder at commit 882b73e. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Hunting Leakage And Validating loads about 974 tokens when it runs. Until then it costs about 76 tokens; SKILL.md has 551 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~76
When it runs · the whole SKILL.md, loaded when a task matches
~974

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 551 words (~974 tokens).

“The sneakiest failure in applied ML: the model quietly reads the answer during training. Scores look amazing; the work is worthless. Your job is to attack your own model before anyone else can.”

— opening of SKILL.md by flyrank-bih, Custom licence
name
hunting-leakage-and-validating

Read the full SKILL.md on GitHub

Files

Just SKILL.md in skills/hunting-leakage-and-validating of flyrank-bih/flyrank-ml-internship-starter.

Open the folder on GitHubat commit 882b73e

Compare with similar skills

Hunting Leakage And Validating next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Hunting Leakage And Validating compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Hunting Leakage And Validating this skillflyrank-bih/flyrank-ml-internship-starter140—~974Automated safety check: PassCustom licence
Design Systemaffaan-m/ECC276k—~698Automated safety check: PassMIT
Design Guidepaperclipai/paperclip100k1 repos~3.1kAutomated safety check: PassMIT
Design Audit Against Rams' Principlesthedotmack/claude-mem99k—~4.6kAutomated safety check: PassApache-2.0
Figma Design to Codewarpdotdev/warp65k4 repos~2.9kAutomated safety check: PassAGPL-3.0
Design Consultationgarrytan/gstack136k—~16kAutomated safety check: NotesMIT

Similar skills

  • Design System

    affaan-m/ECC

    Generate a design system from an existing codebase or audit one for visual consistency: extract tokens (colors, typography, spacing, shadows) into design-tokens.json and CSS custom properties with…

    276k GitHub stars~698 tokensUpdated yesterday
    Frontend & DesignAuto-check passed
  • Design Guide

    paperclipai/paperclip

    Paperclip UI design system guide for building consistent, reusable frontend components.

    100k GitHub starsUsed in 1 repo~3.1k tokens
    Frontend & DesignAuto-check passed
  • Audits a design against Dieter Rams' ten principles of good design, scores each with evidence, and hands off a make-plan prompt for a new, refined or redesigned outcome.

    99k GitHub stars~4.6k tokensUpdated 2 days ago
    Frontend & DesignAuto-check passed
  • Figma Design to Code

    warpdotdev/warp

    Turns a Figma frame or component into production code that matches the design, using the Figma MCP server and the project's own design system.

    65k GitHub starsUsed in 4 repos~2.9k tokens
    Frontend & DesignAuto-check passed
  • Design Consultation

    garrytan/gstack

    Learns about your product, studies the landscape and writes a DESIGN.md with a full design system covering type, color, layout, spacing and motion.

    136k GitHub stars~16k tokensUpdated today
    Frontend & DesignAuto-check: notes
  • Product Design Workflow Bundle

    XiaomiMiMo/MiMo-Code

    Entry point to a bundle of product design workflows covering context, research, audits, ideation, URL or image to code, design QA and sharing a prototype.

    14k GitHub stars~721 tokensUpdated 2 days ago
    Frontend & DesignAuto-check passed

More from flyrank-bih/flyrank-ml-internship-starter

All 13 skills in this repo
  • Querying Big Datasets

    flyrank-bih/flyrank-ml-internship-starter

    Works with datasets far too big to download or load in pandas — SQL over remote Parquet with DuckDB, aggregate-then-model, iterate on samples.

    140 GitHub stars~750 tokensUpdated 1 mo ago
    Auto-check passed
  • Building Baselines

    flyrank-bih/flyrank-ml-internship-starter

    Builds the transparent rule-based baseline every model must beat — a hand-written score with reason codes, ranked output, and precision@K evaluation.

    140 GitHub stars~587 tokensUpdated 1 mo ago
    Auto-check passed
  • Deploying Static Pages

    flyrank-bih/flyrank-ml-internship-starter

    Deploys a static page (research paper, portfolio piece) for free from a GitHub repo using GitHub Pages — setup, file layout, verification, and recording the final URL.

    140 GitHub stars~564 tokensUpdated 1 mo ago
    Auto-check passed
  • Framing ML Problems

    flyrank-bih/flyrank-ml-internship-starter

    Frames a data/ML problem before any modeling — the decision, the action, the cost of a wrong call, task type, target, and success metric.

    140 GitHub stars~700 tokensUpdated 1 mo ago
    Auto-check passed
  • Training Honest Models

    flyrank-bih/flyrank-ml-internship-starter

    Trains a first model the honest way — method chosen to fit the question, compared against the baseline on the same split and metric, errors read before scores are believed.

    140 GitHub stars~588 tokensUpdated 1 mo ago
    Auto-check passed
  • Writing Honest Claims

    flyrank-bih/flyrank-ml-internship-starter

    Writes findings in language the evidence can carry — the claim ladder (observed → directional → decision-support, never causal without a design), effect sizes over drama, banned phrasings.

    140 GitHub stars~665 tokensUpdated 1 mo ago
    Auto-check passed

Questions about Hunting Leakage And Validating

What does Hunting Leakage And Validating do?

Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist. Hunting Leakage And Validating is an agent skill from flyrank-bih/flyrank-ml-internship-starter. Finds label leakage and designs honest validation — leakage taxonomy, grouped and time-aware splits, base rates, and the attack-your-own-model checklist.

How do I install Hunting Leakage And Validating in Claude Code?

Run `npx skills add flyrank-bih/flyrank-ml-internship-starter --skill hunting-leakage-and-validating -a claude-code`. Or copy the skill folder (skills/hunting-leakage-and-validating in flyrank-bih/flyrank-ml-internship-starter) into .claude/skills/hunting-leakage-and-validating in your project. Claude Code loads it when a task matches its description.

How do I install Hunting Leakage And Validating in Codex?

Run `npx skills add flyrank-bih/flyrank-ml-internship-starter --skill hunting-leakage-and-validating -a codex`. Or copy the skill folder (skills/hunting-leakage-and-validating in flyrank-bih/flyrank-ml-internship-starter) into .agents/skills/hunting-leakage-and-validating in your project. Codex loads it when a task matches its description.

Can I use Hunting Leakage And Validating in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add flyrank-bih/flyrank-ml-internship-starter --skill hunting-leakage-and-validating -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/hunting-leakage-and-validating, .gemini/skills/hunting-leakage-and-validating, .github/skills/hunting-leakage-and-validating and .opencode/skills/hunting-leakage-and-validating in your project.

What does Hunting Leakage And Validating need to run?

SKILL.md names no scripts, command-line tools or credentials: Hunting Leakage And Validating is instructions for the agent only.

Does Hunting Leakage And Validating access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Hunting Leakage And Validating safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Hunting Leakage And Validating use?

Hunting Leakage And Validating has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.

How many tokens does Hunting Leakage And Validating use?

About 974 tokens (SKILL.md is roughly 3.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Hunting Leakage And Validating?

Skills that share tags, products or a category with Hunting Leakage And Validating: Design System (affaan-m/ECC, 276k stars), Design Guide (paperclipai/paperclip, 100k stars), Design Audit Against Rams' Principles (thedotmack/claude-mem, 99k stars) and Figma Design to Code (warpdotdev/warp, 65k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Hunting Leakage And Validating?

flyrank-bih (a GitHub organization) maintains it in flyrank-bih/flyrank-ml-internship-starter, which has 140 GitHub stars. The repository holds 13 skills in this directory. The repository was last updated on August 20, 2026.

Source: flyrank-bih/flyrank-ml-internship-starter on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.