Agent skill

Robots Meta Conflict

by thedaviddias in thedaviddias/Front-End-Checklist

A skill your agent uses when applies to sites that use both robots.txt disallow rules and meta robots noindex tags.

MITAuto-check passedMarketing & SEO

Install Robots Meta Conflict

skills CLI
$ npx skills add thedaviddias/Front-End-Checklist --skill robots-meta-conflict -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install thedaviddias/Front-End-Checklist robots-meta-conflict --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/thedaviddias/Front-End-Checklist.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/robots-meta-conflict .claude/skills/robots-meta-conflict && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
robots-meta-conflict
GitHub stars
74k
Token cost
~555 tokens
SKILL.md length
265 words
Files
2 (incl. references)
Skills in repo
390
Repo updated
First seen
Licence
MIT

At a glance

A skill your agent uses when applies to sites that use both robots.txt disallow rules and meta robots noindex tags.

  • Applies to sites that use both robots.txt disallow rules and meta robots noindex tags
  • SKILL.md covers Quick Reference, Check, Fix and Explain, plus 1 more section
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
  • Investigating why deindexed pages still appear in search results

What it does

Robots Meta Conflict is an agent skill from thedaviddias/Front-End-Checklist. Use when applies to sites that use both robots.txt disallow rules and meta robots noindex tags. Use when investigating why deindexed pages still appear in search results.

Its SKILL.md is about 560 tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files, including reference files (for example `references/rule.md`).

It sits in Marketing & SEO, covering Technical SEO. The repository describes itself as: 🗂 The essential checklist for modern web development, for humans and AI agents. The licence is MIT.

When your agent uses it

  • Applies to sites that use both robots.txt disallow rules and meta robots noindex tags
  • Investigating why deindexed pages still appear in search results

Example prompts

  • “/robots-meta-conflict”

What it can do on your machine

Read from SKILL.md and the folder at commit e8d14d0. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Links to these hosts (documentation or services it may open):

    • frontendchecklist.io

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Robots Meta Conflict loads about 555 tokens when it runs, and up to ~1.5k if it reads all its reference files. Until then it costs about 48 tokens; SKILL.md has 265 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~48
When it runs · the whole SKILL.md, loaded when a task matches
~555
With references · SKILL.md plus every file in references/, read only if the agent opens them
~1.5k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from thedaviddias/Front-End-Checklist at commit e8d14d0, republished under its MIT licence (© thedaviddias). 265 words, ~555 tokens.

Download SKILL.mdSave it as .claude/skills/robots-meta-conflict/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.
name
robots-meta-conflict
description
Use when applies to sites that use both robots.txt disallow rules and meta robots noindex tags. Use when investigating why deindexed pages still appear in search results.
metadata.category
seo
metadata.priority
high
metadata.difficulty
intermediate
metadata.estimatedTime
10
metadata.source
frontendchecklist.io
metadata.url
https://frontendchecklist.io/rules/seo/robots-meta-conflict

Robots Meta Conflict

Pages that appear in search results despite your intention to deindex them are usually blocked in robots.txt — Google cannot read the noindex directive it cannot fetch.

Quick Reference

  • If a page is blocked in robots.txt, crawlers never fetch it and therefore never read its noindex meta tag
  • To properly deindex a page, use <meta name="robots" content="noindex"> without blocking it in robots.txt
  • If you block a URL in robots.txt, Google may still show it in results using anchor text from backlinks
  • Audit pages that need to be deindexed by confirming they are crawlable but carry noindex

Check

Cross-reference the site's robots.txt disallow paths against pages carrying <meta name="robots" content="noindex">. Flag any page where the URL matches a Disallow rule AND has a noindex tag — the noindex will never be seen by crawlers.

Fix

For pages that must be deindexed: remove the robots.txt Disallow rule for that URL and keep the noindex meta tag. For pages that must simply be uncrawled without appearing: keep the Disallow rule and remove the noindex tag (understanding Google may still show the URL).

Explain

Explain why a page blocked by robots.txt can still appear in Google search results, why the noindex meta tag inside a blocked page is never processed, and how to correctly deindex a page.

Code Review

Review metadata generation, rendered HTML, structured data, and response headers related to Robots Meta Conflict. Flag exact routes or templates where search-facing output violates the rule, and describe how to verify the final page output.


For full implementation details, code examples, and framework-specific guidance, see references/rule.md.

Rule page: https://frontendchecklist.io/rules/seo/robots-meta-conflict

© thedaviddias, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 1 other file (references) in skills/robots-meta-conflict of thedaviddias/Front-End-Checklist.

  • SKILL.md
  • references/rule.md

Open the folder on GitHubat commit e8d14d0

Compare with similar skills

Robots Meta Conflict next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Robots Meta Conflict compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Robots Meta Conflict this skillthedaviddias/Front-End-Checklist74k—~555Automated safety check: PassMIT
Hreflang and International SEOAgriciDaniel/claude-seo19k5 repos~3.4kAutomated safety check: PassMIT
Google SEO APIsAgriciDaniel/claude-seo19k1 repos~4.2kAutomated safety check: PassMIT
GEO-First SEO Audit Toolzubair-trabzada/geo-seo-claude11k—~2.8kAutomated safety check: NotesMIT
SEO Optimizerailabs-393/ai-labs-claude-skills4551 repos~3.2kAutomated safety check: PassMIT
SEO Drift MonitorAgriciDaniel/claude-seo19k1 repos~1.9kAutomated safety check: PassMIT

Similar skills

  • Hreflang and International SEO

    AgriciDaniel/claude-seo

    Audits, validates and generates hreflang tags for multi-language and multi-region sites in HTML, HTTP headers or XML sitemaps, flagging common code and return-tag mistakes.

    19k GitHub starsUsed in 5 repos~3.4k tokens
    Marketing & SEOAuto-check passed
  • Google SEO APIs

    AgriciDaniel/claude-seo

    Pulls real Google data for SEO work: Search Console, PageSpeed Insights, CrUX field data, the Indexing API and GA4 organic traffic, through /seo google commands.

    19k GitHub starsUsed in 1 repo~4.2k tokens
    Marketing & SEOAuto-check passed
  • GEO-First SEO Audit Tool

    zubair-trabzada/geo-seo-claude

    Audits a website for AI search visibility across ChatGPT, Claude, Perplexity and Google AI Overviews while checking traditional SEO, schema and E-E-A-T content quality.

    11k GitHub stars~2.8k tokensUpdated yesterday
    Marketing & SEOAuto-check: notes
  • SEO Optimizer

    ailabs-393/ai-labs-claude-skills

    This skill should be used when analyzing HTML/CSS websites for SEO optimization, fixing SEO issues, generating SEO reports, or implementing SEO best practices.

    455 GitHub starsUsed in 1 repo~3.2k tokens
    Marketing & SEOAuto-check passed
  • SEO Drift Monitor

    AgriciDaniel/claude-seo

    Captures baselines of a page's SEO-critical elements, compares later snapshots against them and flags regressions by severity, like version control for on-page SEO.

    19k GitHub starsUsed in 1 repo~1.9k tokens
    Marketing & SEOAuto-check passed
  • llms.txt Analyzer and Generator

    zubair-trabzada/geo-seo-claude

    Validates an existing llms.txt file or crawls a site to generate a new one, following the format rules for a root-level Markdown file aimed at AI systems.

    11k GitHub starsUsed in 2 repos~3.9k tokens
    Marketing & SEOAuto-check: notes

More from thedaviddias/Front-End-Checklist

All 390 skills in this repo
  • Content Dates Audit

    thedaviddias/Front-End-Checklist

    Audits article and blog pages for visible publish dates, Article JSON-LD with datePublished and dateModified, and Open Graph time tags, then fixes what is missing.

    74k GitHub stars~691 tokensUpdated 4 days ago
    Auto-check passed
  • FAQPage Schema Markup

    thedaviddias/Front-End-Checklist

    Adds, checks and fixes FAQPage JSON-LD on pages with visible question-and-answer sections so it matches what readers see and can qualify for rich results.

    74k GitHub stars~774 tokensUpdated 4 days ago
    Auto-check passed
  • Favicon Audit and Setup

    thedaviddias/Front-End-Checklist

    Checks that a site's favicon is linked, reachable and large enough for Google search results, and sets up ICO, SVG and Apple touch icon files when they are missing.

    74k GitHub stars~731 tokensUpdated 4 days ago
    Auto-check passed
  • Content Freshness Signals

    thedaviddias/Front-End-Checklist

    Audits article pages for freshness signals, covering the Last-Modified header, Article JSON-LD dateModified and a visible last-updated date, and fixes mismatches.

    74k GitHub stars~741 tokensUpdated 4 days ago
    Auto-check passed
  • Geo Meta Tags Audit

    thedaviddias/Front-End-Checklist

    Audits and fixes geo.region, geo.placename and geo.position meta tags on regional pages, noting where they help (Bing) and where they do not (Google).

    74k GitHub stars~763 tokensUpdated 4 days ago
    Auto-check passed
  • Improve a Front-End Checklist Rule

    thedaviddias/Front-End-Checklist

    Scores and rewrites a Front-End Checklist rule MDX file against a scored quality rubric, replacing generic stub prompts with specific, actionable ones.

    74k GitHub stars~1.3k tokensUpdated 4 days ago
    Auto-check passed

Categories

Questions about Robots Meta Conflict

What does Robots Meta Conflict do?

A skill your agent uses when applies to sites that use both robots.txt disallow rules and meta robots noindex tags. Robots Meta Conflict is an agent skill from thedaviddias/Front-End-Checklist.txt disallow rules and meta robots noindex tags.

When should I use Robots Meta Conflict?

Robots Meta Conflict fits situations like: applies to sites that use both robots.txt disallow rules and meta robots noindex tags; investigating why deindexed pages still appear in search results.

How do I install Robots Meta Conflict in Claude Code?

Run `npx skills add thedaviddias/Front-End-Checklist --skill robots-meta-conflict -a claude-code`. Or copy the skill folder (skills/robots-meta-conflict in thedaviddias/Front-End-Checklist) into .claude/skills/robots-meta-conflict in your project. Claude Code loads it when a task matches its description.

How do I install Robots Meta Conflict in Codex?

Run `npx skills add thedaviddias/Front-End-Checklist --skill robots-meta-conflict -a codex`. Or copy the skill folder (skills/robots-meta-conflict in thedaviddias/Front-End-Checklist) into .agents/skills/robots-meta-conflict in your project. Codex loads it when a task matches its description.

Can I use Robots Meta Conflict in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add thedaviddias/Front-End-Checklist --skill robots-meta-conflict -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/robots-meta-conflict, .gemini/skills/robots-meta-conflict, .github/skills/robots-meta-conflict and .opencode/skills/robots-meta-conflict in your project.

What does Robots Meta Conflict need to run?

SKILL.md names no scripts, command-line tools or credentials: Robots Meta Conflict is instructions for the agent only.

Does Robots Meta Conflict access the network?

SKILL.md names 1 domain. As links in the text: frontendchecklist.io. This is read from the text; nothing was executed.

Is Robots Meta Conflict safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Robots Meta Conflict use?

Robots Meta Conflict is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Robots Meta Conflict use?

About 555 tokens (SKILL.md is roughly 2.2k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 915 tokens, read only when the agent opens those files.

What are the alternatives to Robots Meta Conflict?

Skills that share tags, products or a category with Robots Meta Conflict: Hreflang and International SEO (AgriciDaniel/claude-seo, 19k stars), Google SEO APIs (AgriciDaniel/claude-seo, 19k stars), GEO-First SEO Audit Tool (zubair-trabzada/geo-seo-claude, 11k stars) and SEO Optimizer (ailabs-393/ai-labs-claude-skills, 455 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Robots Meta Conflict?

thedaviddias (a GitHub user) maintains it in thedaviddias/Front-End-Checklist, which has 74,421 GitHub stars. The repository holds 390 skills in this directory. The repository was last updated on October 6, 2026.

Source: thedaviddias/Front-End-Checklist on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.