Crawl4AI Web Scraping
smallnest/goclaw
Scrapes sites, handles JavaScript-heavy pages and extracts structured data with Crawl4AI, through its crwl CLI or Python SDK, including schema-based extraction without an LLM.
Fix ghost variable warnings by remapping charts from old to new variable IDs.
Install Remap Ghost Variables
$ npx skills add owid/etl --skill remap-ghost-variables -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install owid/etl remap-ghost-variables --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .claude/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .claude/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variablesType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add owid/etl --skill remap-ghost-variables -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install owid/etl remap-ghost-variables --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .agents/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .agents/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add owid/etl --skill remap-ghost-variables -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install owid/etl remap-ghost-variables --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .cursor/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .cursor/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/owid/etl.git --path .claude/skills/remap-ghost-variables--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add owid/etl --skill remap-ghost-variables -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install owid/etl remap-ghost-variables --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .gemini/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .gemini/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install owid/etl remap-ghost-variablesInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add owid/etl --skill remap-ghost-variables -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .github/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .github/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .github/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add owid/etl --skill remap-ghost-variables -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install owid/etl remap-ghost-variables --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/owid/etl.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.claude/skills/remap-ghost-variables .opencode/skills/remap-ghost-variables && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "remap-ghost-variables" agent skill from https://github.com/owid/etl/tree/master/.claude/skills/remap-ghost-variables into .opencode/skills/remap-ghost-variables/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "remap-ghost-variables", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
remap-ghost-variables- GitHub stars
- 158
- Token cost
- ~1.2k tokens
- SKILL.md length
- 376 words
- Files
- 1
- Skills in repo
- 35
- Repo updated
- First seen
- Licence
- MIT
At a glance
Fix ghost variable warnings by remapping charts from old to new variable IDs.
- Works in 6 steps: Create a staging PR → Identify the affected chart and old… → Find what the chart currently uses → …
- ETL grapher step shows Variables used in charts will not be deleted automatically
- SKILL.md covers When This Happens, Workflow and Guidelines
- Calls make and python
What it does
Remap Ghost Variables is an agent skill from owid/etl. Fix ghost variable warnings by remapping charts from old to new variable IDs. Use when ETL grapher step shows "Variables used in charts will not be deleted automatically", when indicator shortnames were renamed and charts still reference old variables, or when the user mentions ghost variables, orphaned indicators, or chart variable remapping.
Its SKILL.md is about 1.2k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in Data & Analytics, covering Data pipelines and ETL. The repository describes itself as: A compute graph for loading and transforming OWID's data. The licence is MIT.
When your agent uses it
- ETL grapher step shows Variables used in charts will not be deleted automatically
- Indicator shortnames were renamed and charts still reference old variables
- The user mentions ghost variables
- Orphaned indicators
Example prompts
- “Variables used in charts will not be deleted automatically”
- “/remap-ghost-variables”
Requirements
- Python 3
Workflow steps
6 steps, taken from the step headings in SKILL.md.
What it can do on your machine
Read from SKILL.md and the folder at commit bf5dc8e. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
makepython
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Remap Ghost Variables loads about 1.2k tokens when it runs. Until then it costs about 92 tokens; SKILL.md has 376 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from owid/etl at commit bf5dc8e, republished under its MIT licence (© owid). 376 words, ~1,242 tokens.
Download SKILL.mdSave it as
.claude/skills/remap-ghost-variables/SKILL.md (or your agent's skills folder).- name
- remap-ghost-variables
- description
- Fix ghost variable warnings by remapping charts from old to new variable IDs. Use when ETL grapher step shows "Variables used in charts will not be deleted automatically", when indicator short_names were renamed and charts still reference old variables, or when the user mentions ghost variables, orphaned indicators, or chart variable remapping.
- metadata.internal
- true
- metadata.owner
- Marigold
Remapping Ghost Variables
Fix charts that reference old (ghost) variable IDs after indicator short_names were renamed in a garden/grapher step.
When This Happens
After running a grapher step, you see a warning like:
[warning] Variables used in charts will not be deleted automatically.
rows= chartId variableId
0 1234 56789
variables=[56780, 56781, ..., 56789]This means someone renamed indicator shortNames in the ETL step, creating new variables while charts still reference the old ones. The ETL can't delete the old variables because charts depend on them.
Workflow
Step 1: Create a staging PR
Create a PR first — all investigation and changes happen against the staging DB.
.venv/bin/etl pr "Remap chart <chart_id> ghost variables for <dataset>" dataWait for the staging server to come up.
Step 2: Identify the affected chart and old variables
From the warning, extract:
- Chart IDs from the
rowstable (e.g., chart 1234) - Ghost variable IDs from the
variableslist
All queries below use make query which automatically targets the staging DB for the current branch.
Query the old variables to understand what was renamed:
make query SQL="SELECT id, name, shortName FROM variables WHERE id IN (<ghost_variable_ids>)"Step 3: Find what the chart currently uses
make query SQL="SELECT variableId FROM chart_dimensions WHERE chartId = <chart_id>"Then get details on each:
make query SQL="SELECT id, name, shortName FROM variables WHERE id IN (<chart_variable_ids>)"Step 4: Find the new replacement variables
The new variables are in the same dataset but with different shortNames. Get the dataset ID first:
make query SQL="SELECT datasetId FROM variables WHERE id = <any_ghost_variable_id>"Then search for the new variables. Look for the pattern difference between old and new shortNames:
make query SQL="SELECT id, shortName FROM variables WHERE datasetId = <dataset_id> AND shortName LIKE '<pattern_matching_new_name>' AND id NOT IN (<ghost_variable_ids>)"Compare old vs new shortNames to confirm the mapping. Common rename patterns:
- Encoding changes (e.g.,
gte_40→gt__40) - Prefix/suffix changes
- Restructured naming conventions
Show full SKILL.md (154 more words)Show less
Step 5: Remap via the indicator upgrader (preferred)
Insert the mappings, then run the upgrader — it updates dimensions, map configs, sortBy, and narrative charts, and records the mapping in wiz__variable_mapping so chart diff picks it up:
STAGING=<branch> .venv/bin/python -c "
from apps.wizard.utils.db import WizardDB
WizardDB.add_variable_mapping(
mapping={<old_id>: <new_id>},
dataset_id_old=<old_ds_id>, dataset_id_new=<new_ds_id>,
comments='<why>')"
STAGING=<branch> .venv/bin/python -m apps.indicator_upgrade.cli upgrade --dry-run
STAGING=<branch> .venv/bin/python -m apps.indicator_upgrade.cli upgradeCheck WizardDB.get_variable_mapping_raw() first — upgrade applies all stored mappings, not just yours.
Step 5 (fallback): Remap the chart via Admin API
If the indicator upgrader can't be used, update the chart config manually on staging with AdminAPI:
from apps.chart_sync.admin_api import AdminAPI
from etl.config import OWIDEnv
env = OWIDEnv.from_staging('<branch_name>')
api = AdminAPI(env)
# Get current chart config
config = api.get_chart_config(<chart_id>)
# Remap old variable IDs to new ones
REMAP = {
<old_var_id>: <new_var_id>,
# ... add more mappings if multiple variables need remapping
}
for dim in config.get('dimensions', []):
if dim['variableId'] in REMAP:
old = dim['variableId']
dim['variableId'] = REMAP[old]
print(f'Remapped {old} -> {REMAP[old]}')
# Update the chart
result = api.update_chart(<chart_id>, config)
assert result['success'], f"Update failed: {result}"
print('Chart updated successfully')Guidelines
- Always use staging — never update charts directly on production
- Verify the data is identical — the new variable should contain the same data as the old one, just with a different shortName
- Handle multiple charts — the warning may list multiple chartId rows; remap all of them
- Handle multiple variables per chart — a chart may reference several ghost variables; remap all in one update
- Check for narrative charts — ghost variables may also be referenced by narrative charts (check
chart_dimensionsisn't the only reference)
© owid, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
Just SKILL.md in .claude/skills/remap-ghost-variables of owid/etl.
Open the folder on GitHubat commit bf5dc8e
Compare with similar skills
Remap Ghost Variables next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Remap Ghost Variables this skillowid/etl | 158 | — | ~1.2k | Automated safety check: Pass | MIT | |
| Crawl4AI Web Scrapingsmallnest/goclaw | 598 | 1 repos | ~2.5k | Automated safety check: Pass | MIT | |
| Glue 09 10 Migrationaws-samples/aws-glue-samples | 1.5k | — | ~2.4k | Automated safety check: Pass | MIT-0 | |
| Migrate Glue Devendpoint To Interactive Sessionsaws-samples/aws-glue-samples | 1.5k | — | ~3.6k | Automated safety check: Pass | MIT-0 | |
| Dbt Databricks PR Readydatabricks/dbt-databricks | 380 | — | ~2.8k | Automated safety check: Pass | Apache-2.0 | |
| Mz Dbt ReleaseMaterializeInc/materialize | 6.4k | — | ~1.2k | Automated safety check: Pass | Custom licence |
Similar skills
- Official
Glue 09 10 Migration
aws-samples/aws-glue-samples
Upgrade an AWS Glue ETL job from Glue version 0.9 or 1.0 to Glue 4.0.
- Official
Migrate Glue Devendpoint To Interactive Sessions
aws-samples/aws-glue-samples
Migrate a legacy AWS Glue development endpoint to a Glue interactive session, following the official AWS migration checklist.
- Official
Dbt Databricks PR Ready
databricks/dbt-databricks
A skill your agent uses for an open dbt-databricks pull request, including your own PR or a fork PR, to assess merge readiness and optionally repair selected gaps on the PR head branch.
Mz Dbt Release
MaterializeInc/materialize
Cut a dbt-materialize PyPI release: bump the version in version.py and setup.py, date the Unreleased CHANGELOG entry, and open the release PR with a Ship: <url body.
Erd Studio Setup
liam-machine/erd-studio
Friendly, step-by-step setup for ERD Studio in an existing dbt project, for people who may be new to dbt or data modelling.
More from owid/etl
All 35 skills in this repoFind Chart References
owid/etl
Find every OWID surface that references a chart, indicator, MDIM, or explorer — articles (links vs embeds), explorers, narrative charts, data insights, static viz, key-chart slots, MDIM views.
Add Gdp Scatter
owid/etl
Add a scatter view (with GDP per capita on x) to existing OWID charts via the admin API, mirroring the admin UI's "Add scatter type" defaults, then retire the old standalone "X vs.
Add Ivs Indicators
owid/etl
Add new survey question codes (e.g. An agent skill from owid/etl.
Create Static Viz
owid/etl
Build or refresh an OWID static visualization end to end — resolve what data it needs from an old static viz image, an indicator, or a grapher chart; check both the ETL catalog and the producer's…
Map Charts To Mdim
owid/etl
Propose redirects from (soon-to-sunset) grapher charts to the matching views of published MDIMs.
Map Explorer To Mdim
owid/etl
Take (soon-to-sunset) OWID explorers to redirected MDIMs, end to end.
Related
Categories
Questions about Remap Ghost Variables
What does Remap Ghost Variables do?
Fix ghost variable warnings by remapping charts from old to new variable IDs. Remap Ghost Variables is an agent skill from owid/etl. Fix ghost variable warnings by remapping charts from old to new variable IDs.
When should I use Remap Ghost Variables?
Remap Ghost Variables fits situations like: ETL grapher step shows Variables used in charts will not be deleted automatically; indicator shortnames were renamed and charts still reference old variables; the user mentions ghost variables; orphaned indicators.
How do I install Remap Ghost Variables in Claude Code?
Run `npx skills add owid/etl --skill remap-ghost-variables -a claude-code`. Or copy the skill folder (.claude/skills/remap-ghost-variables in owid/etl) into .claude/skills/remap-ghost-variables in your project. Claude Code loads it when a task matches its description.
How do I install Remap Ghost Variables in Codex?
Run `npx skills add owid/etl --skill remap-ghost-variables -a codex`. Or copy the skill folder (.claude/skills/remap-ghost-variables in owid/etl) into .agents/skills/remap-ghost-variables in your project. Codex loads it when a task matches its description.
Can I use Remap Ghost Variables in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add owid/etl --skill remap-ghost-variables -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/remap-ghost-variables, .gemini/skills/remap-ghost-variables, .github/skills/remap-ghost-variables and .opencode/skills/remap-ghost-variables in your project.
What does Remap Ghost Variables need to run?
Going by SKILL.md and its folder, Remap Ghost Variables needs the command-line tools its instructions call (make and python). Our summary lists: Python 3.
Does Remap Ghost Variables access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Remap Ghost Variables safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Remap Ghost Variables use?
Remap Ghost Variables is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Remap Ghost Variables use?
About 1.2k tokens (SKILL.md is roughly 5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
What are the alternatives to Remap Ghost Variables?
Skills that share tags, products or a category with Remap Ghost Variables: Crawl4AI Web Scraping (smallnest/goclaw, 598 stars), Glue 09 10 Migration (aws-samples/aws-glue-samples, 1.5k stars), Migrate Glue Devendpoint To Interactive Sessions (aws-samples/aws-glue-samples, 1.5k stars) and Dbt Databricks PR Ready (databricks/dbt-databricks, 380 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Remap Ghost Variables?
owid (a GitHub organization) maintains it in owid/etl, which has 158 GitHub stars. The repository holds 35 skills in this directory. The repository was last updated on October 8, 2026.
Source: owid/etl on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.