Create Snapshot

5. Verify links and field consistency

• Links: run the HEAD-check loop from /update-dataset §6c ("Link verification") on every URL in the new .dvc (url_main, url_download, license.url, and any URL inside description). A curl non-2xx is a signal, not proof — Cloudflare-fronted hosts return false 404s to curl. Escalate with WebFetch, then the Wayback availability API, but per §6c no automated signal is decisive (a missing Wayback capture is non-evidence, and bot-blocked hosts fail curl and WebFetch while serving browsers fine): a link that fails every automated check gets reported with its evidence trail for the user to confirm in a browser — never mark it broken or swap it for an alternative on automated failures alone. URLs carrying a #fragment also need §6c's anchor pass — HTTP status alone can't validate a fragment.

• Citation year vs date_published year: the year inside citation_full should normally match date_published's year. A deliberate mismatch is fine when the producer labels the release by edition rather than publish date (e.g. a "2025 report" published 2026-03-17) — leave a one-line # comment in the .dvc so the next reviewer doesn't re-flag it (/review-data-pr §5 checks exactly this pair).

• Typos in the .dvc: run /check-metadata-typos on the new file, passing the .dvc path itself (or the extensionless snapshots/<namespace>/<version>/<short_name> stem) to its "current step only" scope, which normalizes both to the .dvc. Confirm it reports a target count of 1 before trusting the result — a scope that matched no file reports no typos. The prose fields written in step 3 — description, title, citation_full, attribution — are user-facing, and nothing downstream spell-checks them: /update-dataset §6c only re-checks them on a later version bump. citation_full is the exception to fixing what it reports: it is verbatim producer text, so only correct a typo there if the producer's own page has it right (see the citation_full note above).

• Outdated practices in the .py, when step 3 wrote one: run /check-outdated-practices on it. Run the skill rather than eyeballing the file; it reads the detector extension as its source of truth and catches helper calls that look current but aren't. /review-data-pr treats a leftover __main__ block in a snapshot as a 🔴 blocker, so this is cheaper to fix here than at review.

  Know what it does and does not cover, and don't read a clean result as more than it is. For snapshot paths the detector's main pattern is the if __name__ == "__main__": guard, plus the metadata-preserving patterns, which only bite when the script parses before storing. Don't assume it covers every convention in step 3 — check the pattern list in vscode_extensions/detect-outdated-practices/src/extension.ts rather than inferring coverage from a clean run, and confirm the @click convention yourself with the grep below (cheap, and correct whether or not the installed build carries a click pattern).

  grep -n "@click" snapshots/<namespace>/<version>/<short_name>.py   # expect no output

  Route the finding by where the pattern came from. The script templates in step 3 are variants of apps/wizard/etl_steps/cookiecutter/snapshot/, which is where the current practices are supposed to live — so a hit on a line that came from the template means the template is stale, and fixing only the generated file leaves every future snapshot carrying it. Fix it upstream in the cookiecutter (and in this skill's copy of it) as well as in the file you just wrote.

• Optional deeper pass — adversarial source verification: /fact-check-dataset goes beyond "the links resolve" and reads the producer's documentation behind them, checking every .dvc claim (description accuracy, counts, date_published, license, citation) against what the docs actually say — its Phase 0 is the slice that applies at snapshot stage (the data cross-checks need a built garden dataset, e.g. via /create-dataset). Don't run it by default — it fetches docs and runs web searches, so it can consume many tokens; offer it when the source looks unreliable (no version labels, self-published, or the page and file seem to disagree).

6. Report to the user

Show:

• The paths of the files created (.dvc, plus the .py if one was needed)
• Whether the snapshot ran successfully
• The results of the step 5 checks — links, typos, and outdated practices — including "clean" results. Silence on a check reads as "not a problem" when it may mean "not run".
• The source's other data files, if any — when the landing page / repository ships several data files (companion indices, summary panels, codebooks with data), list the ones NOT snapshotted so the user can opt in now or skip deliberately. Persist the inventory in the .dvc, not just in chat — a # NOTE: comment above url_download listing the release's other data files as of date_accessed (e.g. # NOTE: the release also ships Foo_Index.csv and codebook.pdf — not snapshotted (2026-07-20)). That comment is the baseline the next /update-dataset cycle diffs against when the host has no file-history API; a new companion file is invisible to every within-file check (see /update-dataset, "Surface new indicators").
• Next steps: "You can now create a meadow step for <namespace>/<version>/<short_name>"

Notes

• date_accessed in the DVC file should always equal the snapshot version date (the date you ran the snapshot).
• If url_download is not provided and cannot be inferred, always set dataset_manual_import = true.
• Data living in an embedded chart (Datawrapper and similar): parse the page's own fallback tables, never the chart platform's CDN endpoint. The chart CDN's latest published version can trail the page's data by a full release. Producer pages server-render each embed's data as an HTML <noscript> table — parse that instead: whole-page pd.read_html(io.StringIO(resp.text)), select the table whose columns exactly match the expected header, assert exactly one match. See /update-dataset Guardrails, "Scraped chart embeds".
• Before accepting manual import because "the site blocks downloads": test a plain UA. Some hosts reject browser-like User-Agents while letting plain, honestly-identified clients through (the inverse of the usual bot-blocking). Test both directions against the direct file URL; if a plain UA works, keep url_download in the .dvc and pass user_agent="owid-etl/1.0 (https://ourworldindata.org)" to snap.create_snapshot(...) — and keep the .py in that case (the script-less path can't set a UA), saying so in the docstring. Also re-check the producer's download page/API for a stable direct endpoint before carrying a manual flow forward. See /update-dataset Guardrails.
• The outs block md5 and size fields are filled in automatically by DVC when the snapshot runs — just set them to empty/zero in the template.
• Omit optional YAML fields entirely (don't leave them blank) to keep the DVC file clean.
• Never guess at citation text — if you can't find it on the page, leave a placeholder like <TO BE FILLED> and ask the user.
• Licenses and citations are often not on the landing page: check the documentation too (sources & methods PDFs, notes sheets inside the workbook, repository cover sheets), and look for dedicated pages elsewhere on the same site (licensing, terms of use, how-to-cite sections). A citation request ("please cite as...") is not a license — you may summarize it as the license name only when the page explicitly frames usage terms (e.g. "when using these data (for whatever purpose), please make the following reference"). If nothing is stated anywhere, warn the user and use © <producer> (<year>).
• citation_full should be the producer's recommended citation verbatim whenever one exists (page "cite as" blocks, repository "Original citation", NBER suggested citations, "Reference:" lines inside the data files); slight modifications are fine to fix typos or spacing issues in the source (e.g. "mirror : a" → "mirror: a"), but never rephrase or reformat the citation style. If the recommended citation is for a working-paper version of a published work, keep it and append the published version (e.g. "Published as: …"). Construct a standard-format citation only when the producer recommends none.
• description describes the data product factually. When the producer's own text is factual (abstracts, methodology summaries), prefer it, exactly or lightly rephrased — don't rewrite for the sake of rewriting. When the producer's page only offers promotional or first-person copy ("our nation", "vital information", how the data guides funding), do NOT paste it — write a neutral description from OWID's point of view instead: who produces it, what it measures, coverage, cadence. (Schema guideline: "If the producer provides a good description, use that, either exactly or conveniently rephrased.")
• When the snapshot is a single file/table of a broader data product, split the metadata: title/description describe the data product, while title_snapshot/description_snapshot describe the specific file. Whenever title_snapshot is set, also write a description_snapshot — that one doesn't need to be verbatim. OWID-side context (hand-transcription notes, "retrieved from the Internet Archive", etc.) belongs in description_snapshot, never inside the producer's description.