Walk Forward Validation
agiprolabs/claude-trading-skills
Walk-forward validation framework for trading strategies and ML models with time-series-aware splits, overfit detection, and regime-aware validation
A skill your agent uses when projecting history forward — sales, demand, units, revenue, signups, traffic — into a defensible number with an error band: method by data shape, rolling-origin…
$ npx skills add ericrisco/rsc-harness --skill forecasting -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install ericrisco/rsc-harness forecasting --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/forecasting .claude/skills/forecasting && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .claude/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/ericrisco/rsc-harness/tree/main/skills/forecastingType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add ericrisco/rsc-harness --skill forecasting -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install ericrisco/rsc-harness forecasting --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/forecasting .agents/skills/forecasting && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .agents/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add ericrisco/rsc-harness --skill forecasting -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install ericrisco/rsc-harness forecasting --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/forecasting .cursor/skills/forecasting && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .cursor/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/ericrisco/rsc-harness.git --path skills/forecasting--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add ericrisco/rsc-harness --skill forecasting -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install ericrisco/rsc-harness forecasting --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/forecasting .gemini/skills/forecasting && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .gemini/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install ericrisco/rsc-harness forecastingInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add ericrisco/rsc-harness --skill forecasting -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/forecasting .github/skills/forecasting && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .github/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add ericrisco/rsc-harness --skill forecasting -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install ericrisco/rsc-harness forecasting --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/ericrisco/rsc-harness.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/forecasting .opencode/skills/forecasting && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "forecasting" agent skill from https://github.com/ericrisco/rsc-harness/tree/main/skills/forecasting into .opencode/skills/forecasting/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "forecasting", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
forecastingA skill your agent uses when projecting history forward — sales, demand, units, revenue, signups, traffic — into a defensible number with an error band: method by data shape, rolling-origin…
Forecasting is an agent skill from ericrisco/rsc-harness. Use when projecting history forward — sales, demand, units, revenue, signups, traffic — into a defensible number with an error band: method by data shape, rolling-origin backtest, MASE vs the naive baseline. NOT an assumption-driven P&L or runway model (that is financial-model), NOT sizing reorder points or safety stock (that is inventory).
Its SKILL.md is about 2.8k tokens, which your agent loads only when the skill is triggered. The skill folder holds 8 other files, including scripts and reference files (for example `evals/README.md`, `evals/cases.yaml` and `references/accuracy-and-backtesting.md`).
It sits in Data & Analytics, covering Forecasting and time series, Financial modeling and Trading and backtesting. The repository describes itself as: Your agent invents things because it has no memory, and can't touch your database because it has no arms. rsc is the meta-harness that gives it both, plus the trade to know the… The licence is MIT.
3 steps, taken from the first numbered list in SKILL.md.
Read from SKILL.md and the folder at commit e3d5b33. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Ships 1 file in scripts/ (Shell), which the agent can run.
From the folder's file list and the shell code blocks in SKILL.md.
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Forecasting loads about 2.8k tokens when it runs, and up to ~4.6k if it reads all its reference files. Until then it costs about 90 tokens; SKILL.md has 1,228 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
The full file from ericrisco/rsc-harness at commit e3d5b33, republished under its MIT licence (© ericrisco). 1,228 words, ~2,758 tokens.
.claude/skills/forecasting/SKILL.md (or your agent's skills folder). This skill also uses 5 other files; get the full folder from GitHub.A forecast that cannot beat "repeat last period" is noise. Baseline first, fancy second. The naive forecast is free, instant, and the bar every model must clear — if your AutoARIMA loses to last-quarter-repeated, ship the repeat and say so.
You are not done when a model produces a number. You are done when you can defend the number: which method, why that method for this data, how it scored against the naive baseline in a backtest, and the interval around the point. A point estimate with no error band is a guess wearing a lab coat.
Every forecast you ship is a reproducible artifact, not a number pasted in chat:
ds, forecast, lo, hi — timestamp, point, interval bounds.If you cannot produce all three, you have not forecast — you have guessed. scripts/verify.sh checks the artifact has these columns, the right row count, and an accuracy line.
Run these in order. Skipping step 3 is the most common failure.
h (how many periods forward), the granularity (daily / weekly / monthly), and exactly what is being predicted (units? revenue? per-SKU or aggregate?). Forecast at the level you will act on — if you reorder per SKU, forecast per SKU, then sanity-check against the aggregate.data-cleaning before modeling. Garbage history, garbage forecast.inventory, financial-model).Match the method to the shape of the history, not to what sounds sophisticated. statsforecast (Nixtla, v2.0.3) provides all of these with built-in intervals.
| Data shape | Method | statsforecast call | Why |
|---|---|---|---|
| Flat, no trend or season | Moving average or SES | AutoCES() / 3-period MA | Nothing to model; a mean is honest. |
| Trend, with or without season | ETS | AutoETS(season_length=m) | ETS captures level+trend+season cleanly, no manual order. |
| Strong known seasonality / autocorrelation | ARIMA | AutoARIMA(season_length=m) | Handles autocorrelated errors; ~20x faster than pmdarima. |
| Many zeros (intermittent / lumpy demand) | Croston / SBA | CrostonOptimized() | SES is provably wrong on sporadic demand (Croston 1972); SBA debiases it. |
| < 2 full seasonal cycles of history | SeasonalNaive only | SeasonalNaive(season_length=m) | Too little data to fit a model. Do not fit one. Full stop. |
When in doubt between two, fit both plus the baseline in one StatsForecast run and let the backtest decide. Theta (AutoTheta) is a strong, cheap default that often wins on monthly business series.
The metrics are not decoration — they decide what you ship.
cross_validation(h=…, n_windows=…).level=[80] or [95]. The interval is half the deliverable, not optional polish.Formulas (WAPE, MASE, bias, pinball, coverage), rolling-origin mechanics, and how to read a backtest table are in references/accuracy-and-backtesting.md. Per-method when-to-use and the exact statsforecast one-liner for each are in references/methods-cheatsheet.md.
The full pipeline: long-format dataframe, fit competing methods + baseline, backtest, forecast with an interval, write the artifact.
# pip install statsforecast (Nixtla, v2.0.3)
import pandas as pd
from statsforecast import StatsForecast
from statsforecast.models import SeasonalNaive, AutoETS, AutoARIMA
# long format: unique_id, ds, y (one row per series per period)
df = pd.read_csv("history.csv", parse_dates=["ds"])
m, h = 12, 12 # monthly seasonality; forecast 12 periods ahead
sf = StatsForecast(
models=[SeasonalNaive(season_length=m), AutoETS(season_length=m), AutoARIMA(season_length=m)],
freq="MS",
)
# rolling-origin backtest BEFORE trusting any forecast
cv = sf.cross_validation(df=df, h=h, n_windows=3, step_size=h)
def wape(a, f): return (a - f).abs().sum() / a.abs().sum()
for col in ["SeasonalNaive", "AutoETS", "AutoARIMA"]:
print(col, "WAPE", round(wape(cv["y"], cv[col]), 4)) # pick the lowest that beats SeasonalNaive
# refit on full history, forecast with an 80% interval
fc = sf.forecast(df=df, h=h, level=[80])
# choose the winning model column from the backtest; here AutoETS as example
out = fc.rename(columns={"AutoETS": "forecast", "AutoETS-lo-80": "lo", "AutoETS-hi-80": "hi"})
out[["ds", "forecast", "lo", "hi"]].to_csv("forecast.csv", index=False)Zero-dependency fallback when you cannot install statsforecast — a seasonal-naive baseline in pure pandas. This is also the thing every model must beat, so it is always worth computing:
import pandas as pd
def seasonal_naive(y: pd.Series, m: int, h: int) -> pd.Series:
"""Repeat the last full season forward h periods."""
last_season = y.iloc[-m:].to_numpy()
return pd.Series([last_season[i % m] for i in range(h)])
s = pd.read_csv("history.csv", parse_dates=["ds"]).set_index("ds")["y"]
fc = seasonal_naive(s, m=12, h=12)
# crude interval from historical residual spread; honest is better than absent
resid_std = (s - s.shift(12)).dropna().std()
out = pd.DataFrame({"forecast": fc, "lo": fc - 1.28 * resid_std, "hi": fc + 1.28 * resid_std})
out.to_csv("forecast.csv", index=False)references/methods-cheatsheet.md.| Anti-pattern | Why it bites | Do instead |
|---|---|---|
| Report a single point number | A point hides uncertainty the reader needs to plan around. | Report point + 80/95% interval |
| Tune ARIMA orders before any baseline | If you cannot beat the free baseline, the tuning was wasted. | Compute naive/seasonal-naive first |
| Score with MAPE on intermittent demand | MAPE explodes near zero actuals and lies about accuracy. | Use WAPE + bias |
| Single train/test holdout | One split is one sample; CV estimates real out-of-sample error. | Rolling-origin CV (n_windows≥3) |
| Fit a model on 8 months of monthly data | Too few points; the model overfits and underreports its own error. | SeasonalNaive only under 2 cycles |
| "The model picked it, so it's right" | A forecast you cannot defend is worse than no forecast. | State method + MASE vs naive |
| Trust SKU forecasts without checking the sum | Per-SKU errors compound; the total exposes nonsense fast. | Sanity-check vs the aggregate |
../inventory/SKILL.md — feed it the demand number; it sizes reorder points and safety stock. Forecasting produces the demand; it does not size the stock.../financial-model/SKILL.md — when the projection is driven by assumptions and drivers (pricing, hiring), not history, that is a model, not a forecast.../unit-economics/SKILL.md — contribution margin, CAC/LTV, payback. No time series, route there.../data-cleaning/SKILL.md — dirty input (dupes, missing rows, mixed units) goes here before you model.../analyze/SKILL.md — when the question is "why" or a backward-looking metric/aggregation rather than forward extrapolation.© ericrisco, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 5 other files (scripts, references) in skills/forecasting of ericrisco/rsc-harness.
Open the folder on GitHubat commit e3d5b33
Forecasting next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Forecasting this skillericrisco/rsc-harness | 167 | — | ~2.8k | Automated safety check: Pass | MIT | |
| Walk Forward Validationagiprolabs/claude-trading-skills | 410 | — | ~2.2k | Automated safety check: Pass | MIT | |
| Options Spread Conviction EngineLeoYeAI/openclaw-master-skills | 2.2k | — | ~5.7k | Automated safety check: Notes | MIT | |
| Actuarial Risk Modelingmagnus919/agent-skills | 113 | — | ~3.2k | Automated safety check: Pass | MIT | |
| Quant Statistical MethodsHKUDS/Vibe-Trading | 35k | — | ~4k | Automated safety check: Pass | MIT | |
| Longbridge Quanthelsome/folio | 269 | 1 repos | ~1.6k | Automated safety check: Pass | MIT |
agiprolabs/claude-trading-skills
Walk-forward validation framework for trading strategies and ML models with time-series-aware splits, overfit detection, and regime-aware validation
LeoYeAI/openclaw-master-skills
Multi-regime options spread analysis engine with quantitative rigor.
magnus919/agent-skills
A skill your agent uses when analyzing, selecting, validating, or communicating models for insurance, actuarial, financial-risk, or other consequential uncertain outcomes.
HKUDS/Vibe-Trading
Guides your agent through unit-root, cointegration, GARCH, bootstrap and regression-diagnostic tests on financial time series, using a tested helper module.
helsome/folio
Quantitative strategy frameworks: pairs trading/cointegration, volatility regime strategies, seasonality/calendar effects, multi-factor models (IC/IR), factor research and screening, correlation…
EveryInc/charlie-cfo-skill
Your AI CFO for bootstrapped startups, named after Charlie Munger who embodied the principle that capital discipline is a competitive advantage.
ericrisco/rsc-harness
A skill your agent uses when designing or analyzing a controlled experiment — falsifiable hypothesis, sample size from an MDE, reading significance/CI/power, CUPED, or rescuing tests that won't go…
ericrisco/rsc-harness
A skill your agent uses when making a web UI conform to WCAG 2.2 Level AA — axe-core or Lighthouse a11y violations, keyboard operability, focus management, ARIA roles/names/live regions, contrast…
ericrisco/rsc-harness
A skill your agent uses when running or fixing paid acquisition on Google or Meta — campaign structure (Performance Max, Demand Gen, Search, Advantage+), platform-fit creative, budget/scaling rules…
ericrisco/rsc-harness
A skill your agent uses when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual…
ericrisco/rsc-harness
A skill your agent uses when a creative goal must become a finished media file: pick and order generative-media models per modality — AI voiceover, image-to-video clips, score — then glue them with…
ericrisco/rsc-harness
A skill your agent uses when instrumenting product or web analytics — GA4/PostHog SDK wiring, event taxonomy, funnels, double-counted events, consent gating, PII scrubbing.
Categories
A skill your agent uses when projecting history forward — sales, demand, units, revenue, signups, traffic — into a defensible number with an error band: method by data shape, rolling-origin…. Forecasting is an agent skill from ericrisco/rsc-harness. Use when projecting history forward — sales, demand, units, revenue, signups, traffic — into a defensible number with an error band: method by data shape, rolling-origin backtest, MASE vs the naive baseline.
Forecasting fits situations like: projecting history forward — sales; traffic — into a defensible number with an error band: method by data shape; rolling-origin backtest; MASE vs the naive baseline.
Run `npx skills add ericrisco/rsc-harness --skill forecasting -a claude-code`. Or copy the skill folder (skills/forecasting in ericrisco/rsc-harness) into .claude/skills/forecasting in your project. Claude Code loads it when a task matches its description.
Run `npx skills add ericrisco/rsc-harness --skill forecasting -a codex`. Or copy the skill folder (skills/forecasting in ericrisco/rsc-harness) into .agents/skills/forecasting in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add ericrisco/rsc-harness --skill forecasting -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/forecasting, .gemini/skills/forecasting, .github/skills/forecasting and .opencode/skills/forecasting in your project.
Going by SKILL.md and its folder, Forecasting needs a shell for the scripts in its folder. Our summary lists: Python 3; A Bash shell.
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
Forecasting is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 2.8k tokens (SKILL.md is roughly 11k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.8k tokens, read only when the agent opens those files.
Skills that share tags, products or a category with Forecasting: Walk Forward Validation (agiprolabs/claude-trading-skills, 410 stars), Options Spread Conviction Engine (LeoYeAI/openclaw-master-skills, 2.2k stars), Actuarial Risk Modeling (magnus919/agent-skills, 113 stars) and Quant Statistical Methods (HKUDS/Vibe-Trading, 35k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
ericrisco (a GitHub user) maintains it in ericrisco/rsc-harness, which has 167 GitHub stars. The repository holds 227 skills in this directory. The repository was last updated on October 7, 2026.
Source: ericrisco/rsc-harness on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.