Agentic Kaggle Workflow

Agentic Kaggle Skill

Operating Loop

Treat every competition as a validation problem first and a modeling problem second. The default target platform is Kaggle, so prefer Kaggle-native notebooks/scripts, datasets, model artifacts, competition submissions, and score receipts. For code competitions, assume the final notebook/kernel will be rerun by Kaggle against hidden data unless the competition docs prove otherwise.

1. Read the competition page, classify the submission mode as classic file submission or code/notebook scoring, then inspect the rules, data-use terms, sharing policy, data dictionary, metric, submission format, train/test construction hints, and leakage warnings.
2. If live competition intelligence tools are available, inspect top public open notebook solutions and relevant discussion activity before major architecture choices; treat them as clues, not authority.
3. Identify the task type: binary, multiclass, multilabel, regression, ranking, image, segmentation, text, time series, grouped entities, or a hybrid.
4. Design folds before feature engineering or modeling. Prefer a fold column saved into the training data so every experiment uses the same comparison surface.
5. Build the simplest metric-correct baseline and produce out-of-fold (OOF) predictions plus a valid submission.
6. Proactively plan a stronger architecture once the baseline is trustworthy: diverse model families, feature/embedding producers, augmentation/pseudo-label/distillation stages, calibration/postprocessing, and an ensemble or stacker.
7. Iterate with validation gates: test one meaningful change at a time when possible, but launch several independent producer notebooks in parallel when they create diverse artifacts that can be compared by OOF score or ensemble diversity.
8. Offload heavy training, inference, embedding generation, image/text experiments, or memory-risky jobs to Kaggle notebooks/scripts when local compute may OOM or take too long.
9. For sophisticated architectures, split work into a small Kaggle pipeline: run several independent producer notebooks/scripts first, save each useful producer output as a private Kaggle dataset, then run one consumer notebook/script that attaches those datasets and creates the final OOF/test/submission outputs. When a producer trains a model, its checkpoint, tokenizer/config, fold metadata, OOF/test predictions, and manifest should be exported as a Kaggle dataset; downstream notebooks load the model from /kaggle/input/....
10. Submit the final submission-producing artifact to Kaggle for scoring and retrieve the resulting submission status/score.
11. If Kaggle returns a code-competition error or vague scoring failure, enter the debugging loop: retrieve available logs, classify likely failure mode, patch defensively, rerun the final kernel, resubmit, and repeat until scored or concretely blocked.
12. Track local CV, remote Kaggle run status, Kaggle submission score, public LB, private-risk notes, seed, code version, data version, and artifact paths for every run.
13. Ensemble only with OOF predictions generated without in-fold leakage.

Resource Map

• Read references/method-map.md for the neutral workflow map behind the skill.
• Read references/information-sharing-policy.md before publishing competition code, notebooks, datasets, models, artifacts, or reports outside the active team or workspace.
• Read references/competition-intel.md when a Kaggle competition slug, URL, title, public leaderboard context, open solutions, or discussion activity may inform the approach.
• Read references/cross-validation-and-metrics.md when choosing folds, metrics, thresholds, or leakage checks.
• Read references/tabular-workflow.md for categorical variables, feature engineering, selection, and hyperparameter tuning.
• Read references/image-text-workflow.md for image, segmentation, and NLP competition approaches.
• Read references/kaggle-code-competition-pipeline.md when the target is a Kaggle code competition, hidden rerun, final notebook scoring flow, or model artifact handoff between producer and consumer notebooks.
• Read references/advanced-notebook-architecture.md when a stronger solution may need multiple model families, staged feature/embedding/model producers, pseudo-labeling, distillation, postprocessing, blending, stacking, or parallel Kaggle GPU notebooks.
• Read references/kaggle-offload.md when local runs are heavy, GPU is useful, Kaggle data access is needed, or remote notebook results must be collected.
• Read references/kaggle-pipeline-datasets.md when using multiple Kaggle notebooks, intermediate datasets, notebook output sources, or kagglehub.
• Read references/submission-endgame.md before stopping work; this skill is not done until Kaggle scoring has been attempted and the result has been collected or a concrete blocker is documented.
• Read references/code-competition-debugging.md when a Kaggle code competition submission fails, times out, OOMs, produces no score, or reports a vague hidden-run/scoring error.
• Read references/ensembling-and-reproducibility.md for project layout, OOF artifacts, stacking, blending, and repeatability.
• Read references/research/ or examples/ only when the user asks for historical case studies, Hermes-era patterns, or concrete competition lessons from this repository.
• Run scripts/scaffold_competition.py to create a competition workspace.
• Run scripts/make_folds.py to add a fold column to a training CSV.
• Run scripts/prepare_kaggle_kernel.py to create a Kaggle kernel folder with metadata and retrievable experiment logs.
• Run scripts/prepare_kaggle_dataset.py to create metadata and commands for versioned intermediate artifact datasets.

Default Kaggle Procedure

Start with the metric and validation:

• Reproduce the competition metric locally, including clipping, transforms, averaging mode, thresholds, and sample weights.
• Check whether the competition allows public code, external data, pretrained weights, generated artifacts, and public repositories before sharing anything outside the team.
• Classify the Kaggle submission path early. If it is a code competition or notebook-scored workflow, design the final consumer notebook as the scoring artifact from the start.
• Use kaggle-competition-intel MCP tools when available: top_open_solutions for score-ranked plus latest high-vote notebooks, and competition_discussions for top-voted plus recently active discussion topics before choosing expensive baselines.
• Choose folds to match the hidden test distribution. Use stratification for classification, binned stratification for regression, group splits for repeated entities, time-aware splits for temporal data, and multilabel-aware splits when label co-occurrence matters.
• Save OOF predictions for every model. OOF files are the currency for error analysis, threshold tuning, blending, and stacking.
• Compare mean fold score and fold variance, not only the best fold.
• Treat a public LB jump that disagrees with CV as a validation question before treating it as a modeling victory.

Then build baselines in this order:

1. Metric-only or constant baseline to verify scoring and submission format.
2. Fast classical baseline: LightGBM/CatBoost/XGBoost for tabular, TF-IDF plus linear model for text, pretrained backbone for images.
3. Strong baseline with stable folds, clean artifacts, and OOF predictions.
4. Feature/model experiments with run logs and one primary metric.
5. Ensembling or stacking after several diverse, individually validated models exist.

Do not wait for the user to explicitly ask for sophistication when the competition warrants it. After a stable baseline, propose and execute a stronger staged architecture if public solutions, discussion intel, data modality, metric pressure, or CV plateau suggests that a single notebook/model will underperform. Keep the architecture gated by OOF evidence, artifact manifests, and final Kaggle scoring.

When a run is likely to exceed local RAM/VRAM, require a GPU/TPU, or needs Kaggle-only data mounts, prepare a Kaggle kernel run instead of forcing it locally. The remote run must emit experiment_log.json, metrics.jsonl, and an artifact manifest so results can be retrieved and interpreted after kaggle kernels output.

When the solution has multiple heavy stages, keep the remote graph shallow and inspectable:

• Use fewer than five independent producer notebooks/scripts per wave.
• Give each producer a single responsibility such as feature generation, embedding extraction, fold training, checkpoint training, model distillation, or inference.
• For code competitions, prefer producer notebooks that train models/checkpoints and export them as private Kaggle datasets; the final consumer should attach those datasets, load models from /kaggle/input/..., and perform hidden-test-safe inference.
• Save producer notebook outputs as versioned private Kaggle datasets for durable reuse and as stable inputs to downstream notebooks; use direct kernel_sources only for short-lived chains where durability/versioning is unnecessary.
• Run one final consumer notebook/script that attaches the produced datasets, reads their artifacts, writes the final OOF, test predictions, blend/stack report, and submission, then submit that final artifact/kernel to Kaggle for scoring.
• Use kagglehub inside Python when it is more convenient to download datasets, competition files, notebook outputs, or upload dataset versions programmatically.