Perforatedai Wandb

PerforatedAI WandB Guardrail Skill

This skill is a strict add-on for WandB + PerforatedAI workflows.

Purpose

• Follow the canonical guidance in api/wandb.md exactly.
• Reduce recurring implementation mistakes by enforcing checkpoints.
• Provide structured places to add new correction notes over time.

When To Use

Use this skill when the user asks for any of the following:

• WandB sweep integration with PerforatedAI
• WandB logging for perforated training
• Fixing a broken or incomplete WandB + PAI setup
• Guardrails/checklist for WandB + PAI edits

Do not use this as a replacement for base PAI integration steps when WandB is not involved.

Mandatory First Action

Before making any code changes, read the full canonical doc:

• api/wandb.md

If any local habit or previous pattern conflicts with wandb.md, wandb.md wins.

Global Guardrails (Always Enforce)

CRITICAL PRIORITY CHECK #1 (must be explicitly verified first on every WandB integration):

• Final metrics must always log from global maxima (global_max_val, global_max_train, global_max_params).
• At end-of-training final logging, the extra final arch log must only run when this condition is true:
  • current_integrated > last_logged_integrated and hasattr(wandb, "run") and wandb.run is not None
• Never skip or weaken this condition in the final logging block.

1. Never skip reading wandb.md first.
2. Never invent alternative WandB flow when wandb.md already defines one.
3. Always preserve the user's existing training logic unless a WandB/PAI requirement forces a change.
4. For sweep mode, ensure wandb.init() occurs in the sweep training function and config is read from wandb.config.
5. Apply dendritic hyperparameters from wandb.config before UPA.perforate_model().
6. Keep save_name aligned with wandb.run.name when available.
7. Use num_dendrites_integrated (not num_dendrites_added) for architecture-level logging.
8. Avoid duplicate final metric logging for perforated models.
9. Keep edits minimal and targeted; do not introduce formatting-only changes.

Required Execution Flow

Follow these sections in order. For each section:

• complete the checklist,
• run the verification,
• then proceed.

---

Section 1: Setup And Imports

Checklist

• import wandb exists.
• PAI imports exist:
  • from perforatedai import globals_perforatedai as GPA
  • from perforatedai import utils_perforatedai as UPA
• Sweep-related CLI args exist when needed (--sweep-id, --sweep-count, --wandb-project, optional --wandb-entity).

Verification

• Confirm script can parse args in both new-sweep and join-sweep modes.

Guardrail: Common Mistakes

• Mistake pattern: Missing one or more WandB CLI args.
• Detection rule: wandb.agent/wandb.sweep present but no --wandb-project arg.
• Correction rule: Add required args exactly as documented in wandb.md.

Space For Future Corrections

• Additional mistakes for Section 1:
  • TODO:
  • TODO:

---

Section 2: Sweep Function Pattern

Checklist

• Dedicated train_with_wandb() (or equivalent) exists.
• wandb.init() is called inside that function.
• config = wandb.config is used.
• If needed, silent mode is overridden for visibility (GPA.pc.set_silent(False)).

Verification

• Confirm training function runs under wandb.agent(..., function=train_with_wandb, ...).

Guardrail: Common Mistakes

• Mistake pattern: Using global/static config instead of wandb.config during sweeps.
• Detection rule: Sweep params exist in config but code never reads wandb.config.
• Correction rule: Route sweep hyperparameters through wandb.config in the sweep function.

Space For Future Corrections

• Additional mistakes for Section 2:
  • TODO:
  • TODO:

---

Section 3: Apply PAI Config Before Perforation

Checklist

• Dendritic sweep params are read from wandb.config.
• GPA.pc.set_improvement_threshold(...) and optional forward-function mapping are set before perforation.
• UPA.perforate_model(...) is called only after required PAI config is applied.

Verification

• Confirm code path that calls UPA.perforate_model(...) has already applied sweep-controlled PAI settings.

Guardrail: Common Mistakes

• Mistake pattern: Applying PAI sweep params after perforation.
• Detection rule: UPA.perforate_model(...) appears before dendritic config setters.
• Correction rule: Move relevant GPA.pc.set_* calls to before UPA.perforate_model(...).

Space For Future Corrections

• Additional mistakes for Section 3:
  • TODO:
  • TODO:

---

Section 4: save_name And Run Identity

Checklist

• save_name is derived from wandb.run.name when available.
• Fallback save name exists when run name is unavailable.
• Local result folders and WandB run identity are consistent.

Verification

• Confirm a run produces local output folder matching WandB run naming pattern.

Guardrail: Common Mistakes

• Mistake pattern: Hardcoded save_name causes mixed/overwritten runs.
• Detection rule: Sweep code present but constant string used for save_name.
• Correction rule: Use wandb.run.name with safe fallback.

Space For Future Corrections

• Additional mistakes for Section 4:
  • TODO:
  • TODO:

---

Section 5: Metric Logging During Training

Checklist

• Epoch-level metrics are logged (Train*, Val*, optional Test*, LR, params, dendrite count).
• Logging checks wandb.run exists before wandb.log(...).

Verification

• Confirm no logging crash when WandB run is missing or disabled.

Guardrail: Common Mistakes

• Mistake pattern: Unconditional wandb.log(...) causing runtime errors.
• Detection rule: No run-availability check around logging.
• Correction rule: Guard wandb.log(...) calls with run existence checks.

Space For Future Corrections

• Additional mistakes for Section 5:
  • TODO:
  • TODO:

---

Section 6: Architecture-Level Logging For Perforated Models

Checklist

• Track architecture maxima only in neuron mode (mode == 'n').
• Log arch metrics when integrated dendrite count increases.
• Use num_dendrites_integrated, not attempted count.
• Reset arch trackers after each successful architecture log.

Verification

• Confirm one arch log entry per integration step and no duplicates.