Knowledge Bootstrap

Skill: Knowledge Bootstrap

Purpose

Initialize the knowledge subsystems for a new session. Resolve the active context source, load the small resident layer, and inventory the selected and compiled context that can be supplied after the user asks a question.

When to Use

• At the start of any session
• After /connect-data or /switch-dataset
• When the system detects missing or stale knowledge files

Instructions

Load each subsystem in order. An absent optional file can be reported as "not yet populated." A configured context source that is missing, invalid or inaccessible is different: report the problem and repair the connection before relying on its business definitions. Do not silently substitute local context.

Step 1: Setup State

Read .knowledge/setup-state.yaml.

• Parse setup_complete and count phases with status: "complete".
• If setup_complete: false, note incomplete phases to offer /setup.
• If missing: Note "Setup: not initialized -- offer /setup".

Step 2: Active Dataset

Read .knowledge/active.yaml.

• If active_dataset is null or missing, note "No active dataset" and continue.

• Resolve the context source first. Call resolve_context_dir(active, project_root) from helpers/knowledge/context_sync.py -> (ctx_dir, source). source: path reads the visible external store directly, without a cache. source: git uses the legacy Git cache; source: local reads .knowledge/datasets/{active}/. Load dataset knowledge (semantic/, metrics/, schema.md, quirks.md) from ctx_dir either way - the same loader, the source just differs. Report the source ("context: local" or "context: team repo @ {ref}") in the readiness summary.

• Inventory from ctx_dir. Load context-policy.yaml and custom_instructions.md as the resident layer. Do not load every metric, relationship, query, and correction into the prompt by default.

• Confirm these components are available:

Store layout — root or semantic/ (backward-compatible). Three of these files can live at EITHER the dataset root ({ctx_dir}/) OR under semantic/ ({ctx_dir}/semantic/), depending on the store's layout: custom_instructions.md, verified_queries.yaml, and corrections.md. Reconciled stores keep them at the dataset root; older stores keep the first two under semantic/. For each of the three, check the dataset root first; if present, load it from there, ELSE fall back to semantic/. Do not require one layout over the other, and do not skip the file just because it is absent from semantic/ — it may be at the root, and vice versa. The five pure-semantic YAMLs (entities, relationships, dimensions, measures, filters) always live under semantic/ and are not root-or-semantic.

corrections.md is the store-level communal corrections home — a human-curated, cross-session list of standing corrections that ship WITH the dataset context (root-or-nothing; there is no semantic/ fallback for it). It is DISTINCT from the per-session correction log at .knowledge/corrections/index.yaml loaded in Step 6 — that one is the local session log, this one is the communal store file. Load both; they are different subsystems.

Question-specific context before SQL. Once the exact analytical question is known, run /context-trace, or call helpers.knowledge.context_manifest directly. Load the selected items from that manifest, not the whole context store. Stop on a blocking conflict. Name stale or missing review evidence before relying on it. Resolve the question's metric, authoritative entities and relationships, real filter values, relevant verified queries, and applicable corrections from the selected bundle. A manifest proves what was supplied. It does not prove the worker used it. Reconcile cited items and SQL-use evidence after the analysis.

The context store separates three delivery modes:

• resident context is small and broadly applicable;
• selected context is chosen for the question and worker;
• compiled context is executable, deterministic context such as a metric compile block.

Workspace guidance and task guides. Call python -m helpers.connected_context --dataset DATASET catalog for version-2 guides, query entries and semantic resources. Follow docs/CONNECTED-CONTEXT.md for typed links, loading and execution. Do not reinterpret a draft or failed dependency as missing optional context. Legacy guide discovery remains available below.

For legacy guides, call helpers.knowledge.context_guides.guide_catalog(project_root, dataset=active). Apply the small workspace_guidance to this session. Inspect guide descriptions and scope once the question is known; do not preload every guide body. For a relevant guide call load_guide with its ID, catalog hash, question, selection reason and the current analysis ID. Read the returned content before querying. The helper logs that delivered content in working/context_loads_<analysis_id>.jsonl. If two sources conflict or scope is unclear, ask rather than silently choosing a definition. Draft, expired and other-dataset guides are listed as excluded. This is a simple agent-selected catalog, not vector search or Hex's proprietary retrieval algorithm. A loading record proves delivery, not correct application.

Schema generation if schema.md is missing (REQUIRED):

The schema is critical for SQL queries and analysis — never proceed without it. Follow this sequence:

1. Check data/schemas/{active}.yaml — if found, import schema_to_markdown() from helpers/data/schema_profiler.py and generate schema.md
2. If no YAML schema file exists, use get_connection_for_profiling() to query the live database and generate schema.md from introspection
3. For CSV datasets, read the first 1000 rows of each file with pandas, infer dtypes, and write schema.md with table/column/type info
4. Staleness check: if last_profile.md exists and is newer than schema.md, regenerate

After generation, write schema.md to .knowledge/datasets/{active}/schema.md so future sessions can load it directly.

System variables from manifest:

Extract these variables for use in SQL queries and agent prompts:

• {{SCHEMA}} — Schema prefix for external warehouses (e.g., "analytics", "prod")
• {{DISPLAY_NAME}} — User-friendly dataset name for status messages
• {{DATE_RANGE}} — Available date range (e.g., "2024-01-01 to 2026-03-31")
• {{DATABASE}} — Database name or connection string

For Snowflake use manager.table_reference(table) to construct DATABASE.SCHEMA.TABLE; schema alone is insufficient. Other data warehouses have their own naming rules. For local DuckDB/CSV a schema prefix is typically absent.