Dataproduct Implement

Implement a data product from its data contract

Turn an Entropy Data data product into a working dbt pipeline. The data contract (ODCS) is the source of truth for output schema; this skill reads it and writes the dbt artifacts that produce data matching the contract.

When to use this vs. other skills

• Empty directory, no dbt project yet → run dataproduct-bootstrap first, then come back here.
• Existing dbt project, need ODPS/ODCS/OpenLineage scaffolding only → use entropy-data-sync instead.
• Existing dbt project, want to derive models from a published data contract → this skill.

How to run this skill

${PLUGIN_ROOT} below refers to the root of this plugin — the directory that contains skills/. On Claude Code it is set automatically as ${CLAUDE_PLUGIN_ROOT} — use that. On any other agent (Codex, Copilot CLI, etc.) it is unset; resolve it as ../.. relative to this SKILL.md file's directory (i.e. the grandparent of skills/<this-skill>/).

Plan announcement (before Step 0)

Before running Step 0, print this plan to the user verbatim:

Running dataproduct-implement. I'll:

1. Pre-checks: confirm this is a dbt project, the dbt CLI is installed, and the entropy-data CLI is connected.
2. Resolve the data product by id or URL (entropy-data dataproducts get).
3. Fetch each selected output port's data contract (entropy-data datacontracts get) and save it next to the SQL it governs, under models/output_ports/v<N>/.
4. Validate the contract against the target platform's conventions (e.g. UPPERCASE identifiers on Snowflake). If fixable bugs are found, offer to patch and publish the corrected contract back to Entropy Data.
5. Translate the ODCS schema into dbt models under models/output_ports/v1/ (column list, types, tests).
6. Implement the dbt model bodies: declare input ports as dbt sources, cache each upstream contract under models/input_ports/<provider-op-id>.odcs.yaml as a trust snapshot, and write the select from input ports to output columns (with confirmation; complex joins left as TODOs).
7. Stamp the data product on Entropy Data with the dataProductBuilder customProperty so the platform knows it is managed by this builder.
8. Hand off to entropy-data-sync to add any missing publishing artifacts (ODPS, OpenLineage, GitHub Actions).
9. Summarize what was generated and the open TODOs.

Then proceed.

Step 0 — Pre-checks

• Confirm dbt_project.yml exists at the working directory root. If not, ask whether to run dataproduct-bootstrap first, then stop.
• Confirm uv run --quiet dbt --version succeeds from the project root. If it fails, run uv sync and retry; if still missing, stop and tell the user to add the dbt adapter for their warehouse to pyproject.toml's [dependency-groups].dev (e.g. dbt-snowflake, dbt-databricks, dbt-bigquery, dbt-postgres) and re-run uv sync. Use uv run dbt … for every dbt CLI invocation in this skill.
• Confirm uv run --quiet entropy-data --version succeeds from the project root. If it fails, run uv sync and retry. Once available, run uv run entropy-data connection test. If that fails, stop and tell the user to run uv run entropy-data connection add <name> --host <host> --api-key <key>. Use uv run entropy-data … for every CLI invocation in this skill.

Step 1 — Resolve the data product

Accept either:

• a full URL (e.g. https://app.entropy-data.com/dataproducts/<id>) — extract the trailing id, or
• a bare data product id.

Run entropy-data dataproducts get <id> -o yaml. Remember the response as DATA_PRODUCT. Extract:

• DATA_PRODUCT_ID, DATA_PRODUCT_NAME, owning team, purpose
• the list of output ports — each has an id, a server (catalog/schema/table), and a linked data contract id

If the data product has more than one output port, ask the user which one(s) to implement in this run. Default to all.

If the data product does not exist in Entropy Data, ask the user if they want to create a new one.

Step 2 — Fetch the data contracts

For each selected output port, run entropy-data datacontracts get <contract-id> -o yaml with the contract id from the data product. Remember the response as CONTRACT, and write it to models/output_ports/v<N>/<contract-id>.odcs.yaml (the version directory matches the output port's version — default v1 if the data product does not declare one). If the file already exists and differs from the fetched contract, surface the diff and ask before overwriting.

The fields you need from CONTRACT:

• models (table name → list of fields with type, required, unique, description, classification)
• servers (so the output port's server config is consistent with the contract)
• terms and quality rules — useful context but not required to materialize the model

Step 2.5 — Validate the contract against the target platform

Scan the contract for convention bugs and offer to fix them in one pass, with the patched contract published back to Entropy Data. Dispatched off servers[].type. Server types not listed below are skipped silently — add a section when extending support.

Snowflake — for every property in every schema covered by a type: snowflake server:

• Mixed-case name. Snowflake folds unquoted identifiers to UPPERCASE; datacontract-cli (≥ 0.11.5) quotes name verbatim. Any lowercase letter in name makes the soda query miss the stored UPPERCASE column. Normalize name to UPPERCASE.
• Redundant physicalName. When physicalName equals the UPPERCASE form of name, drop it.

If nothing is flagged, continue silently to Step 3. Otherwise list every fix (one bullet per property × issue) and ask:

Found N convention issue(s) for <server-type> on contract <CONTRACT_ID>:

• property <old-name>: rename name → <NEW-NAME>
• property <NEW-NAME>: drop redundant physicalName: <value>

Apply, save to models/output_ports/v<N>/<contract-file>.odcs.yaml, and publish back to Entropy Data? [Y/n]

On Y: patch the local file with yq -i, keep version unchanged (convention fix, not a schema change), entropy-data datacontracts put <CONTRACT_ID> --file <path>, re-read CONTRACT, continue. Stop on non-2xx.

On n: warn that datacontract test will fail on the un-normalized properties and continue. Don't re-ask this run.

Step 3 — Translate ODCS schema to dbt artifacts

Output column identifier rule (applies to this step and Step 4). Use the contract property's name directly as the SQL alias and the _models.yml columns: - name: entry. Don't substitute physicalName — datacontract test queries by name. Per-warehouse case conventions are enforced by Step 2.5; this step trusts the post-validation name.

For each contract:

1. Decide a dbt-side table name. Default: the models key in the contract. Confirm with the user if it differs from the output-port server's table name.

2. Identify candidate input ports. Run entropy-data access list --consumer-dataproduct <DATA_PRODUCT_ID> -o json to list the access agreements where this product is the consumer. Each entry's provider.dataProductId / provider.outputPortId is an input port this product can read. Keep only agreements with info.active: true (status approved); ignore pending / rejected. Only fall back to a broader entropy-data search query if the user explicitly asks. If models/input_ports/<provider-output-port-id>.source.yaml already exists for an agreement, treat it as authoritative and skip recreating it.

3. Generate models/output_ports/v1/<table>.sql — a stub select that lists the contract columns explicitly with cast(... as <warehouse-type>) as <column>. Leave the from clause as a TODO with a comment listing the candidate input ports from the previous step; do not invent business logic. Prepend a one-line header comment so a reader of the file knows which contract governs the schema:

   -- Governed by <contract-file>.odcs.yaml (ODCS id: <CONTRACT_ID>)

   The contract file sits in the same directory as the SQL, so the comment names the file without a path prefix.

4. Append the column list to models/output_ports/v1/_models.yml under models: — name, description (from contract), and tests derived from the contract: not_null for required: true, unique for unique: true, accepted_values if the contract defines an enum. Add a config.meta.data_contract block on the model that points back to the contract — this is the machine-readable counterpart to the SQL header comment, so dbt list --select config.meta.data_contract.id:<id>, dbt-docs, and lineage tooling can discover the link:

   models:
     - name: <table>
       description: <from contract>
       config:
         meta:
           data_contract:
             id: <CONTRACT_ID>
             file: models/output_ports/v<N>/<contract-file>.odcs.yaml
       columns:
         - ...

5. Map ODCS types to the warehouse dialect:

Pick the dialect from the contract's servers[].type (or, if absent, ask).