Sync Catalog

Sync Catalog Plugin

Propagate OpenAPI spec changes through all downstream files for a catalog plugin.

Phase 1: Identify Plugin

Ask the user which plugin to sync. Validate by checking that these exist:

• api/openapi/src/plugins/<name>.yaml — the spec
• catalog/internal/plugins/<name>/plugin.go — the plugin entry point
• catalog/internal/catalog/<name>catalog/ — the domain package

Phase 2: Regenerate Auto-Generated Code

Run these three make targets in order:

make api/openapi/catalog.yaml
make -C catalog gen/openapi-server
make -C catalog gen/openapi

Report what was regenerated. Check go build ./catalog/... — if it fails, proceed to Phase 4 (service sync) first since interface mismatches are the most common cause.

Note: catalog/internal/server/openapi/type_asserts_overrides.go is a hand-maintained file that overrides auto-generated assert functions for polymorphic types (e.g., AssertCatalogArtifactRequired, AssertFilterOptionRequired). If gen/openapi-server regenerates type_asserts.go and the build fails with undefined assert function errors, check whether this overrides file needs updating for the new types.

Phase 3: Sync Datastore Entries + Entity Mappings

3a: Detect new fields

Read the plugin's OpenAPI spec (api/openapi/src/plugins/<name>.yaml) and extract all properties from each entity schema under components.schemas.

Read the current DatastoreEntries in catalog/internal/plugins/<name>/plugin.go and extract the registered property names (the string arguments to .AddString(), .AddStruct(), .AddBoolean()).

Compute the diff: fields in the spec but not in DatastoreEntries.

Exclude these fields from the diff (they are handled automatically by the schema layer, not as properties):

• id, name, externalId, external_id
• createTimeSinceEpoch, lastUpdateTimeSinceEpoch
• customProperties, custom_properties
• nextPageToken, pageSize, size, items (list-level fields)

3b: Ask user which fields to persist

Present the diff to the user. For each new field show:

• Field name
• OpenAPI type
• Suggested Add*() call based on type:
  • string → AddString
  • boolean → AddBoolean
  • integer / number → AddString (stored as string property by convention)
  • array / object → AddStruct

Ask: "Which of these new fields should be persisted in the datastore?" (multi-select)

3c: Apply changes

For each confirmed field, update three files:

1. DatastoreEntries in catalog/internal/plugins/<name>/plugin.go:

• Find the entity's DatastoreEntry block
• Add the new .Add*("<field_name>") call before the closing comma

2. Entity mappings in catalog/internal/catalog/<name>catalog/service/entity_mappings_<entity>.go:

• Add entry to the properties map variable:

  "<field_name>": {Location: filter.PropertyTable, ValueType: filter.<ValueType>, Column: "<field_name>"},

  Where ValueType is:
  • StringValueType for string/integer/number
  • BoolValueType for boolean
  • ArrayValueType for array

3. Entity mappings test in catalog/internal/catalog/<name>catalog/service/entity_mappings_<entity>_test.go:

• Add entry to the expected properties map with the same definition

3d: Ask about filterability

Ask the user: "Which of these fields should be directly filterable as query parameters?" (multi-select from the persisted fields)

For filterable fields:

• Add a field to the ListOptions struct in catalog/internal/catalog/<name>catalog/models/<entity>.go
• Add a WHERE clause in apply<Entity>ListFilters in catalog/internal/catalog/<name>catalog/service/<entity>.go:

  if listOptions.<FieldName> != nil {
      query = query.Where("<field_name> = ?", *listOptions.<FieldName>)
  }

Phase 4: Sync Service Implementation

Read the regenerated service interface from catalog/internal/server/openapi/api_<name>.go. Read the current service implementation from catalog/internal/server/openapi/api_<name>_catalog_service_service.go.

Compare method signatures:

• Changed signatures: Update the implementation method to match the new interface signature. Keep the existing body logic, just fix the parameters.
• New methods: Add a stub implementation returning an empty/not-found response (same pattern as init-catalog Phase 6).
• Removed methods: Delete the implementation method.

If the service implementation file doesn't exist yet (e.g., it was deleted during a regen), recreate it following the init-catalog Phase 6 pattern.