Kaniop Development

Kaniop development

Detailed project guidance for agents working with code in this repository.

Project Overview

Kaniop is a Kubernetes operator for managing Kanidm identity services. It provides declarative management of Kanidm clusters and identity resources (persons, groups, OAuth2 clients, service accounts) through Kubernetes Custom Resources.

Tech Stack: Rust (edition 2024), Cargo workspace, Kubernetes controller-runtime (kube-rs), Kanidm client SDK, Helm charts, Kind for e2e testing.

Essential Commands

Development Workflow

# Lint and format check (must pass with zero warnings)
make lint

# Run unit tests (includes lint)
make test

# Build debug binaries (kaniop and kaniop-webhook)
make build

# Build optimized release binaries
make release

# Regenerate CRDs after any CRD spec changes (REQUIRED)
make crdgen

# Regenerate example YAML files (do not hand-edit examples/)
make examples

# Update version across workspace and Helm chart
make update-version

Testing

# Integration tests (requires Tempo tracing container)
make integration-test

# Create Kind cluster, build images, install operator
make e2e

# Run e2e tests (requires Kind cluster from `make e2e`)
make e2e-test

# Rapid iteration: rebuild operator and reload into Kind cluster
make update-e2e-kaniop

# Clean e2e resources but keep cluster
make clean-e2e

# Delete Kind cluster completely
make delete-kind

Test a Single e2e Test

# After `make e2e` has created the cluster:
RUST_TEST_THREADS=16 cargo test -p kaniop-e2e-tests --features e2e-test <test_name>

Run One e2e Shard Locally

# After `make e2e` has created the cluster:
make e2e-test-shard SHARD=kanidm-core

# Verify shard filters still cover every test (run after adding/removing tests):
make check-e2e-shards

Valid shards: kanidm-core, kanidm-ha, kanidm-backup, kanidm-restore, kanidm-restore-hardening, oauth2, resources, misc. Shard filters are defined in the Makefile (E2E_SHARD_FILTER_* / E2E_SHARD_SKIP_*) and consumed by the CI e2e job matrix in .github/workflows/rust.yml.

Build and Push Images

# Build single-arch local images
make images

# Build and push multi-arch images (amd64, arm64)
make push-images

Architecture

Workspace Structure

Binaries (cmd/):

• cmd/operator: Main operator binary with multiple controllers
• cmd/webhook: Validating admission webhook
• cmd/crdgen: CRD generator (outputs to charts/kaniop/crds/crds.yaml)
• cmd/examples: Example YAML generator (outputs to examples/)

Libraries (libs/):

• libs/operator: Core operator framework (controller runner, metrics, telemetry, Kanidm cluster CRD)
• libs/person: KanidmPersonAccount controller and reconciler
• libs/group: KanidmGroup controller and reconciler
• libs/oauth2: KanidmOAuth2 controller and reconciler
• libs/service-account: KanidmServiceAccount controller and reconciler
• libs/k8s-util: Kubernetes client utilities, error types, patch helpers

Tests (tests/):

• tests/e2e/: End-to-end tests with feature flag e2e-test
• tests/integration/: Integration tests with feature flag integration-test

Important: Never use both feature flags together.

Controller Pattern

Each identity resource follows the same pattern:

1. CRD (crd.rs): Defines the Kubernetes Custom Resource
2. Controller (controller.rs): Sets up the kube-rs controller with watchers and reconciler
3. Reconciler (reconcile.rs): Implements the reconciliation logic using Kanidm client SDK

All controllers:

• Use exponential backoff on errors via backoff_reconciler! macro
• Share a common Context trait pattern with IdmClientContext and BackoffContext
• Access Kanidm via KanidmClient cached per-cluster
• Report metrics via Prometheus client

Key Components

libs/operator/src/controller.rs: Core controller infrastructure

• State: Shared state across all controllers (metrics registry, Kanidm client cache)
• Context trait: Provides access to k8s client, metrics, Kanidm client
• create_subscriber(): Sets up controller with retry backoff

libs/operator/src/kanidm/: Kanidm cluster management

• controller.rs: Reconciles Kanidm StatefulSet, Services, PVCs
• crd.rs: Kanidm cluster CRD definition
• reconcile/: Stateful set, service, secret management logic

libs/k8s-util/src/error.rs: Centralized error handling

• Error enum for all controller errors
• Result<T> type alias used throughout

Reconciliation Flow

1. Kubernetes watch event triggers reconciler
2. Reconciler retrieves Kanidm client from shared cache (via Context::get_idm_client())
3. Reconciler performs idempotent operations against Kanidm API
4. Controller tracks backoff state on errors, retries with exponential backoff
5. Status updates written back to CR

e2e Testing

• Cluster: Kind cluster named chart-testing (context: kind-chart-testing)
• Namespace: kaniop
• Test execution: Runs in serial per-resource (RUST_TEST_THREADS=16 for parallel tests, override via E2E_TEST_THREADS)
• Diagnostics: On failure, operator logs dumped automatically
• Image tagging: Uses git SHA as version tag

e2e Iteration Workflow

When iterating on code changes with e2e tests:

1. Initial setup (once per session):

   make e2e  # Creates Kind cluster, builds images, installs operator

2. Rapid iteration cycle:

   # After code changes, rebuild and reload operator into cluster
   make update-e2e-kaniop
   
   # Run a specific test
   RUST_TEST_THREADS=16 cargo test -p kaniop-e2e-tests --features e2e-test <test_name>
   
   # Or run all e2e tests
   make e2e-test

3. Cleanup between test runs (if tests fail with "already exists" errors):

   # Clean up leftover test resources but keep cluster running
   make clean-e2e
   
   # Then re-run tests
   make e2e-test

4. Delete specific leftover resources (for targeted cleanup):

   kubectl delete kanidmgroup <name> -n default --ignore-not-found=true
   kubectl delete kanidmpersonaccount <name> -n default --ignore-not-found=true
   kubectl delete kanidmoauth2client <name> -n default --ignore-not-found=true
   kubectl delete kanidmserviceaccount <name> -n default --ignore-not-found=true
   kubectl delete kanidm <name> -n default --ignore-not-found=true

5. Full reset (when cluster is in bad state):

   make delete-kind && make e2e

Common e2e test failure patterns:

• "already exists" errors → Run make clean-e2e to remove leftover resources
• "admission webhook denied" with duplicate message → Previous test didn't clean up; delete the specific resource
• Tests pass individually but fail in batch → Resource name collision; ensure test names are unique

Development Practices

Code Style

• Imports: Always at top-level, grouped: std → external crates → internal crates → crate-local
• Never inline imports within functions or impl blocks
• Async: Never block Tokio runtime; use retry/backoff for transient failures
• Performance: Avoid unnecessary clone, prefer references and iterators
• Error handling: Use anyhow::Context for error wrapping, domain-specific error enums

Helm Chart Changes Workflow

1. Modify charts/kaniop/values.yaml — add value with YAML doc comments
2. Update charts/kaniop/values.schema.json — add matching JSON schema entry
3. Update relevant template(s) in charts/kaniop/templates/
4. Add unit tests in charts/kaniop/tests/ — test both default and custom values
5. Run helm unittest charts/kaniop to verify

Operator Configuration Pattern

When adding a new operator config via env var:

1. Add clap arg with #[arg(long, default_value = "...", env)] in cmd/operator/src/main.rs
2. Add global OnceLock getter/setter in libs/operator/src/controller/mod.rs
3. Call setter in main() after args parsing
4. Import and use in reconcilers via crate::controller::function_name