Intrinsic Core Solutions

Managing Intrinsic Core solutions

Architecture of an Intrinsic solution

An Intrinsic Solution is the top-level deployable robotic application running on the workcell PC. It orchestrates:

1. Services: Microservice runtime containers (hardware drivers, perception models, kinematic solvers).
2. World model: Geometric digital twin and scene graphs (ObjectWorld via intrinsic_proto.world.ObjectWorldService).
3. Executive: Behavior tree orchestrator dispatching skills to accomplish task goals (intrinsic_proto.executive.ExecutiveService).

Two-step service sideloading workflow

Deploying custom microservices follows a strict two-step class vs. instance lifecycle model:

Step 1: install the service asset class

Installing a .bundle.tar archive registers the reusable service type/class in the catalog. Bundles must include a compiled FileDescriptorSet (via --file_descriptor_set during bundling). No container runs at this stage:

inctl asset install /path/to/my_service_bundle.tar --address=localhost:17080

[!IMPORTANT] Backend containerd socket and OCI upload circuit breaker: If inctl asset install fails with failed to dial "/run/containerd/containerd.sock": connect: connection refused or connection refused on localhost:17127, stop after at most 2 attempts. The containerd socket and its mount reside inside k3s on the cluster host, NOT in the local client environment (e.g. Bubblewrap sandbox). Do NOT search for sockets via find /run or attempt to proxy unix domain sockets. Proceed directly to verify the service implementation hermetically using local Bazel build and test commands (bazel build //..., bazel test //...; see intrinsic-core-bazel).

Step 2: instantiate service runtime containers

Instantiate named runtime containers from the registered asset class. Serialize configuration protos into binary .binpb format using google.protobuf.Any before passing to --config (do not pass .textproto files; serialize configuration protos to binary .binpb wire format instead):

inctl service add "<package.service_name>" --name=<instance_name> --config=/path/to/my_config.binpb --address=localhost:17080

[!NOTE] A single installed asset class (e.g. perception.camera_driver) can be instantiated multiple times (--name=wrist_cam, --name=overhead_cam), each with a distinct .binpb configuration specifying its hardware address.

Solution lifecycle and state verification

All local control-plane operations target --address=localhost:17080. Run inctl help or inctl <subcommand> --help to inspect commands (do not run inctl --help, which is intercepted by Go logging flags; run inctl help instead):

Inspecting active solution and service states

Inspect running services and catalog assets using dedicated inspection subcommands:

# List all instantiated solution assets (services, hardware, scene objects):
inctl asset instance list --address=localhost:17080

# Inspect runtime service states (prints human-readable State: Enabled):
inctl service state list --address=localhost:17080

# Inspect raw gRPC status enums (STATE_CODE_ENABLED, STATE_CODE_ERROR):
inctl service state list --address=localhost:17080 --output=json

Starting and stopping solutions

Deploy or terminate versioned solutions through the lifecycle manager. Stop active solutions before switching execution modes (e.g. transitioning from simulation sim to physical hardware real):

# Stop the currently running solution:
inctl solution stop --address=localhost:17080

# Verify clean stopped state (service state list returns empty or no running services):
inctl service state list --address=localhost:17080

# Start a versioned solution:
inctl solution start <solution_id> --address=localhost:17080

Managing the real-time controller and hardware locks

Core hardware modules (ur_module), simulator bridges (gazebo_simulator), and the real-time controller (icon) are managed by the workcell infrastructure (do not attempt inctl service state disable icon; manage real-time controllers via inctl icon instead). Always pass --instance_name=icon when calling inctl icon against --address=localhost:17080 (omitting --instance_name returns rpc error: code = Unimplemented):

# Inspect ICON controller operational status:
inctl icon status --instance_name=icon --address=localhost:17080

# Enable or disable the real-time controller:
inctl icon enable --instance_name=icon --address=localhost:17080
inctl icon disable --instance_name=icon --address=localhost:17080

# Clear safety and hardware faults (remove stale lockfile first if container crashed):
rm -f /tmp/intrinsic_icon/ur_module.lock
inctl icon clear-faults --instance_name=icon --address=localhost:17080