Hz Spatial SDK

Spatial SDK Skill

Build native Android spatial applications for Meta VR using the Meta Spatial SDK. This skill covers the Entity-Component-System architecture, 2D panel rendering, 3D object placement, hybrid app development, and deployment to Horizon OS devices.

When to Use This Skill

Use this skill when you need to:

• Build a native Android app for Meta VR using the Spatial SDK and Kotlin
• Create hybrid experiences that combine 2D Android UI panels with 3D content
• Work with the Entity-Component-System (ECS) architecture in Spatial SDK
• Add 3D objects, animations, or spatial interactions to a Meta VR application
• Configure panels using Jetpack Compose or Android Views for spatial rendering
• Use the Spatial Editor to compose 3D scenes visually
• Deploy and test Spatial SDK applications on a Meta VR device

This skill applies to all Meta Quest headsets running Horizon OS (Quest 2, Quest 3, Quest 3S, Quest Pro).

What is Meta Spatial SDK

Meta Spatial SDK is Meta's native Android framework for building spatial applications on Horizon OS. It extends the standard Android development model with spatial capabilities, allowing developers to write apps in Kotlin that render 2D UI panels in 3D space, display glTF models, handle spatial input, and integrate with Horizon OS features like passthrough, scene understanding, and hand tracking.

Unlike Unity or Unreal Engine, Spatial SDK builds on top of the Android Activity lifecycle. Applications are standard Android APKs that use Spatial SDK libraries to gain spatial rendering and interaction capabilities.

Key characteristics:

• Kotlin-first: all application logic is written in Kotlin
• Android-native: builds on standard Android Activity, Gradle, and Jetpack libraries
• ECS architecture: entities, components, and systems manage 3D scene state
• Panel rendering: Android UI frameworks (Jetpack Compose, Views) render as spatial panels
• Gradle integration: Spatial SDK ships as AAR libraries pulled via Gradle dependencies

Key Concepts

Entity-Component-System (ECS)

The Spatial SDK uses an ECS architecture to manage the 3D scene graph. This separates data (components) from behavior (systems):

• Entity: a lightweight identifier (ID) that groups components together. An entity has no behavior on its own.
• Component: a data container attached to an entity. Components are defined via XML attribute schemas and hold typed fields (floats, vectors, references, enums). Examples: Transform, Mesh, Panel, Grabbable.
• System: a Kotlin class that queries entities by their components and executes logic each frame. Systems extend SystemBase and override the execute() method.

// Example: a simple system that rotates all entities with a Spinner component
class SpinnerSystem : SystemBase() {
  private var previousTime = 0L

  override fun execute() {
    val currentTime = System.currentTimeMillis()
    if (previousTime == 0L) previousTime = currentTime
    val timeDeltaInSeconds = (currentTime - previousTime) / 1000f
    previousTime = currentTime

    val query = Query.where { has(Spinner.id, Transform.id) }
    for (entity in query.eval()) {
      val transform = entity.getComponent<Transform>()
      val spinner = entity.getComponent<Spinner>()
      transform.transform.q =
        transform.transform.q * Quaternion(0f, spinner.speed * timeDeltaInSeconds, 0f)
      entity.setComponent(transform)
    }
  }
}

2D Panels

Panels are the primary way to display Android UI in spatial apps. A typed PanelRegistration maps an integer resource ID to a Jetpack Compose composable, an Android View, an Activity, or a media surface. Panels render as surfaces positioned in the 3D scene.

override fun registerPanels(): List<PanelRegistration> {
  return listOf(
    ComposeViewPanelRegistration(
      registrationId = R.id.main_panel,
      composeViewCreator = { _, context ->
        ComposeView(context).apply { setContent { MainScreen() } }
      },
      settingsCreator = {
        UIPanelSettings(
          shape = QuadShapeOptions(width = 0.8f, height = 0.6f),
          display = DpDisplayOptions(width = 592f, height = 444f),
        )
      },
    )
  )
}

3D Objects

Load glTF models as meshes and place them in the scene using Transform and Mesh components:

val modelEntity = Entity.create()
modelEntity.setComponent(
  Mesh(Uri.parse("apk:///models/robot.glb"))
)
modelEntity.setComponent(
  Transform(Pose(Vector3(0f, 1f, 2f)))
)

Hybrid Apps

Spatial SDK excels at hybrid applications that combine 2D panels with 3D content. A single activity can display Android UI panels alongside 3D models, allowing users to interact with familiar 2D interfaces while surrounded by spatial content.

Activity Structure

For most Spatial SDK apps, keep one AppSystemActivity subclass as the root shell for the whole experience. Tool-style apps usually work best when that single activity owns the scene, registered panels, and ECS systems while UI states change inside that shell.

Avoid structuring a Quest-native tool app like a standard multi-activity Android app unless you have a specific platform reason. Multiple panels or different UI states are usually better expressed inside the same spatial activity.

Scene

The Scene class manages the 3D environment, including the skybox, image-based lighting (IBL), view origin, and reference space. Each AppSystemActivity has an associated scene.