Geo Assets

Geo assets: download & process lunar terrain for a Twin

The application pipeline is composed by crates/lunco-assets; the heavy native processors live in crates/lunco-assets-processing. Pure Rust — no GDAL. Sources may be GeoTIFF or PDS3 .IMG (attached or detached .LBL; lunco-assets-processing/src/pds_img.rs); polar-stereographic products are refused loudly because the crop affine is equirectangular-only. Use the target Twin's own Assets.toml as the worked example and inspect its current scene before wiring outputs.

Wire baked terrain outputs into a scene through LunCoSim's USD document authoring commands. Open the exact USD source, inspect its layer and target, apply typed ApplyUsdOp(s) or the existing terrain-material planner, save, and read back the authored value. Do not patch .usd* text directly; if the available command surface cannot author a required standard field, add the smallest typed operation at the owning USD layer first.

Quick commands (run from this repo's root)

cargo run -p lunco-assets -- list     --twin <TWIN>
cargo run -p lunco-assets -- download --twin <TWIN>            # ALL entries (can be GBs)
cargo run -p lunco-assets -- download --twin <TWIN> -a <key>   # one entry
cargo run -p lunco-assets -- process  --twin <TWIN> -a <key> --quality coarse|good

<TWIN> = a folder holding Assets.toml + twin.toml. -a <key> = the [section] name in its Assets.toml. The same entries appear in-app under Settings ▸ Downloadable data and the Twin inspector once the Twin is open (scanned on open). Asset-consuming domains begin only after the asset owner publishes TwinAssetMounted; they must not depend on observer registration order. If a declared dataset is missing, the interactive app asks which entries to download before terrain generation; nothing downloads until the user confirms there or uses a CLI run. Closing a Twin retires its dataset rows under the shared download/process commit barrier and cancels their cooperative tasks before another Twin can reuse the authority. A failed mount or poisoned lifecycle lock is surfaced as an error, never reported as a successful asset postcondition. --quality coarse quarters target_resolution (floor 64) for a seconds-fast quick-start bake; re-run with good (default) for full res.

Downloads use the shared download section in the user settings file (lunco-settings::DownloadSettings): attempts include the first request, delays are exponential and capped, and a failed body read resumes from the received prefix with HTTP Range when the source supports it. The CLI and the interactive window therefore have one policy and one cache/path resolver.

Joining a cropped DEM to the rendered globe

The site keeps its own cropped DEM as the only elevation input for its handoff. The DEM asset and retained base grid remain unchanged, and the composed local surface preserves their relief throughout the crop. The crop's border datum sets the visible globe shell radius; celestial and physics state keep the canonical body radius. A worker-prepared exterior collar continues the measured edge profile from the exact crop boundary to the analytic sphere. This collar is visual closure outside the crop; terrain queries and colliders remain on the local surface. Globe tiles cut out the collar footprint with bounded orthographic edge sampling. Do not copy the collar into each tile or make global LOD follow DEM posting density. No body-wide raster asset is required.

Exterior smoothing preserves the first native posting and filters only the continuation in its preparation worker, without increasing mesh density. Keep the DEM appearance visibly distinct. See the geometry contract for ownership and sampling.

Derive the visual collar width from this crop's measured edge relief and one-sided edge slope with a 0.60 relief-grade sizing target. Continue the measured slope over one posting, then fade one nearest-perimeter signal to the sphere. Geometry and appearance share one width sized from maximum edge relief; the native inner posting lattice tapers to an outer boundary with at least 32 segments per side. Keep the DEM and all in-crop heights unchanged. Refine the globe only in the band between the exact crop and the collar's outer cutout; set the local minimum tile size from that handoff footprint, independent of DEM posting density. The collar reads map roles from the cropped DEM prim and appearance from the body's USD-selected UsdShade material. A continuation-capable WGSL shader and its USD Shader prim must declare the matching lunco.lunar-surface-continuation.v1 interface. Rhai admits compatible sources, waits for reflection, and holds simulation on a mismatch. The compositor keeps unadmitted collars hidden; the globe cutout waits for the composed material and visibility. RunLint checks the composed declaration against reflected WGSL. Read the body's composed look from GlobeLod, with its authored declaration entity as validation provenance. The shader asset path is never selected in Rust. Inspect the actual boundary vertices through bounded TerrainLodStatus geometry pages; mesh_entity selects a CPU-retained DEM boundary tile, including its morph positions. Use boundary_only: true to page only the finite perimeter. When bake sampling math changes, update the persistent visual tile cache revision in the same change. Check both rendered edges against mission queries before accepting a corner join.

The active crop supplies its own georeference, posting spacing, border datum, and measured edge profile, so this works with Twin-local crops at different sites and resolutions. A body currently has one built crop handoff; multiple built crops for the same body are a visible input error, not a size-based selection.

Verify automatic lunar DEM registration

For automatic lunar DEM registration, verify both repository fixtures scenes/tests/lunar_dem_continuation.usda (omitted coordinates) and scenes/tests/lunar_dem_georeferenced.usda (DEM-authored coordinates) in an owned headful production session. The shared continuation scenario waits for material admission and settled DEM streaming. Headless scene tests do not provide the GPU-retained boundary geometry required by this graphics verdict. The scene-site projection and zero defaults are specified in the geometry contract; no tutorial script installs the handoff.

Where files live (cache resolution)

• Shared cache — the OS-global cache (~/.cache/lunco on Linux, ~/Library/Caches/lunco on macOS, %LOCALAPPDATA%\\lunco on Windows). LUNCOSIM_CACHE remains an explicit CI/custom-install override. Every worktree and Twin therefore shares one pool of regenerable data (source libraries, textures, ephemeris, downloaded sources).
• Twin cache — <TWIN>/.cache. A Twin's default-owned downloads land beside the Twin. twin:// reads resolve <twin>/<rel> first, then <twin>/.cache/<rel>, then the global cache <cache>/<rel>.
• shared = true on an entry sends its write to the global pool instead (<cache>/sources/<sha256(url)[..16]>/<basename>) — one download per URL, reused by every twin and worktree. Use it when several Twins intentionally share a multi-GB upstream product; a Twin that must be self-contained should set shared = false.
• Raw downloads: entries without dest land in <owner cache>/sources/<url-hash>/<basename> — owner being the twin (--twin) or the shared cache (crate manifest). Author dest only when a file must sit at a specific path; it is then resolved against that same owner cache.
• Baked outputs (output_root = "twin"): inside the twin at output, where the scene's demSource/layer attrs expect them. Per-twin, always.

Git hygiene for downloaded and generated bytes

Before downloading a Twin dataset, add and verify ignore rules for the raw cache and generated processing outputs. The canonical policy is:

twins/**/.cache/
twins/**/terrain/*/materials/
twins/**/terrain/*/.bakekey

Raw downloads belong under the Twin's .cache (or the shared global cache when shared = true); processed DEMs, maps, and bake stamps are regenerable bytes. Commit Assets.toml, processing/reprojection tools, provenance and parameter reports, and USD references. Do not force-add cache or generated terrain bytes. Use git check-ignore -v <raw-file> <processed-file> before staging. A download-only manifest entry is honest when a source projection is unsupported; do not add a misleading [entry.process] section just to make a polar product look native. Use an explicit reprojection adapter and record its command and vertical datum, or record the native-tool gap as blocked work.

Process kinds (in [key.process])

The built-in processors are registered through lunco-assets-processing::process::ProcessorRegistry. A domain-specific native baker should register a ProcessorSpec and publish its sidecars through the shared staging/commit path. Put processor-specific manifest values in [key.process.parameters]; do not add a new shared manifest field for every domain. Keep selection, ordering, and onboarding policy in the reusable Rhai assets tool library or a Twin-owned script; Rust remains the owner of decoding, heavy math, cancellation, and atomic publication.

DatasetRegistry rejects process outputs that overlap another process output or any declared download source, including file paths nested under a directory output. The dem processor replaces its complete configured folder when it commits. Store independent material maps in sibling paths and point the scene at the DEM folder.

Use the existing assets Rhai library to process declared sources at runtime: assets::bake(id) dispatches ProcessDataset, and assets::bake_scope(scope) queues each idle processing declaration in a scope. Both read source identity and pipeline settings from Assets.toml; the command carries only the dataset id. ListDatasets reports completion through each entry's state. Calling bake again is safe: the content bake key skips current outputs. Adding another source or changing its output size/ROI therefore needs manifest and Rhai edits only; add a Rust processor only for a new transform.

Shared ROI fields: center_lat, center_lon, window_m, target_resolution = [n, n], pixel_scale_m, src_min/max_lat, src_min/max_lon, frame = "MOON_ME", output_root = "twin".

For a grayscale orthophoto used as terrain colour, add a separate albedo process entry. Its optional native-only parameters are albedo_base_linear (neutral material base, default 0.13), albedo_detail_strength (retained local contrast, default 0.35), and albedo_illumination_radius_m (low-frequency field radius, default 40). The processor writes a stable albedo.png; it does not claim to perform full photometric calibration. PDS3 .IMG sources supply their projection extent and pixel scale from the attached or detached label. A grayscale TIFF albedo source must author pixel_scale_m and all four src_* bounds in the process table; the grayscale TIFF decoder does not infer those geographic facts from its tags. Keep map for analysis/display outputs. In Rhai, assembly_builder::lunar_albedo_material_plan(...) returns standard USD SetAttribute operations for the produced albedo/normal assets; submit those through assembly_edit::batch or assembly_edit::propose. This keeps heavy image math in Rust while making the assembly policy replaceable and extensible.