Bio Spatial Transcriptomics Spatial Domains

Version Compatibility

Reference examples tested with: squidpy 1.4+, scanpy 1.10+, anndata 0.10+, scikit-learn 1.4+; BANKSY (banksy_py / R Banksy), BayesSpace 1.12+ (R), STAGATE/GraphST optional

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show <package> then help(module.function) to check signatures
• R: packageVersion('<pkg>') then ?function_name to verify parameters

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Spatial Domain Detection

"Identify tissue domains in my section" -> Partition the tissue into spatially contiguous regions of homogeneous expression/composition, using BOTH transcriptional similarity AND spatial proximity, so the output is a region label per spot/cell that is spatially coherent.

• Python: squidpy.gr.spatial_neighbors for the graph, then a neighbor-augmented or graph-based domain method (BANKSY, STAGATE, GraphST)
• R: BayesSpace (spatialCluster), BASS, or Banksy

Governing Principle

A spatial domain is a REGION, not a cell type, and not a niche -- conflating the three is the central conceptual error of this analysis. A CELL TYPE is one cell's transcriptional identity. A NICHE (cellular neighborhood) is the local cell-type COMPOSITION around a cell -- which types co-occur. A SPATIAL DOMAIN is a contiguous tissue region (a cortical layer, tumor core, stromal band) that contains MANY cell types and several niches. If the question is "which cell types co-occur," that is a niche question and belongs to neighborhood-enrichment analysis (spatial-statistics), NOT domain segmentation; running the wrong one answers the wrong question.

Domain methods exist because plain Leiden/Louvain on expression alone ignores coordinates and produces salt-and-pepper, spatially-incoherent labels. Every domain method deliberately adds a spatial term so neighbors tend to share a label. The load-bearing decision is therefore NOT the clustering algorithm -- it is the SPATIAL-WEIGHT knob (BANKSY lambda, BayesSpace smoothing, SpaGCN histology weight, GNN graph radius). Too much spatial weight is the #1 trap, over-smoothing: it erases real boundaries and merges biologically distinct regions into blobs. Too little reverts to salt-and-pepper. The number of domains k is the second decision, and it is a BIOLOGICAL choice (how fine a regionalization the question needs), not a statistic to optimize against a silhouette score -- report k, justify it, and show sensitivity to k+-1.

Domain vs Niche vs Cell Type

A domain can contain several niches; a niche can span domain boundaries. BANKSY and BASS switch between cell-typing and domain detection via a single parameter, which underscores that these are different outputs of related machinery, not the same task.

Domain Methods by Mechanism

The mechanism for "using the neighbors" is the axis that separates the methods. No method is a universal winner (Yuan 2024 Nat Methods); selection is scenario-specific.

Mechanism families: MRF/Bayesian smoothing (BayesSpace, BASS, PRECAST) vs graph neural net (STAGATE, GraphST, SpaGCN, SEDR) vs neighbor-augmentation (BANKSY, UTAG) vs histology-guided (stLearn, SpaGCN). Benchmarks evolve fast -- verify the current verdict for the tissue geometry before committing.

Read the Benchmark With the DLPFC Caveat

The standard ground truth is DLPFC (Maynard 2021 Nat Neurosci 24:425-436): 12 Visium sections of human dorsolateral prefrontal cortex, manually annotated into 6 cortical layers + white matter, scored by ARI. The Yuan 2024 (Nat Methods 21:712-722) benchmark of 13 methods x 34 datasets concludes there is NO single winner -- methods are complementary across accuracy, spatial continuity, marker detection, scalability, and robustness. On DLPFC, GNN/graph methods (STAGATE, SEDR, DeepST) and BayesSpace are among the most robust, and a technology-stratified benchmark (Chen 2025 iMeta 4:e70084) finds STAGATE/GraphST best on low-resolution Visium while BASS/stLearn/BANKSY lead on high-resolution platforms.

The caveat: DLPFC is a CONTINUOUS, LAMINAR tissue, which flatters smoothing-friendly methods. Do not over-generalize these rankings to non-laminar tissue. ALL methods struggle with NON-CONTIGUOUS domains (the same region appearing in separated patches -- scattered tumor nests, immune aggregates) because the spatial prior assumes contiguity; for non-contiguous biology, lower the spatial weight or switch to a niche/neighborhood analysis rather than domain segmentation.

Build the Spatial Graph

Goal: Construct the spatial neighbor graph that every domain method inherits.

Approach: Use a hex lattice for Visium (6 neighbors) and a generic kNN/Delaunay graph for imaging point clouds; the graph radius IS a spatial-weight knob (too dense over-smooths, too sparse fragments).

import squidpy as sq
import scanpy as sc

adata = sc.read_h5ad('preprocessed.h5ad')

# Visium hex lattice: 6 immediate neighbors. For imaging point clouds use
# coord_type='generic' with KNN or Delaunay. Build in microns where possible --
# a graph built in pixels has a different radius than one built in microns.
sq.gr.spatial_neighbors(adata, coord_type='grid', n_neighs=6)

Expression-Only Clustering Is the Salt-and-Pepper Baseline

Goal: Show why a domain method is needed at all.

Approach: Cluster on expression PCA with no spatial term; the result is spatially incoherent and demonstrates the failure domain methods correct.

sc.pp.neighbors(adata, n_neighbors=15, n_pcs=30)
sc.tl.leiden(adata, resolution=0.5, key_added='expr_leiden',
             flavor='igraph', n_iterations=2, directed=False)

# Plot on tissue: speckled, no contiguous regions -- this is the baseline a
# domain method is built to beat, not a domain result.
sq.pl.spatial_scatter(adata, color='expr_leiden')

Neighbor-Augmented Domains (BANKSY-style) and the Spatial-Weight Knob

Goal: Produce spatially coherent domains and make the over-smoothing knob explicit.

Approach: BANKSY concatenates each cell's own expression with its neighborhood-mean expression, mixed by lambda; lambda0.2 yields cell typing, lambda0.8 yields domains. A transparent neighbor-augmented matrix reproduces the idea with Squidpy when the BANKSY package is unavailable -- the lambda here is the load-bearing decision, not the clustering call.

import numpy as np
from sklearn.preprocessing import normalize

# Mean expression over each spot's spatial neighbors (the neighborhood signal).
W = normalize(adata.obsp['spatial_connectivities'], norm='l1', axis=1)
neighbor_mean = W @ adata.obsm['X_pca']

# lambda is the spatial-weight knob: ~0.8 for domains, ~0.2 for cell typing.
# Too high -> blobs (boundaries erased); too low -> salt-and-pepper.
lam = 0.8
augmented = np.concatenate(
    [np.sqrt(1 - lam) * adata.obsm['X_pca'], np.sqrt(lam) * neighbor_mean], axis=1)
adata.obsm['X_banksy'] = augmented

sc.pp.neighbors(adata, use_rep='X_banksy', key_added='banksy')
sc.tl.leiden(adata, resolution=0.5, key_added='domains', neighbors_key='banksy',
             flavor='igraph', n_iterations=2, directed=False)
sq.pl.spatial_scatter(adata, color='domains')

If the BANKSY package is installed, prefer it: import banksy_py (Python) or the R Banksy Bioconductor package compute the own + neighborhood-mean + azimuthal Gabor (AGF) features and expose lambda directly.