Bio Spatial Transcriptomics Spatial Deconvolution

Version Compatibility

Reference examples tested with: cell2location 0.1.4+, scvi-tools 1.0+, scanpy 1.10+, anndata 0.10+, numpy 1.26+

RCTD/SPOTlight/STdeconvolve are R packages (spacexr, SPOTlight, STdeconvolve via Bioconductor); call them from R or via rpy2.

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show <package> then help(module.function) to check signatures
• R: packageVersion('<pkg>') then ?function_name to verify parameters

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Spatial Deconvolution

"What cell types are in each spot?" -> Decompose a multi-cell capture spot into the fractions (or absolute numbers) of each cell type, using an annotated scRNA-seq reference as the basis.

• Python: cell2location (RegressionModel -> Cell2location), stereoscope/DestVI (scvi-tools), Tangram (tg.map_cells_to_space), STdeconvolve (reference-free, R)
• R: RCTD (spacexr::create.RCTD/run.RCTD), SPOTlight, SpatialDWLS (Giotto), CARD

The operation BRANCHES on the platform before any tool is chosen. The first question is not "which method" but "does this data even contain mixtures?" -- answered by the resolution fork below.

The resolution fork

Deconvolution recovers the cell-type composition of a MIXTURE. Whether a mixture exists is set by the capture unit size relative to a mammalian cell (~8-30um diameter), and it sorts every dataset into three regimes. Choosing the wrong regime is the most expensive error in spatial analysis -- far more damaging than picking the second-best method within a regime.

• DECONVOLVE: a capture array has no per-transcript cell assignment, so the spot is an unavoidable mixture and only mixture modeling recovers composition. The H&E underlay can count nuclei but cannot say which transcript came from which cell.
• SEGMENT (imaging platforms): the data are already single cells. The work is segmentation (assign transcripts to cells) then annotation (cluster + markers, or label-transfer). Running deconvolution here is conceptually WRONG -- it fabricates fractional mixtures inside cells that are already pure. See image-analysis for segmentation and single-cell/cell-annotation for label transfer.
• AMBIGUOUS (the near-single-cell middle): bins/beads still hold partial or multiple cells. When a high-quality registered image exists (Visium HD), morphology-driven cell RECONSTRUCTION (Bin2cell, StarDist/Cellpose nuclei expansion) is increasingly preferred over treating bins as fixed mixtures; without per-bead morphology (Slide-seqV2), assignment/deconvolution (often RCTD doublet-mode) remains standard. No settled consensus -- see high-resolution-binning.

Governing Principle

A deconvolution result is a PROJECTION of the single-cell reference onto the spatial data. The reference is not a neutral input -- it is the dominant determinant of the answer, more than the algorithm.

The #1 trap is the missing or mismatched cell type. If a real type is ABSENT from the reference, its transcripts are forced onto the transcriptionally nearest type that IS present. The proportions still sum to 1 and look clean, and there is no internal warning -- garbage reference produces confident garbage proportions. The reference must match the TISSUE, the CONDITION (a healthy reference mis-estimates activated-immune or malignant states whose expression has shifted), and ideally the technology/protocol (snRNA-seq vs whole-cell dissociation bias propagates straight into the numbers). The reference is NOT ground truth.

Every benchmark reinforces the same lesson: reference quality and the cell-type abundance pattern swing results MORE than method choice, and a plain NNLS baseline outperforms nearly half the dedicated methods (Sang-Aram 2024 eLife 12:RP88431; Li 2022 Nat Methods 19:662-670). Method-shopping is the wrong lever; reference quality is the right one. Rare-type fractions (below a few percent) are the least reliable numbers in the output -- corroborate any rare type with its spatial marker genes before believing it. More tools is not more confidence: two NB-regression methods agreeing is pseudo-replication, not orthogonal validation. The defensible confidence move is reference-sensitivity analysis (perturb or swap the reference) plus an orthogonal modality.

Outputs are COMPOSITIONAL: per-spot proportions live on a simplex (sum to 1), so an increase in one type mechanically decreases the others. Downstream comparison must use CLR/ILR or compositional tests -- naive per-type t-tests/correlations on raw proportions are mis-specified.

Choosing a method

The first axis is output: cell2location uniquely returns ABSOLUTE cell abundance (expected cell numbers per spot); the rest return proportions only. "Number of cells of type X" and "fraction of the spot that is type X" answer different biological questions. The second axis is runtime and whether spatial coordinates or a platform-shift correction are modeled. When competing methods exist, verify current best practice against the latest benchmark before committing -- this field moves fast.

cell2location and RCTD are reliably top-tier across independent benchmarks (Li 2022; Sang-Aram 2024; Nat Commun 2023 14:1548). When no matched reference exists, STdeconvolve is the escape hatch -- it is immune to the missing-type trap because it uses no reference, but its topics are de novo and must be annotated afterward.

cell2location step 1: reference signatures

Goal: Learn each cell type's per-gene expression signature from the annotated scRNA-seq reference, correcting for the technical/batch structure of the reference.

Approach: Filter genes to informative ones, fit a negative-binomial RegressionModel with the cell-type label and any batch as covariates, then export the posterior per-cluster mean expression. cell2location consumes RAW integer counts -- do NOT pass log-normalized data.

import cell2location
import numpy as np
import scanpy as sc
from cell2location.utils.filtering import filter_genes
from cell2location.models import RegressionModel

adata_ref = sc.read_h5ad('reference_scrna.h5ad')           # raw counts in .X
adata_ref.obs['cell_type'] = adata_ref.obs['cell_type'].astype('category')

selected = filter_genes(adata_ref, cell_count_cutoff=5, cell_percentage_cutoff2=0.03, nonz_mean_cutoff=1.12)
adata_ref = adata_ref[:, selected].copy()

# batch_key absorbs technical structure across reference samples; drop if a single batch
RegressionModel.setup_anndata(adata_ref, labels_key='cell_type', batch_key='sample')
mod = RegressionModel(adata_ref)
mod.train(max_epochs=250, accelerator='gpu')               # use_gpu= is deprecated; accelerator in {'gpu','cpu','auto'}
adata_ref = mod.export_posterior(adata_ref, sample_kwargs={'num_samples': 1000})

factors = adata_ref.uns['mod']['factor_names']
if 'means_per_cluster_mu_fg' in adata_ref.varm:
    inf_aver = adata_ref.varm['means_per_cluster_mu_fg'][[f'means_per_cluster_mu_fg_{f}' for f in factors]].copy()
else:
    inf_aver = adata_ref.var[[f'means_per_cluster_mu_fg_{f}' for f in factors]].copy()
inf_aver.columns = factors                                 # genes x cell_types signature matrix

cell2location step 2: spatial mapping

Goal: Decompose each spatial spot into absolute cell-type abundances using the reference signatures.

Approach: Restrict both objects to shared genes, set up the Cell2location model with the signature matrix and the expected cells-per-spot, then train. N_cells_per_location is a tissue-dependent prior (Visium ~10-30, denser tissue higher); detection_alpha controls within-experiment normalization (20 is the tutorial default; raise toward 200 if technical variability in total counts is high).

adata_vis = sc.read_h5ad('visium.h5ad')                    # raw counts
shared = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, shared].copy()
inf_aver = inf_aver.loc[shared, :]

cell2location.models.Cell2location.setup_anndata(adata_vis, batch_key='sample')
mod_sp = cell2location.models.Cell2location(adata_vis, cell_state_df=inf_aver,
                                            N_cells_per_location=30, detection_alpha=20)
mod_sp.train(max_epochs=30000, batch_size=None, train_size=1, accelerator='gpu')
adata_vis = mod_sp.export_posterior(adata_vis, sample_kwargs={'num_samples': 1000, 'batch_size': mod_sp.adata.n_obs})

# q05 = 5% posterior quantile = 'at least this many cells of this type are present' (conservative)
abundance = adata_vis.obsm['q05_cell_abundance_w_sf']      # ABSOLUTE expected cell numbers per spot
abundance.columns = factors
adata_vis.obs[abundance.columns] = abundance.values

cell2location returns absolute abundances. Convert to proportions only if proportions are the question -- doing so discards the information that distinguishes cell2location from the proportion-only methods.