Shape-based methods for biological imaging

Biological measurement technologies now generate data at unprecedented spatial and temporal resolution, but the measurements are corrupted by variability that hides the biology. A cell's orientation on a slide says nothing about its morphology; brain tissue deforms between imaging days; people are filmed from arbitrary angles and move at different speeds. Following David Kendall, who defined the shape of an object as what remains after its position, orientation and size are removed, I build probabilistic models that are invariant to these nuisance dimensions, so that what remains is the biological signal. The tools come from shape analysis, statistics, Riemannian geometry and Bayesian inference.

Nuclear phenotyping from spatial transcriptomics

Nuisance removed: position, rotation and size of each cell

Tangent principal component analysis pipeline: resample contour landmarks, normalize position and size, align to the mean shape, and compute principal components; example nuclei from an RNAi screen and reconstructed shapes along each component. — Tangent PCA on Kendall shape space, applied to nuclei from a published RNAi screen: example knockdowns, nuclei sorted along each component, and reconstructed shapes along the leading modes.

Spatial transcriptomics measures the shape and the gene expression of every cell in a tissue section. Conventional image representations mix a cell's intrinsic shape with its size, position and orientation. KSTITCH represents cell contours in Kendall shape space, which removes translation, rotation and scale, and uses tangent principal component analysis to find interpretable modes of shape variation that it links to gene-expression programs in the same cells.

KSTITCH recovers genes that perturb nuclear morphology in an RNAi screen. In melanoma, it identifies elongated, triangular fibroblasts near malignant cells that carry a myofibroblast-like transcriptional program, and it independently recovers the link between mesenchymal-like malignant states and cell area across patient cohorts.

Keratinocytes in a skin section colored by epidermal layer, a zoomed fluorescence image of segmented nuclei, and reconstructed nuclear shapes along the first four shape modes. — Keratinocyte layers in a Xenium skin section, segmented nuclei, and the leading nuclear shape modes.

With Vishaka Gopalan and Sridhar Hannenhalli (National Cancer Institute), Anuj Srivastava (Johns Hopkins) and P.-A. Absil (UCLouvain). KSTITCH preprint

Linking nuclear shape to sparse gene counts

Nuisance removed: sequencing depth

Decoded nuclear contours along three shared latent axes, from minus to plus 4.5 standard deviations, beside heatmaps of the genes that change most along each axis. — Walking the shared latent axes in skin keratinocytes: decoded nuclear shapes (left) and the genes that rise and fall along each axis (right).

Transcript counts in spatial data are sparse, and larger cells capture more transcripts, so a naive shape–expression correlation largely tracks sequencing depth. Count pCCA models the counts directly: each cell's latent gene composition, in log-ratio coordinates, is scaled by its depth and observed with negative binomial noise, while elastic nuclear shape and composition share a low-dimensional latent state. Fitted by Laplace EM, the shared axes are far less driven by depth. In skin, the leading axis runs from basal cells with elongated nuclei and anchoring genes to spinous cells with round nuclei, and it recovers the epidermal layers without ever seeing spatial coordinates. Shape adds the most when transcripts are scarce, as with small targeted gene panels.

Keratinocytes colored by epidermal layer beside the same cells colored by their posterior scores on the first two shared latent axes. — Epidermal layers (left, not used by the model) and the cells' posterior scores on the first two shared axes.

Work in progress, with Vishaka Gopalan, Sridhar Hannenhalli and Anuj Srivastava.

Tracking synaptic plasticity with physics-based generative models

Nuisance removed: nonlinear tissue deformation between imaging days

Eight imaging sessions of a two-photon volume: raw frames, the inferred synapse template, the estimated tissue deformation field, and the deformed posterior synapse intensities. — Synapses in a longitudinal volume (N_t) are noisy and move nonlinearly. The model fits a latent template of point sources (λ^tpl) moving under a smooth deformation field (u_t), tracking synapses (λ_t) directly from the raw data.

Synapses are densely packed, submicron structures that reorganize during learning and memory formation. Longitudinal two-photon imaging of fluorescently tagged synaptic receptors lets us watch this happen in the living brain, but the data are hard: low laser power means low signal-to-noise and heavy shot noise, the tissue deforms nonlinearly between days, synaptic fluorescence fluctuates, and the microscope's point spread function blurs neighboring synapses together.

I model synapses as point sources of varying brightness that move under a nonlinear tissue deformation. A single Bayesian posterior combines a diffeomorphic warp for tissue motion, a point spread function for the optics and a Poisson model for raw photon counts. Solving it simultaneously builds a probabilistic template of synapse locations, denoises and deconvolves the images, infers fluorescence intensities, registers the tissue across days, and gives confidence regions for every estimate. We demonstrate it on simulated data and on a 3D longitudinal recording of fluorescent synapses in a mouse imaged over two weeks. A complementary graph-based tracker, SynTrack, scales to tens of thousands of synapses and matches expert annotators.

With Michael I. Miller, Richard L. Huganir, Adam Charles, Austin Graves, Gabrielle Coste and Anuj Srivastava (Johns Hopkins). Bayesian tracking preprint · SynTrack preprint

Elastic shape models of human pose trajectories

Nuisance removed: camera rotation and speed of motion

Five skeletons showing the mean pose and traversals along five latent variables of the elastic shape VAE. — Modes of variation learned by the elastic shape VAE. z₁: shorter stride and stiffer limbs; z₂: left-arm variability; z₃: left arm and right knee; z₄: right arm; z₅: subtle right elbow. Black: mean shape; red and blue: ±3 traversals.

Standard variational autoencoders applied to skeleton sequences spend much of their capacity on nuisance factors such as camera orientation, subject size, viewpoint and execution speed rather than on the movement itself. The elastic shape VAE (ES-VAE) represents trajectories with transported square-root velocity fields on Kendall's shape manifold, which removes translation, rotation, scale and timing, isolating the underlying shape dynamics. It predicts clinical mobility scores and separates healthy from post-stroke gait, and on the NTU RGB+D action-recognition benchmark it outperforms standard VAEs and sequence-model baselines.

Earlier, with wearable sensors, I developed a shape-based functional index of upper-body motor function for children with Duchenne muscular dystrophy and spinal muscular atrophy that tracks clinical assessments.

With Arafat Rahman, Laura Barnes (University of Virginia) and Anuj Srivastava. ES-VAE preprint · PLOS ONE 2025

Optimization on manifolds

Many statistical models constrain parameters to probability simplices. Smooth reparameterizations turn these constrained problems into unconstrained Riemannian optimization, with applications to probabilistic tensor decomposition and functional data registration.

With P.-A. Absil (UCLouvain), Arafat Rahman and Anuj Srivastava. Preprint

Future directions

Two-photon microscopy, spatial omics and high-resolution motion capture are growing rapidly in scale, and with scale come more confounders. My goal is to build frameworks that combine imaging physics, probabilistic modeling and geometry-aware representations, summarizing variability in a few interpretable modes while staying invariant to nuisance variation.

Generating cellular phenotypes and gene expression jointly

Diagram of a shape–gene variational autoencoder: landmarks and genes are mapped to tangent vectors on shape space, encoded into a shared Gaussian latent space, and decoded back to shapes and genes. — A joint shape–gene variational autoencoder maps paired readouts of cell shape and gene expression to a shared latent space.

Building on shape-based phenotyping, we are developing a joint shape–gene variational autoencoder that generates gene-expression counts and contour variability from a shared latent state. Latent shape–gene dimensions inferred this way can serve as features that, combined with histopathology slides, help predict cancer risk.

Scalable inference for synapse tracking across brain regions

Inference by MCMC or direct optimization becomes prohibitive for larger cortical volumes and longer recordings. I plan to develop variational, amortized and structure-exploiting inference so that uncertainty-aware tracking scales to millions of synapses. That would open questions that are hard to study today: how synaptic populations reorganize during learning, whether plasticity events cluster within dendritic branches or cortical microcircuits, and whether subtle changes in synaptic turnover are early indicators of neurodegenerative disease.