Geometric deep learning reveals the spatiotemporal fingerprint of microscopic motion

The characterization of dynamical processes in living systems provides important clues for their mechanistic interpretation and link to biological functions. Thanks to recent advances in microscopy techniques, it is now possible to routinely record the motion of cells, organelles, and individual molecules at multiple spatiotemporal scales in physiological conditions. However, the automated analysis of dynamics occurring in crowded and complex environments still lags behind the acquisition of microscopic image sequences. Here, we present a framework based on geometric deep learning that achieves the accurate estimation of dynamical properties in various biologically-relevant scenarios. This deep-learning approach relies on a graph neural network enhanced by attention-based components. By processing object features with geometric priors, the network is capable of performing multiple tasks, from linking coordinates into trajectories to inferring local and global dynamic properties. We demonstrate the flexibility and reliability of this approach by applying it to real and simulated data corresponding to a broad range of biological experiments.Comment: 17 pages, 5 figure, 2 supplementary figure

Quantitative evaluation of methods to analyze motion changes in single-particle experiments

The analysis of live-cell single-molecule imaging experiments can reveal valuable information about the heterogeneity of transport processes and interactions between cell components. These characteristics are seen as motion changes in the particle trajectories. Despite the existence of multiple approaches to carry out this type of analysis, no objective assessment of these methods has been performed so far. Here, we have designed a competition to characterize and rank the performance of these methods when analyzing the dynamic behavior of single molecules. To run this competition, we have implemented a software library to simulate realistic data corresponding to widespread diffusion and interaction models, both in the form of trajectories and videos obtained in typical experimental conditions. The competition will constitute the first assessment of these methods, provide insights into the current limits of the field, foster the development of new approaches, and guide researchers to identify optimal tools for analyzing their experiments.Comment: 19 pages, 4 figure, 2 tables. Stage 1 registered report, accepted in principle in Nature Communications (https://springernature.figshare.com/articles/journal_contribution/Quantitative_evaluation_of_methods_to_analyze_motion_changes_in_single-particle_experiments_Registered_Report_Stage_1_Protocol_/24771687

Single-shot self-supervised object detection in microscopy

Object detection is a fundamental task in digital microscopy, where machine learning has made great strides in overcoming the limitations of classical approaches. The training of state-of-the-art machine-learning methods almost universally relies on vast amounts of labeled experimental data or the ability to numerically simulate realistic datasets. However, experimental data are often challenging to label and cannot be easily reproduced numerically. Here, we propose a deep-learning method, named LodeSTAR (Localization and detection from Symmetries, Translations And Rotations), that learns to detect microscopic objects with sub-pixel accuracy from a single unlabeled experimental image by exploiting the inherent roto-translational symmetries of this task. We demonstrate that LodeSTAR outperforms traditional methods in terms of accuracy, also when analyzing challenging experimental data containing densely packed cells or noisy backgrounds. Furthermore, by exploiting additional symmetries we show that LodeSTAR can measure other properties, e.g., vertical position and polarizability in holographic microscopy

Deep Learning Enhanced Optical Methods for Single-Plankton Studies

Among Earth’s earliest life forms, cyanobacteria reshaped the planet by oxygenating the atmosphere during the Great Oxidation Event 2.4 billion years ago. This process, which led to ozone formation and UV protection, paved the way for more complex photosynthetic organisms—phytoplankton, the eukaryotic descendants of cyanobacteria. Today, phytoplankton drive the global carbon cycle, producing 50–80% of Earth’s oxygen and fueling the marine food web. Microzooplankton consume nearly two-thirds of the organic carbon generated, yet despite their ecological significance, tracking biomass flow at the single-cell level remains a major challenge. This thesis presents novel methodologies that integrate advanced optical techniques, deep learning, and simulated datasets to analyze microplankton dynamics with unprecedented resolution. A key contribution is a deep-learning-enhanced holographic microscopy approach that quantifies microplankton biomass at the single-cell level while simultaneously capturing their three-dimensional swimming behavior. This method overcomes computational bottlenecks in traditional holography, enabling high-throughput analysis across diverse species and size ranges. Expanding on this, I demonstrate its application in mixed-species experiments to examine feeding interactions between phytoplankton and microzooplankton, capturing biomass transfer and behavioral shifts during predation. Beyond direct imaging, this thesis leverages synthetic data to advance microscopy-based research. Neural networks trained on simulated microscopy datasets are used to detect, segment, and classify plankton species while reconstructing motion dynamics. To showcase the versatility of this approach, I present its application in a non-biological setting—detecting bubble-propelled artificial micromotors within complex experimental backgrounds. In addition to object detection, these methods also enable motion characterization of microscopic entities. To demonstrate this, I introduce synthetic microscopy videos that model microscopic organisms undergoing various anomalous diffusion behaviors. This framework is then used to develop a method that extracts motion characteristics without explicit trajectory linking, broadening its applications beyond plankton ecology. Finally, I investigate how zooplankton—key players in the marine food web—respond to ocean wave-induced light patterns using an LED matrix. The results suggest that zooplankton use steady light sources, such as celestial objects, to ascend more rapidly during favorable low-turbulent conditions, offering new insights into their migratory strategies. Collectively, this thesis bridges marine ecology, microscopy, artificial intelligence, and biophysics to provide new tools for exploring the unseen dynamics that shape our planet

Diffusion models for super-resolution microscopy: a tutorial

Diffusion models have emerged as a prominent technique in generative modeling with neural networks, making their mark in tasks like text-to-image translation and super-resolution. In this tutorial, we provide a comprehensive guide to build denoising diffusion probabilistic models from scratch, with a specific focus on transforming low-resolution microscopy images into their corresponding high-resolution versions in the context of super-resolution microscopy. We provide the necessary theoretical background, the essential mathematical derivations, and a detailed Python code implementation using PyTorch. We discuss the metrics to quantitatively evaluate the model, illustrate the model performance at different noise levels of the input low-resolution images, and briefly discuss how to adapt the tutorial for other applications. The code provided in this tutorial is also available as a Python notebook in the supplementary information

Bubble-propelled micromotors for ammonia generation

Micromotors have emerged as promising tools for environmental remediation, thanks to their ability to autonomously navigate and perform specific tasks at the microscale. In this study, we present the development of MnO2 tubular micromotors modified with laccase for enhanced oxidation of organic pollutants by providing an additional oxidative catalytic pathway for pollutant removal. These modified micromotors exhibit efficient ammonia generation through the catalytic decomposition of urea, suggesting their potential application in the field of green energy generation. Compared to bare micromotors, the MnO2 micromotors modified with laccase exhibit a 20% increase in rhodamine B degradation. Moreover, the generation of ammonia increased from 2 to 31 ppm in only 15 min, evidencing their high catalytic activity. To enable precise tracking of the micromotors and measurement of their speed, a deep-learning-based tracking system was developed. Overall, this work expands the potential applicability of bio-catalytic tubular micromotors in the energy field. Here, we introduce self-propelled biocatalytic micromotors for simultaneous organic pollutant removal and green energy generation. The study demonstrates remarkable results, showcasing the potential to generate ammonia from wastewater in short time

Roadmap on Deep Learning for Microscopy

A multi-authored roadmap paper on deep learning in microscopy. Our contribution a section on deep learning in microscopy of plankto

Harnessing nonlinear frequency upconversion of Talbot effect with flexible Talbot lengths

We report on a simple experimental scheme demonstrating nonlinear frequency upconversion of the Talbot effect with controllable Talbot lengths at high conversion efficiency. Using a microlens array (MLA) as an array illuminator at 1064 nm onto a 1.2 -mm -thick BiBO crystal, we have observed the second harmonic Talbot effect in green at 532 nm with a Talbot length twice that of the pump Talbot length. However, the Talbot length is constant for fixed parameters of the periodic object and the laser wavelength. With the formulation of a suitable theoretical framework, we have implemented a generic experimental scheme based on the Fourier transformation technique to independently control the Talbot lengths of the MLA in both the pump and the second harmonic, overcoming the stringent dependence of MLA parameters on the self-images. Deploying the current technique, we have been able to tune the Talbot lengths from z T = 26 cm to z T = 62.4 cm in the pump and z T = 12.4 cm to z T = 30.8 cm in the second harmonic, respectively. The single pass conversion efficiency of the Talbot images is 2.91% W - 1 , an enhancement of a factor of 10 6 as compared to the previous reports. This generic experimental scheme can be used to generate long-range self-images of periodic structures and also to program desired Talbot planes at required positions at both pump and upconverted frequency to avoid any mechanical constraints of experiments

AnDiChallenge/andi_datasets: AnDi Challenge 2

<p>This release contains all the changes developed for the AnDi Challenge 2. Among others: new tutorials, the scoring program for Codalab and other few enhancements</p&gt