Programmable 3D snapshot microscopy with Fourier convolutional networks

3D snapshot microscopy enables fast volumetric imaging by capturing a 3D volume in a single 2D camera image, and has found a variety of biological applications such as whole brain imaging of fast neural activity in larval zebrafish. The optimal microscope design for this optical 3D-to-2D encoding is both sample- and task-dependent, with no general solution known. Highly programmable optical elements create new possibilities for sample-specific computational optimization of microscope parameters, e.g. tuning the collection of light for a given sample structure. We perform such optimization with deep learning, using a differentiable wave-optics simulation of light propagation through a programmable microscope and a neural network to reconstruct volumes from the microscope image. We introduce a class of global kernel Fourier convolutional neural networks which can efficiently decode information from multiple depths in the volume, globally encoded across a 3D snapshot image. We show that our proposed networks succeed in large field of view volume reconstruction and microscope parameter optimization where traditional networks fail. We also show that our networks outperform the state-of-the-art learned reconstruction algorithms for lensless computational photography.Comment: Make zebrafish Types A,B,C,D more clea

Functional Clustering Drives Encoding Improvement in a Developing Brain Network during Awake Visual Learning

Sensory experience drives dramatic structural and functional plasticity in developing neurons. However, for single-neuron plasticity to optimally improve whole-network encoding of sensory information, changes must be coordinated between neurons to ensure a full range of stimuli is efficiently represented. Using two-photon calcium imaging to monitor evoked activity in over 100 neurons simultaneously, we investigate network-level changes in the developing Xenopus laevis tectum during visual training with motion stimuli. Training causes stimulus-specific changes in neuronal responses and interactions, resulting in improved population encoding. This plasticity is spatially structured, increasing tuning curve similarity and interactions among nearby neurons, and decreasing interactions among distant neurons. Training does not improve encoding by single clusters of similarly responding neurons, but improves encoding across clusters, indicating coordinated plasticity across the network. NMDA receptor blockade prevents coordinated plasticity, reduces clustering, and abolishes whole-network encoding improvement. We conclude that NMDA receptors support experience-dependent network self-organization, allowing efficient population coding of a diverse range of stimuli.Canadian Institutes of Health Researc

Simultaneous imaging of structural and functional plasticity in the awake brain

During learning, and particularly during development, neurons in the brain undergo structural and functional changes that are intricately interrelated. This plasticity is guided by patterns of activity that encode information about the environment, allowing the brain to adapt to an organism's specific experiences. Here I developed optical methods and analysis tools to measure and analyze sensory-evoked activity patterns in the awake brain, and track how sensory information guides plasticity. Several different methods and their applications are presented. I described models and analysis tools for nonlinear decoding of somatic activity patterns in populations of neurons, and used them to track functional reorganization of neural circuits during training. I identified a group of ultrabright and stable organic dyes that enable two-photon imaging deep within living tissue, and applied them to produce a sensitive intracellular label for excitatory synapses. I developed a random access microscope capable of tracking activity at all excitatory synapses on a neuron simultaneously, enabling the first comprehensive measurements of a single neuron's dendritic input and firing output within the awake brain. I used this microscope to track neurons' comprehensive activity and structural changes across plasticity-inducing training, and identified rules by which somatic and dendritic activity direct the detailed growth patterns of dendrites, producing spatially clustered input patterns along neurons' dendritic arbor. Throughout this work, I've taken advantage of the Xenopus laevis model system to observe rapid experience-dependent plasticity in the awake, developing brain. These results demonstrate ways in which specific experiences direct the detailed connectivity of developing neural circuits.Medicine, Faculty ofGraduat

AI to the rescue of voltage imaging

In a recent issue of Nature Methods, Platisa et al. present an approach for long-term, in vivo population voltage imaging with single spike resolution across a local population of 100 neurons.(1) Key to this step for-ward was the combination of a customized high-speed two-photon microscope with an optimized, positive -going, genetically encoded voltage indicator and a tailored machine learning denoising algorithm.ISSN:2667-237

<i>In vivo</i> imaging of evoked network activity in the unanesthetized developing brain.

<p>(a) Experimental setup. Motion stimuli were presented to the left eye of awake, immobilized <i>Xenopus</i> tadpoles while imaging the right optic tectum. Neurons in the tectum (green circles) extend dendrites to receive visual input from retinal ganglion cells (red) of the contralateral eye. (b) Transmitted light image of a tadpole brain seen through the head. Green box, optic tectum. (c) Two-photon image of optical section corresponding to green box in (b). Tectum is loaded with OGB1-AM, a calcium-sensitive dye. Red box corresponds to the region of tectum monitored in our experiments. (d) Two-photon image of a patched neuron in awake tectum. (e) Simultaneous recording of somatic fluorescence (Δ<i>F</i>/<i>F</i>₀, top) and action potentials (green) in response to full field light stimuli of varying intensity, with actual (gray) and inferred (black) firing rates in the 5 s following each stimulus. (f) Expanded voltage trace for electrophysiological recording. Pink shading marks time of stimulus. The electrical transients bounding the stimulus period are clipped. Colored dots mark individual action potentials, which are magnified in the boxes at bottom.</p

Orientation and direction responses in optic tectum.

<p>(a,d) Maps of direction and orientation selectivity in naive <i>Xenopus</i> tectum obtained through rapid two-photon imaging and firing rate inference. Stimuli were dark bars moving over a light background for 600 ms in eight directions. Black circles mark neurons that responded significantly to stimuli. Colored arrows mark preferred directions (a) and orientations (d) of neurons showing stimulus specificity. Coronal optical section, rostrum to the left. Scale bar = 20 µm. (b,e) Tuning curves of a direction- (b) and an orientation- (e) selective neuron highlighted in (a,d). Error bars denote SEM. (c,f) Average temporal response of the two neurons to each stimulus direction. Colors match those in (b,e). Gray bar marks time of stimulus presentation. All measures calculated from <i>n</i> = 48 stimulus presentations for each of eight directions (1 h).</p

Schematic of receptive field and noise correlation plasticity for trained (red) and untrained (blue) stimuli.

<p>Tectal neurons are represented as circles, circle color marks preferred direction (red, down; blue, up), and dotted lines represent noise correlations. Training with down direction increases and clusters receptive fields oriented toward the trained stimuli and decreases long-distance noise correlations (dashed lines). Receptive fields preferring untrained stimuli (blue) are reduced, and noise correlations to these stimuli are increased on all spatial scales. Note that noise correlations can differ across stimuli and are not necessarily determined by neurons' preferred directions.</p

Tectal noise correlations influence network decoding.

<p>(a) Recorded responses of two neurons (black and grey) in the same tadpole to eight consecutive presentations of the same stimulus. Responses vary in amplitude around their means (dotted lines). These neurons were noise correlated: variations in amplitude were shared. (b) Distribution of measured pairwise noise correlations (black dotted e) taken over a 1-h stimulation period, and values expected if neurons were independent (gray). Noise correlations were more positive (<i>p</i><10⁻⁵, <i>t</i>-test) and more variable (<i>p</i><10⁻⁸; X² variance test) than chance. (c) Scatterplot of pairwise linear noise correlations measured in two consecutive 30-min periods. Consecutive noise correlation measurements are correlated (<i>r</i> = 0.41, <i>p</i><10⁻⁸; linear regression). (d) Distribution of decoding errors under independent and noise correlation decoding of actual response patterns (left) and with responses shuffled for each stimulus type to remove noise correlations (right). Data from seven tadpoles, 277 neurons (b,d), 384 stimulus presentations (c), 192 stimulus presentations each 30 min. Error bars denote SEM. *<i>p</i><0.05; **<i>p</i><0.01.</p

Training strengthens clustering of receptive fields and network correlations.

<p>(a–d) Tuning curve similarity (a,c) and mean noise correlation (b,d) of neuron pairs binned by spatial distance, during early (teal) and late (purple) epochs, in control (a,b) and MK801-treated (c,d) tadpoles. (e,f) Tuning curve similarity (e) and noise correlations (f) in tadpoles trained with four stimuli (0°–135°), binned by distance, in response to trained (orange) and untrained (yellow) stimuli. (f) Shaded area highlights the range of plots in (b,d). Noise correlations to untrained stimuli were significantly lower than in naive control animals (<i>p</i><10⁻⁵, two-way ANOVA) and those to trained stimuli were significantly higher than in naive controls (<i>p</i><10⁻⁵, two-way ANOVA). Error bars denote SEM. Control, <i>n</i> = 7 tadpoles (277 neurons), MK801, <i>n</i> = 7 tadpoles (255 neurons) (e,f) <i>n</i> = 3 tadpoles (152 neurons). *<i>p</i><0.05; **<i>p</i><0.01.</p

Training induces NMDAR-dependent improvement of whole-network encoding.

<p>(a) Time course of noise-correlation–based (red) and independent (blue) decoding performance. Light curves, improvement is blocked by MK-801. Bars denote early and late epochs. Decoding improvement is the decrease in decoding error relative to the independent decoder at the first timepoint. Both decoders improved from early to late epochs in control, but not MK-801–treated tadpoles (paired <i>t</i>-tests). (b) Decoding error of control (left, blue) and MK-801–treated (right, red) tadpoles over first hour of stimulation. Lighter shades denote decoding using the optimal independent decoder, darker shades mark noise correlation-based decoding. (c) Improvement, relative to the early epoch, of decoders trained on data from early (left two panels) or late (right two panels) epochs, used to decode early or late neuronal firing patterns. Performance decreased when decoding the epoch on which the decoder was not trained (center two panels; ANOVA). Asterisks in rightmost panel denote significant difference from corresponding value in leftmost panel. Error bars denote SEM. *<i>p</i><0.05; **<i>p</i><0.01.</p