Neural Network Learning for PDEs with Oscillatory Solutions and Causal Operators

In this thesis, we focus on developing neural networks algorithms for scientific computing. First, we proposed a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. Several linearized learning schemes have been proposed for neural networks solving nonlinear Navier-Stokes equations. We also proposed a causality deep neural network (Causality-DeepONet) to learn the causal response of a physical system. An extension of the Causality-DeepONet to time-dependent PDE systems is also proposed. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and constructs a series of moderately-sized DNNs trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function’s specific high frequency range at the speed of learning for low frequency. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. To solve the stationary nonlinear Navier-Stokes(NS) equation with deep neural networks, we integrate linearization of the nonlinear convection term in the NS equation into the training process of multi-scale deep neural network (DNN) approximations of the NS solution. Four forms of linearization are considered. We solve highly oscillating stationary flows in complex domains utilizing the proposed linearized learning with multiscale neural networks. The theorem of universal approximations to nonlinear operators proposed by Chen et al. [11] is extended to operators with causalities, and the proposed Causality-DeepONet implements the physical causality in its framework. The proposed Causality-DeepONet considers causality (the state of the system at the current time is not affected by that of the future, but only by its current state and past history) and uses a convolution-type weight in its design. To demonstrate its effectiveness in handling the causal response of a physical system, the Causality-DeepONet is applied to learn the operator representing the response of a building due to earthquake ground accelerations. Finally, we proposed a deep neural network approximation to the evolution operator for time dependent PDE systems over long time period by recursively using one single neural network propagator, in the form of POD-DeepONet with built-in causality feature, for a small-time interval

Fourier Transforms

The 21st century ushered in a new era of technology that has been reshaping everyday life, simplifying outdated processes, and even giving rise to entirely new business sectors. Today, contemporary users of products and services expect more and more personalized products and services that can meet their unique needs. In that sense, it is necessary to further develop existing methods, adapt them to new applications, or even discover new methods. This book provides a thorough review of some methods that have an increasing impact on humanity today and that can solve different types of problems even in specific industries. Upgrading with Fourier Transformation gives a different meaning to these methods that support the development of new technologies and have a good projected acceleration in the future

Deep Tissue Light Delivery and Fluorescence Tomography with Applications in Optogenetic Neurostimulation

Study of the brain microcircuits using optogenetics is an active area of research. This method has a few advantages over the conventional electrical stimulation including the bi-directional control of neural activity, and more importantly, specificity in neuromodulation. The first step in all optogenetic experiments is to express certain light sensitive ion channels/pumps in the target cell population and then confirm the proper expression of these proteins before running any experiment. Fluorescent bio-markers, such as green fluorescent protein (GFP), have been used for this purpose and co-expressed in the same cell population. The fluorescent signal from such proteins provides a monitory mechanism to evaluate the expression of optogenetic opsins over time. The conventional method to confirm the success in gene delivery is to sacrifice the animal, retract and slice the brain tissue, and image the corresponding slices using a fluorescent microscope. Obviously, determining the level of expression over time without sacrificing the animal is highly desirable. Also, optogenetics can be combined with cell-type specific optical recording of neural activity for example by imaging the fluorescent signal of genetically encoded calcium indicators. One challenging step in any optogenetic experiment is delivering adequate amount of light to target areas for proper stimulation of light sensitive proteins. Delivering sufficient light density to a target area while minimizing the off-target stimulation requires a precise estimation of the light distribution in the tissue. Having a good estimation of the tissue optical properties is necessary for predicting the distribution of light in any turbid medium. The first objective of this project was the design and development of a high resolution optoelectronic device to extract optical properties of rats\u27 brain tissue (including the absorption coefficient, scattering coefficient, and anisotropy factor) for three different wavelengths: 405nm, 532nm and 635nm and three different cuts: transverse, sagittal, and coronal. The database of the extracted optical properties was linked to a 3D Monte Carlo simulation software to predict the light distribution for different light source configurations. This database was then used in the next phase of the project and in the development of a fluorescent tomography scanner. Based on the importance of the fluorescent imaging in optogenetics, another objective of this project was to design a fluorescence tomography system to confirm the expression of the light sensitive proteins and optically recording neural activity using calcium indicators none or minimally invasively. The method of fluorescence laminar optical tomography (FLOT) has been used successfully in imaging superficial areas up to 2mm deep inside a scattering medium with the spatial resolution of ~200µm. In this project, we developed a FLOT system which was specifically customized for in-vivo brain imaging experiments. While FLOT offers a relatively simple and non-expensive design for imaging superficial areas in the brain, still it has imaging depth limited to 2mm and its resolution drops as the imaging depth increases. To address this shortcoming, we worked on a complementary system based on the digital optical phase conjugation (DOPC) method which was shown previously that is capable of performing fluorescent tomography up to 4mm deep inside a biological tissue with lateral resolution of ~50 µm. This system also provides a non-invasive method to deliver light deep inside the brain tissue for neurostimulation applications which are not feasible using conventional techniques because of the high level of scattering in most tissue samples. In the developed DOPC system, the performance of the system in focusing light through and inside scattering mediums was quantified. We also showed how misalignments and imperfections of the optical components can immensely reduce the capability of a DOPC setup. Then, a systematic calibration algorithm was proposed and experimentally applied to our DOPC system to compensate main aberrations such as reference beam aberrations and also the backplane curvature of the spatial light modulator. In a highly scattering sample, the calibration algorithm achieved up to 8 fold increase in the PBR

Artificial confocal microscopy for deep label-free imaging

Widefield microscopy methods applied to optically thick specimens are faced with reduced contrast due to spatial crosstalk, in which the signal at each point is the result of a superposition from neighboring points that are simultaneously illuminated. In 1955, Marvin Minsky proposed confocal microscopy as a solution to this problem. Today, laser scanning confocal fluorescence microscopy is broadly used due to its high depth resolution and sensitivity, which come at the price of photobleaching, chemical, and photo-toxicity. Here, we present artificial confocal microscopy (ACM) to achieve confocal-level depth sectioning, sensitivity, and chemical specificity, on unlabeled specimens, nondestructively. We augmented a laser scanning confocal instrument with a quantitative phase imaging module, which provides optical pathlength maps of the specimen on the same field of view as the fluorescence channel. Using pairs of phase and fluorescence images, we trained a convolution neural network to translate the former into the latter. The training to infer a new tag is very practical as the input and ground truth data are intrinsically registered and the data acquisition is automated. Remarkably, the ACM images present significantly stronger depth sectioning than the input images, enabling us to recover confocal-like tomographic volumes of microspheres, hippocampal neurons in culture, and 3D liver cancer spheroids. By training on nucleus-specific tags, ACM allows for segmenting individual nuclei within dense spheroids for both cell counting and volume measurements. Furthermore, taking the estimated fluorescence volumes, as annotation for the phase data, we extracted dry mass information for individual nuclei. Finally, our results indicate that the network learning can be transferred between spheroids suspended in different media.Comment: 35 pages, 6 figure