Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Measurements, Models, Systems and Design

531 s.

Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites

This paper presents a spike-based model which employs neurons with functionally distinct dendritic compartments for classifying high dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron a capacity to perform a large number of input-output mappings. The model utilizes sparse synaptic connectivity; where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin enhancing learning algorithm inspired by the mechanism of structural plasticity in biological neurons. The learning algorithm groups correlated synaptic inputs on the same dendritic branch. Since the learning results in modified connection patterns, it can be incorporated into current event-based neuromorphic systems with little overhead. This work also presents a branch-specific spike-based version of this structural plasticity rule. The proposed model is evaluated on benchmark binary classification problems and its performance is compared against that achieved using Support Vector Machine (SVM) and Extreme Learning Machine (ELM) techniques. Our proposed method attains comparable performance while utilizing 10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio

DEVELOPMENT OF A CEREBELLAR MEAN FIELD MODEL: THE THEORETICAL FRAMEWORK, THE IMPLEMENTATION AND THE FIRST APPLICATION

Brain modeling constantly evolves to improve the accuracy of the simulated brain dynamics with the ambitious aim to build a digital twin of the brain. Specific models tuned on brain regions specific features empower the brain simulations introducing bottom-up physiology properties into data-driven simulators. Despite the cerebellum contains 80 % of the neurons and is deeply involved in a wide range of functions, from sensorimotor to cognitive ones, a specific cerebellar model is still missing. Furthermore, its quasi-crystalline multi-layer circuitry deeply differs from the cerebral cortical one, therefore is hard to imagine a unique general model suitable for the realistic simulation of both cerebellar and cerebral cortex. The present thesis tackles the challenge of developing a specific model for the cerebellum. Specifically, multi-neuron multi-layer mean field (MF) model of the cerebellar network, including Granule Cells, Golgi Cells, Molecular Layer Interneurons, and Purkinje Cells, was implemented, and validated against experimental data and the corresponding spiking neural network microcircuit model. The cerebellar MF model was built using a system of interdependent equations, where the single neuronal populations and topological parameters were captured by neuron-specific inter- dependent Transfer Functions. The model time resolution was optimized using Local Field Potentials recorded experimentally with high-density multielectrode array from acute mouse cerebellar slices. The present MF model satisfactorily captured the average discharge of different microcircuit neuronal populations in response to various input patterns and was able to predict the changes in Purkinje Cells firing patterns occurring in specific behavioral conditions: cortical plasticity mapping, which drives learning in associative tasks, and Molecular Layer Interneurons feed-forward inhibition, which controls Purkinje Cells activity patterns. The cerebellar multi-layer MF model thus provides a computationally efficient tool that will allow to investigate the causal relationship between microscopic neuronal properties and ensemble brain activity in health and pathological conditions. Furthermore, preliminary attempts to simulate a pathological cerebellum were done in the perspective of introducing our multi-layer cerebellar MF model in whole-brain simulators to realize patient-specific treatments, moving ahead towards personalized medicine. Two preliminary works assessed the relevant impact of the cerebellum on whole-brain dynamics and its role in modulating complex responses in causal connected cerebral regions, confirming that a specific model is required to further investigate the cerebellum-on- cerebrum influence. The framework presented in this thesis allows to develop a multi-layer MF model depicting the features of a specific brain region (e.g., cerebellum, basal ganglia), in order to define a general strategy to build up a pool of biology grounded MF models for computationally feasible simulations. Interconnected bottom-up MF models integrated in large-scale simulators would capture specific features of different brain regions, while the applications of a virtual brain would have a substantial impact on the reality ranging from the characterization of neurobiological processes, subject-specific preoperative plans, and development of neuro-prosthetic devices

Stochastic modeling and control of neural and small length scale dynamical systems

Recent advancements in experimental and computational techniques have created tremendous opportunities in the study of fundamental questions of science and engineering by taking the approach of stochastic modeling and control of dynamical systems. Examples include but are not limited to neural coding and emergence of behaviors in biological networks. Integrating optimal control strategies with stochastic dynamical models has ignited the development of new technologies in many emerging applications. In this direction, particular examples are brain-machine interfaces (BMIs), and systems to manipulate submicroscopic objects. The focus of this dissertation is to advance these technologies by developing optimal control strategies under various feedback scenarios and system uncertainties. Brain-machine interfaces (BMIs) establish direct communications between living brain tissue and external devices such as an artificial arm. By sensing and interpreting neuronal activity to actuate an external device, BMI-based neuroprostheses hold great promise in rehabilitating motor disabled subjects such as amputees. However, lack of the incorporation of sensory feedback, such as proprioception and tactile information, from the artificial arm back to the brain has greatly limited the widespread clinical deployment of these neuroprosthetic systems in rehabilitation. In the first part of the dissertation, we develop a systematic control-theoretic approach for a system-level rigorous analysis of BMIs under various feedback scenarios. The approach involves quantitative and qualitative analysis of single neuron and network models to the design of missing sensory feedback pathways in BMIs using optimal feedback control theory. As a part of our results, we show that the recovery of the natural performance of motor tasks in BMIs can be achieved by designing artificial sensory feedbacks in the proposed optimal control framework. The second part of the dissertation deals with developing stochastic optimal control strategies using limited feedback information for applications in neural and small length scale dynamical systems. The stochastic nature of these systems coupled with the limited feedback information has greatly restricted the direct applicability of existing control strategies in stabilizing these systems. Moreover, it has recently been recognized that the development of advanced control algorithms is essential to facilitate applications in these systems. We propose a novel broadcast stochastic optimal control strategy in a receding horizon framework to overcome existing limitations of traditional control designs. We apply this strategy to stabilize multi-agent systems and Brownian ensembles. As a part of our results, we show the optimal trapping of an ensemble of particles driven by Brownian motion in a minimum trapping region using the proposed framework

High Frequency trading via convolutional neural networks

L'objectiu d'aquest projecte és desenvolupar i entrenar a una CNN capaç de realitzar intercanvis borsaris dins un HFT. La metodologia seguida va consistir en el disseny de models per pronosticar característiques particulars del LOB. El punt de partida va ser provar mètodes tradicionals en finances per tal de predir els preus de futurs. Aquests mètodes van ser rebutjats a través de l'experimentació. Posteriorment, es va realitzar un estudi sobre arquitectures de ANN per construir models capaços de predir la direcció de l'preu mitjà. A més, es van dur a terme experiments en diferents models, provant diferents representacions de l'LOB. Els resultats van mostrar com, amb un disseny d'entrada i d'arquitectura adequats, una CNN supera lleugerament a un MLP en la predicció de la direcció de l'preu. Finalment, es van introduir noves etiquetes per detectar quan l'intercanvi borsària produeix beneficis. Tot seguit, es van realitzar experiments per predir aquestes etiquetes. En aquest cas, tots dos models van tenir resultats positius, però CNN no va superar a MLP.El objetivo de este proyecto es desarrollar y entrenar a una CNN capaz de realizar intercambios bursátiles en un HFT. La metodología seguida consistió en el diseño de modelos para pronosticar características particulares del LOB. El punto de partida fue probar métodos tradicionales en finanzas con el fin de predecir los precios de futuros. Estos métodos fueron rechazados a través de la experimentación. Posteriormente, se realizó un estudio sobre arquitecturas de ANN para construir modelos capaces de predecir la dirección del precio medio. Además, se llevaron a cabo experimentos en diferentes modelos, probando diferentes representaciones del LOB. Los resultados mostraron como, con un diseño de entrada y de arquitectura adecuado, una CNN supera ligeramente a un MLP en la predicción de la dirección del precio. Finalmente, se introdujeron nuevas etiquetas para detectar cuándo el intercambio bursátil produce beneficios. Acto seguido, se realizaron experimentos para predecir estas etiquetas. En ese caso, ambos modelos tuvieron resultados positivos, pero CNN no superó a MLP.The objective of this project is to develop and train a CNN capable to trade in a HFT. The methodology followed consisted in design models to forecast particular features of the LOB. The start point was to test traditional methods in finance with the purpose of predicting futures prices. These methods were rejected through experimentation. Subsequently, a study on ANN architectures was conducted to build models capable of predicting the direction of the mid price. Furthermore, experiments were carried out on different models, testing different representations of the LOB. The results showed how, with proper input and architecture design, a CNN slightly outperforms a MLP in predicting price direction. Finally, new labels were introduced to detect when the trade has benefits. Thereupon, experiments were conducted to predict these labels. In that case, both models had positive results but CNN did not outperform MLP. Subsequently, a study on ANN architectures was conducted to build models capable of predicting the direction of the mid price. Furthermore, experiments were carried out on different models, testing different representations of the LOB. The results showed how, with proper input and architecture design, a CNN slightly outperforms a MLP in predicting price direction. Finally, new labels were introduced to detect when the trade has benefits. Thereupon, experiments were conducted to predict these labels. In that case, both models had positive results but CNN did not outperform MLP.Outgoin