Development of Robust and Dynamic Control Solutions for Energy Storage Enabled Hybrid AC/DC Microgrids

Development of Robust and Dynamic Control Solutions for Energy Storage Enabled Hybrid AC/DC Microgrid

Executive Attention, Action Selection and Attention-Based Learning in Neurally Controlled Autonomous Agents

I describe the design and implementation of an integrated neural architecture, modelled on human executive attention, which is used to control both automatic (reactive) and willed action selection in a simulated robot. The model, based upon Norman and Shallice's supervisory attention system, incorporates important features of human attentional control: selection of an intended task over a more salient automatic task; priming of future tasks that are anticipated; and appropriate levels of persistence of focus of attention. Recognising that attention-based learning, mediated by the limbic system, and the hippocampus in particular, plays an important role in adaptive learning, I extend the Norman and Shallice model, introducing an intrinsic, attention-based learning mechanism that enhances the automaticity of willed actions and reduces future need for attentional effort required for dealing with distractions. These enhanced features support a new level of attentional autonomy in the operation of the simulated robot. Some properties of the model are explored using lesion studies, leading to the identification of a correspondence between the behavioural pathologies of the simulated robot and those seen in human patients suffering dysfunction of executive attention

Convective heat transfer control in turbulent boundary layers

Mención Internacional en el título de doctorThe sustainable development of our society opens up concerns in several fields of engineering, including energy management, production, and the impact of our technology, being thermal management a common issue to be addressed. The investigation reported in this manuscript focuses on understanding, controlling and optimizing the physical processes involving convective heat transfer in turbulent wall-bounded flows. The content is divided into two main blocks, namely the investigation of classic open-loop active-control techniques to control heat transfer, and the technological development of machine-learning strategies to enhance the performance of flow control in the field of convective heat transfer. The first block focuses on actuator technology, applying dielectric-barrier discharge (DBD) plasma actuators and a pulsed slot jet in crossflow (JICF), respectively, to control the convective heat transfer in a turbulent boundary layer (TBL) over a flat plate. In the former, an array of DBD plasma actuators is employed to induce pairs of counter-rotating, streamwise-aligned vortices embedded in the TBL to reduce heat transfer downstream of the actuation. The whole three-dimensional mean flow field downstream of the plasma actuator is reconstructed from stereoscopic particle image velocimetry (PIV). Infrared thermography (IR) measurements coupled with a heated thin foil provide ensemble-averaged convective heat transfer distributions downstream of the actuators. The combination of the flow field and heat transfer measurements provides a complete picture of the fluid-dynamic interaction of plasma-induced flow with local turbulent transport effects. The plasmainduced streamwise vortices are stationary and confined across the spanwise direction due to the action of the plasma discharge. The opposing plasma discharge causes a mass- and momentum-flux deficit within the boundary layer, leading to a low-velocity region that grows in the streamwise direction and which is characterised by an increase in displacement and momentum thicknesses. This low-velocity ribbon travels downstream, promoting streak-alike patterns of reduction in the convective heat transfer distribution. Near the wall, the plasma-induced jets divert the main flow due to the DBD-actuator momentum injection and the suction on the surrounding fluid by the emerging jets. The stationarity of the plasma-induced vortices makes them persistent far downstream, reducing the convective heat transfer. Conversely, the target of the second paper in this first block is to enhance convective heat transfer rather than reduce it. A fully modulated, pulsed, slot JICF is used to perturb the TBL. The slot-jet actuator, flush-mounted and aligned in the spanwise direction, is controlled based on two design parameters, namely the duty cycle (DC) and the pulsation frequency (f). Heat transfer and flow-field measurements are performed to characterise the control performance using IR thermography and planar PIV, respectively. A parametric study on f and DC is carried out to assess their effect on the heat transfer distribution. The vorticity fields are reconstructed from the Proper Orthogonal Decomposition (POD) modes, retrieving phase information. The flow topology is considerably altered by the jet pulsation, even compared to the case of a steady jet. The results show that both the jet penetration in the streamwise direction and the overall Nusselt number increase with increasing DC. However, the frequency at which the Nusselt number is maximised is independent of the duty cycle. A wall-attached jet rises from the slot accompanied by a pair of counter-rotating vortices that promote flow entrainment and mixing. Eventually, a simplified model is proposed which decouples the effect of f and DC in the overall heat transfer enhancement, with a good agreement with experimental data. The cost of actuation is also quantified in terms of the amount of injected fluid during the actuation, leading to conclude that the lowest duty cycle is the most efficient for heat transfer enhancement among the tested set. The second block of the thesis splits into a comparative assessment of machine learning (ML) methods for active feedback flow control and an application of linear genetic algorithms to an experimental convective heat transfer enhancement problem. First, the comparative study is carried out numerically based on a well-established benchmark problem, the drag reduction of a two-dimensional Kármán vortex street past a circular cylinder at a low Reynolds number (Re = 100). The flow is manipulated with two blowing/suction actuators on the upper and lower side of a cylinder. The feedback employs several velocity sensors. Two probe configurations are evaluated: 5 and 11 velocity probes located at different points around the cylinder and in the wake. The control laws are optimized with Deep Reinforcement Learning (DRL) and Linear Genetic Programming Control (LGPC). Both methods successfully stabilize the vortex alley and effectively reduce drag while using small mass flow rates for the actuation. DRL features higher robustness with respect to variable initial conditions and noise contamination of the sensor data; on the other hand, LGPC can identify compact and interpretable control laws, which only use a subset of sensors, thus allowing reducing the system complexity with reasonably good results. The gained experience and knowledge of machine-learning methods motivated the last study enclosed in this thesis, which utilises linear genetic algorithm control (LGAC) to identify the best actuation parameters in an experimental application. The actuator is a set of six slot jets in crossflow aligned with the freestream. An open-loop optimal periodic forcing is defined by the carrier frequency (f), the duty cycle (DC) and the phase between actuators (ϕ) as control parameters. The control laws are optimised with respect to the unperturbed TBL and the steady-jet actuation. The cost function includes wall convective heat transfer and the cost of the actuation, thus leading to a multi-objective optimisation problem. Surprisingly, the LGAC algorithm converges to the same frequency and duty cycle for all the actuators. This frequency is equivalent to the optimal frequency reported in the second study of the first block of this thesis. The performance of the controller is characterised by IR thermography and PIV measurements. The action of the jets considerably alters the flow topology compared to the steady-jet actuation, yielding a slightly asymmetric flow field. The phase difference between multiple jet actuation has shown to be very relevant and the main driver of flow asymmetry. A POD analysis concludes the shedding phenomena characterising the steady-jet actuation, while the optimised controller exhibits an elongated large-scale structure just downstream of the actuator. The investigation carried out in this thesis sheds some light on the application of different flow control strategies to the field of convective heat transfer. From the utilisation of plasma actuators and a single jet in cross flow to the development of sophisticated control logic, the results point to the exceptional potential of machine learning control in unravelling unexplored controllers within the actuation space. Ultimately, this work demonstrates the viability of employing sophisticated measurement techniques together with advanced algorithms in an experimental investigation, paving the way towards more complex applications involving feedback information.El desarrollo sostenible de nuestra sociedad abre preocupaciones en varios campos de la ingeniería, incluyendo la gestión de la energía, la producción y el impacto de nuestra tecnología, siendo la gestión térmica un tema común a tratar. La investigación que se presenta en este manuscrito se centra en la comprensión, el control y la optimización de los procesos físicos que implican la transferencia de calor por convección en flujos turbulentos de pared. El contenido se divide en dos bloques principales: la investigación de las técnicas clásicas de control activo de lazo abierto para controlar la transferencia de calor, y el desarrollo tecnológico de estrategias de aprendizaje automático para mejorar el rendimiento del control del flujo en el campo de la transferencia de calor por convección. El primer bloque se centra en la tecnología de los actuadores, aplicando actuadores de plasma de descarga de barrera dieléctrica (dielectric barrier dicharge, DBD) y un chorro con forma de ranura pulsado en flujo cruzado (jet in crossflow, JICF), respectivamente, para controlar la transferencia de calor por convección en una capa límite turbulenta (turbulent boundary layer, TBL) sobre una placa plana. En el primero, se emplea un conjunto de actuadores de plasma DBD para inducir pares de vórtices contra-rotativos, alineados con la corriente e incrustados en la TBL para reducir la transferencia de calor aguas abajo de la actuación. El campo de flujo medio tridimensional completo aguas abajo del actuador de plasma se reconstruye a partir de la velocimetría de imagen de partículas estereoscópica (particle image velocimetry, PIV). Las mediciones de termografía infrarroja (IR) junto a una fina lámina calentada proporcionan distribuciones de transferencia de calor convectiva promediadas aguas abajo de los actuadores. La combinación de las mediciones del campo de flujo y de la transferencia de calor proporciona una imagen completa de la interacción fluido-dinámica del flujo inducido por el plasma con los efectos locales de transporte turbulento. Los vórtices en el sentido de la corriente inducidos por plasma son estacionarios y están confinados transversalmente debido a la acción de la descarga de plasma. La descarga de plasma en oposición causa un déficit de flujo de masa y de momento dentro de la capa límite, lo que conduce a una región de baja velocidad que crece en la dirección de la corriente y que se caracteriza por un aumento de los espesores de desplazamiento y de momento. Esta zona de baja velocidad se desplaza corriente abajo, promoviendo patrones de reducción similares a rayas en los que se reduce la transferencia de calor por convección. Cerca de la pared, los chorros inducidos por el plasma desvían el flujo principal debido a la inyección de momento del actuador DBD y a la succión sobre el fluido circundante por parte de los chorros emergentes. La estacionariedad de los vórtices inducidos por el plasma los hace persistentes aguas abajo, reduciendo la transferencia de calor por convección. Por el contrario, el objetivo del segundo trabajo de este primer bloque es mejorar la transferencia de calor por convección en lugar de reducirla. Se utiliza un JICF, pulsado y con forma de ranura, totalmente modulado para perturbar la TBL. El actuador de chorro, montado a ras y alineado en la dirección transversal, se controla en base a dos parámetros de diseño, a saber, el ciclo de trabajo (DC) y la frecuencia de pulsación (f). Se realizan mediciones de la transferencia de calor y del campo de flujo para caracterizar el rendimiento del control mediante termografía IR y PIV planar, respectivamente. Se lleva a cabo un estudio paramétrico de f y DC para evaluar su efecto en la distribución de la transferencia de calor. Los campos de vorticidad se reconstruyen a partir de los modos de descomposición ortogonal adecuada (POD), recuperando la información de fase. La topología del flujo se ve considerablemente alterada por la pulsación del chorro, incluso en comparación con el caso de un chorro estacionario. Los resultados muestran que tanto la penetración del chorro en la dirección de la corriente como el número Nusselt global aumentan con el incremento de DC. Sin embargo, la frecuencia a la que se maximiza el número Nusselt es independiente del ciclo de trabajo. Un chorro adherido a la pared sale de la ranura acompañado de un par de vórtices contrarrotantes que promueven el arrastre y la mezcla del flujo. Finalmente, se propone un modelo simplificado que desacopla el efecto de f y DC en la mejora global de la transferencia de calor, con un buen acuerdo con los datos experimentales. También se cuantifica el coste de la actuación en términos de la cantidad de fluido inyectado durante la actuación, llegando a la conclusión de que el ciclo de trabajo más bajo es el más eficiente para la mejora de la transferencia de calor entre el conjunto probado. El segundo bloque de la tesis se divide en una evaluación comparativa de los métodos de aprendizaje automático (machine learning, ML) para el control activo del flujo por retroalimentación y una aplicación de algoritmos genéticos lineales a un problema experimental de mejora de la transferencia de calor por convección. En primer lugar, el estudio comparativo se realiza numéricamente a partir de un problema de referencia bien establecido: la reducción de la resistencia aerodinámica de una calle de vórtices de Kármán bidimensional tras un cilindro circular a un número de Reynolds bajo (Re = 100). El flujo se manipula con dos actuadores de soplado/succión en la parte superior e inferior de un cilindro. La retroalimentación emplea varios sensores de velocidad. Se evalúan dos configuraciones de sondas: 5 y 11 sondas de velocidad situadas en diferentes puntos alrededor del cilindro y en la estela. Las leyes de control se optimizan con el aprendizaje profundo por refuerzo (Deep Reinforcement Learning, DRL) y el control por programación genética lineal (Linear Genetic Programming Control, LGPC). Ambos métodos estabilizan con éxito la calle de vórtices y reducen de manera efectiva la resistencia al tiempo que usan caudales másicos pequeños para la actuación. El DRL se caracteriza por una mayor robustez con respecto a la variación de la condición inicial y a la contaminación por ruido de los datos de los sensores; por otro lado, el LGPC puede identificar leyes de control compactas e interpretables, que sólo utilizan un subconjunto de sensores, lo que permite reducir la complejidad del sistema con resultados razonablemente buenos. La experiencia adquirida y el conocimiento de los métodos de aprendizaje automático motivaron el último estudio incluido en esta tesis, que utiliza el control por algoritmo genético lineal (Linear Genetic Algorithm Control, LGAC) para identificar los mejores parámetros de actuación en una aplicación experimental. El actuador es un conjunto de seis chorros con forma de ranura en flujo cruzado y alineados con la corriente principal. Se define una ley de forzado periódica en lazo abierto mediante la frecuencia portadora (f), el ciclo de trabajo (DC) y la fase entre actuadores (ϕ) como parámetros de control. Las leyes de control se optimizan con respecto a la TBL no perturbada y la actuación de chorro constante. La función de coste incluye la transferencia de calor por convección de la pared y el coste de la actuación, lo que da lugar a un problema de optimización multiobjetivo. Sorprendentemente, el algoritmo LGAC converge a la misma frecuencia y ciclo de trabajo para todos los actuadores. Esta frecuencia es equivalente a la frecuencia óptima reportada en el segundo estudio del primer bloque de esta tesis. El rendimiento del controlador se caracteriza mediante termografía IR y mediciones PIV. La acción de los chorros altera considerablemente la topología del flujo en comparación con la actuación de los chorros constantes, dando lugar a un campo de flujo ligeramente asimétrico. La diferencia de fase entre la actuación de múltiples chorros ha demostrado ser muy relevante y el principal impulsor de la asimetría del flujo. Un análisis POD concluye los fenómenos de desprendimiento de vórtices que caracterizan la actuación de chorro constante, mientras que el controlador optimizado muestra una estructura alargada a gran escala justo aguas abajo del actuador. La investigación llevada a cabo en esta tesis arroja algo de luz sobre la aplicación de diferentes estrategias de control de flujo en el campo de la transferencia de calor por convección. Desde la utilización de actuadores de plasma y un único chorro en flujo cruzado hasta el desarrollo de una sofisticada lógica de control, los resultados apuntan al excepcional potencial del control por aprendizaje automático para desentrañar controladores inexplorados dentro del espacio de actuación. En última instancia, este trabajo demuestra la viabilidad de emplear sofisticadas técnicas de medición junto con algoritmos avanzados en una investigación experimental, allanando el camino hacia aplicaciones más complejas que implican información de retroalimentación.The work enclosed in this thesis has been partially supported by the Universidad Carlos III de Madrid through a PIPF scholarship awarded on a competitive basis, and by the following research projects: ARTURO (Active contRol of Turbulence for sUstainable aiRcraft propulsiOn), ref. PID2019-109717RB-I00/AEI/10.13039/501100011033, funded by the Spanish State Research Agency (SRA); the 2020 Leonardo Grant for Researchers and Cultural Creators AEROMATIC (Active flow control of aerodynamic flows with machine learning), funded by the BBVA Foundation with grant number IN[20]_ING_ING_0163; and GloWing Starting Grant, funded by the European Research Council (ERC), under grant agreement ERC-2018.StG-803082.Programa de Doctorado en Mecánica de Fluidos por la Universidad Carlos III de Madrid; la Universidad de Jaén; la Universidad de Zaragoza; la Universidad Nacional de Educación a Distancia; la Universidad Politécnica de Madrid y la Universidad Rovira i VirgiliPresidente: Octavio Armas Vergel.- Secretario: Manuel García-Villalba Navaridas.- Vocal: Gioacchino Cafier

On the intrinsic control properties of muscle and relexes: exploring the interaction between neural and musculoskeletal dynamics in the framework of the equilbrium-point hypothesis

The aim of this thesis is to examine the relationship between the intrinsic dynamics of the body and its neural control. Specifically, it investigates the influence of musculoskeletal properties on the control signals needed for simple goal-directed movements in the framework of the equilibriumpoint (EP) hypothesis. To this end, muscle models of varying complexity are studied in isolation and when coupled to feedback laws derived from the EP hypothesis. It is demonstrated that the dynamical landscape formed by non-linear musculoskeletal models features a stable attractor in joint space whose properties, such as position, stiffness and viscosity, can be controlled through differential- and co-activation of antagonistic muscles. The emergence of this attractor creates a new level of control that reduces the system’s degrees of freedom and thus constitutes a low-level motor synergy. It is described how the properties of this stable equilibrium, as well as transient movement dynamics, depend on the various modelling assumptions underlying the muscle model. The EP hypothesis is then tested on a chosen musculoskeletal model by using an optimal feedback control approach: genetic algorithm optimisation is used to identify feedback gains that produce smooth single- and multijoint movements of varying amplitude and duration. The importance of different feedback components is studied for reproducing invariants observed in natural movement kinematics. The resulting controllers are demonstrated to cope with a plausible range of reflex delays, predict the use of velocity-error feedback for the fastest movements, and suggest that experimentally observed triphasic muscle bursts are an emergent feature rather than centrally planned. Also, control schemes which allow for simultaneous control of movement duration and distance are identified. Lastly, it is shown that the generic formulation of the EP hypothesis fails to account for the interaction torques arising in multijoint movements. Extensions are proposed which address this shortcoming while maintaining its two basic assumptions: control signals in positional rather than force-based frames of reference; and the primacy of control properties intrinsic to the body over internal models. It is concluded that the EP hypothesis cannot be rejected for single- or multijoint reaching movements based on claims that predicted movement kinematics are unrealistic

Dynamics Days Latin America and the Caribbean 2018

This book contains various works presented at the Dynamics Days Latin America and the Caribbean (DDays LAC) 2018. Since its beginnings, a key goal of the DDays LAC has been to promote cross-fertilization of ideas from different areas within nonlinear dynamics. On this occasion, the contributions range from experimental to theoretical research, including (but not limited to) chaos, control theory, synchronization, statistical physics, stochastic processes, complex systems and networks, nonlinear time-series analysis, computational methods, fluid dynamics, nonlinear waves, pattern formation, population dynamics, ecological modeling, neural dynamics, and systems biology. The interested reader will find this book to be a useful reference in identifying ground-breaking problems in Physics, Mathematics, Engineering, and Interdisciplinary Sciences, with innovative models and methods that provide insightful solutions. This book is a must-read for anyone looking for new developments of Applied Mathematics and Physics in connection with complex systems, synchronization, neural dynamics, fluid dynamics, ecological networks, and epidemics

State-of-the-art MR imaging in the work-up of primary hepatocellular tumors

Magnetic resonance (MR) imaging is an imaging modality that has evolved rapidly in the past two decades. The development of advanced hardware and new sophisticated pulse sequences have allowed faster imaging, with increased temporal and spatial resolution. This has resulted in the development and implementation of new acquisition techniques that facilitate improved visualisation of neoplastic processes. In addition, faster sequences enable multiphasic dynamic imaging after intravenous administration of contrast material, which results in better tumor characterisation and improved diagnostic confidence by the reading radiologist. The radiol

Neuronal oscillations, information dynamics, and behaviour: an evolutionary robotics study

Oscillatory neural activity is closely related to cognition and behaviour, with synchronisation mechanisms playing a key role in the integration and functional organization of different cortical areas. Nevertheless, its informational content and relationship with behaviour - and hence cognition - are still to be fully understood. This thesis is concerned with better understanding the role of neuronal oscillations and information dynamics towards the generation of embodied cognitive behaviours and with investigating the efficacy of such systems as practical robot controllers. To this end, we develop a novel model based on the Kuramoto model of coupled phase oscillators and perform three minimally cognitive evolutionary robotics experiments. The analyses focus both on a behavioural level description, investigating the robot’s trajectories, and on a mechanism level description, exploring the variables’ dynamics and the information transfer properties within and between the agent’s body and the environment. The first experiment demonstrates that in an active categorical perception task under normal and inverted vision, networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally, and to adapt to different behavioural conditions. The second experiment relates assembly constitution and phase reorganisation dynamics to performance in supervised and unsupervised learning tasks. We demonstrate that assembly dynamics facilitate the evolutionary process, can account for varying degrees of stimuli modulation of the sensorimotor interactions, and can contribute to solving different tasks leaving aside other plasticity mechanisms. The third experiment explores an associative learning task considering a more realistic connectivity pattern between neurons. We demonstrate that networks with travelling waves as a default solution perform poorly compared to networks that are normally synchronised in the absence of stimuli. Overall, this thesis shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, produce an asymmetric flow of information and can generate minimally cognitive embodied behaviours