Learning effective dynamics from data-driven stochastic systems

Multiscale stochastic dynamical systems have been widely adopted to a variety of scientific and engineering problems due to their capability of depicting complex phenomena in many real world applications. This work is devoted to investigating the effective dynamics for slow-fast stochastic dynamical systems. Given observation data on a short-term period satisfying some unknown slow-fast stochastic systems, we propose a novel algorithm including a neural network called Auto-SDE to learn invariant slow manifold. Our approach captures the evolutionary nature of a series of time-dependent autoencoder neural networks with the loss constructed from a discretized stochastic differential equation. Our algorithm is also validated to be accurate, stable and effective through numerical experiments under various evaluation metrics

Parameterizing and Aggregating Activation Functions in Deep Neural Networks

The nonlinear activation functions applied by each neuron in a neural network are essential for making neural networks powerful representational models. If these are omitted, even deep neural networks reduce to simple linear regression due to the fact that a linear combination of linear combinations is still a linear combination. In much of the existing literature on neural networks, just one or two activation functions are selected for the entire network, even though the use of heterogenous activation functions has been shown to produce superior results in some cases. Even less often employed are activation functions that can adapt their nonlinearities as network parameters along with standard weights and biases. This dissertation presents a collection of papers that advance the state of heterogenous and parameterized activation functions. Contributions of this dissertation include three novel parametric activation functions and applications of each, a study evaluating the utility of the parameters in parametric activation functions, an aggregated activation approach to modeling time-series data as an alternative to recurrent neural networks, and an improvement upon existing work that aggregates neuron inputs using product instead of sum

Collective estimation of multiple bivariate density functions with application to angular-sampling-based protein loop modeling

This article develops a method for simultaneous estimation of density functions for a collection of populations of protein backbone angle pairs using a data-driven, shared basis that is constructed by bivariate spline functions defined on a triangulation of the bivariate domain. The circular nature of angular data is taken into account by imposing appropriate smoothness constraints across boundaries of the triangles. Maximum penalized likelihood is used to fit the model and an alternating blockwise Newton-type algorithm is developed for computation. A simulation study shows that the collective estimation approach is statistically more efficient than estimating the densities individually. The proposed method was used to estimate neighbor-dependent distributions of protein backbone dihedral angles (i.e., Ramachandran distributions). The estimated distributions were applied to protein loop modeling, one of the most challenging open problems in protein structure prediction, by feeding them into an angular-sampling-based loop structure prediction framework. Our estimated distributions compared favorably to the Ramachandran distributions estimated by fitting a hierarchical Dirichlet process model; and in particular, our distributions showed significant improvements on the hard cases where existing methods do not work well

Surrogate - Assisted Optimisation -Based Verification & Validation

This thesis deals with the application of optimisation based Validation and Verification (V&V) analysis on aerospace vehicles in order to determine their worst case performance metrics. To this end, three aerospace models relating to satellite and launcher vehicles provided by European Space Agency (ESA) on various projects are utilised. As a means to quicken the process of optimisation based V&V analysis, surrogate models are developed using polynomial chaos method. Surro- gate models provide a quick way to ascertain the worst case directions as computation time required for evaluating them is very small. A sin- gle evaluation of a surrogate model takes less than a second. Another contribution of this thesis is the evaluation of operational safety margin metric with the help of surrogate models. Operational safety margin is a metric defined in the uncertain parameter space and is related to the distance between the nominal parameter value and the first instance of performance criteria violation. This metric can help to gauge the robustness of the controller but requires the evaluation of the model in the constraint function and hence could be computationally intensive. As surrogate models are computationally very cheap, they are utilised to rapidly compute the operational safety margin metric. But this metric focuses only on finding a safe region around the nominal parameter value and the possibility of other disjoint safe regions are not explored. In order to find other safe or failure regions in the param- eter space, the method of Bernstein expansion method is utilised on surrogate polynomial models to help characterise the uncertain param- eter space into safe and failure regions. Furthermore, Binomial failure analysis is used to assign failure probabilities to failure regions which might help the designer to determine if a re-design of the controller is required or not. The methodologies of optimisation based V&V, surrogate modelling, operational safety margin, Bernstein expansion method and risk assessment have been combined together to form the WCAT-II MATLAB toolbox

VI Workshop on Computational Data Analysis and Numerical Methods: Book of Abstracts

The VI Workshop on Computational Data Analysis and Numerical Methods (WCDANM) is going to be held on June 27-29, 2019, in the Department of Mathematics of the University of Beira Interior (UBI), Covilhã, Portugal and it is a unique opportunity to disseminate scientific research related to the areas of Mathematics in general, with particular relevance to the areas of Computational Data Analysis and Numerical Methods in theoretical and/or practical field, using new techniques, giving especial emphasis to applications in Medicine, Biology, Biotechnology, Engineering, Industry, Environmental Sciences, Finance, Insurance, Management and Administration. The meeting will provide a forum for discussion and debate of ideas with interest to the scientific community in general. With this meeting new scientific collaborations among colleagues, namely new collaborations in Masters and PhD projects are expected. The event is open to the entire scientific community (with or without communication/poster)

Design and implementation of machine learning techniques for modeling and managing battery energy storage systems

The fast technological evolution and industrialization that have interested the humankind since the fifties has caused a progressive and exponential increase of CO2 emissions and Earth temperature. Therefore, the research community and the political authorities have recognized the need of a deep technological revolution in both the transportation and the energy distribution systems to hinder climate changes. Thus, pure and hybrid electric powertrains, smart grids, and microgrids are key technologies for achieving the expected goals. Nevertheless, the development of the above mentioned technologies require very effective and performing Battery Energy Storage Systems (BESSs), and even more effective Battery Management Systems (BMSs). Considering the above background, this Ph.D. thesis has focused on the development of an innovative and advanced BMS that involves the use of machine learning techniques for improving the BESS effectiveness and efficiency. Great attention has been paid to the State of Charge (SoC) estimation problem, aiming at investigating solutions for achieving more accurate and reliable estimations. To this aim, the main contribution has concerned the development of accurate and flexible models of electrochemical cells. Three main modeling requirements have been pursued for ensuring accurate SoC estimations: insight on the cell physics, nonlinear approximation capability, and flexible system identification procedures. Thus, the research activity has aimed at fulfilling these requirements by developing and investigating three different modeling approaches, namely black, white, and gray box techniques. Extreme Learning Machines, Radial Basis Function Neural Networks, and Wavelet Neural Networks were considered among the black box models, but none of them were able to achieve satisfactory SoC estimation performances. The white box Equivalent Circuit Models (ECMs) have achieved better results, proving the benefit that the insight on the cell physics provides to the SoC estimation task. Nevertheless, it has appeared clear that the linearity of ECMs has reduced their effectiveness in the SoC task. Thus, the gray box Neural Networks Ensemble (NNE) and the white box Equivalent Neural Networks Circuit (ENNC) models have been developed aiming at exploiting the neural networks theory in order to achieve accurate models, ensuring at the same time very flexible system identification procedures together with nonlinear approximation capabilities. The performances of NNE and ENNC have been compelling. In particular, the white box ENNC has reached the most effective performances, achieving accurate SoC estimations, together with a simple architecture and a flexible system identification procedure. The outcome of this thesis makes it possible the development of an interesting scenario in which a suitable cloud framework provides remote assistance to several BMSs in order to adapt the managing algorithms to the aging of BESSs, even considering different and distinct applications

Energy management and shifting stability control for a novel dual input clutchless transmission system

© 2019 Elsevier Ltd A dual input clutchless transmission system based on automated manual transmission (AMT) structure is developed for pure electric vehicles. An energy management strategy (EMS) is proposed to determine the power distribution between two motors and the optimal gear state. A mathematical model is built to minimize the energy consumption of the motors at each instant based on the motor efficiency maps. However, the proposed EMS in line with other energy-oriented strategies often result in excessive gear shifts and compromised drivability. To avoid the undesired gear shift, a shifting stabilizer is built in the EMS objective function to improve the shift quality. Accordingly, to achieve a balance between the energy consumption and the drivability, a multi-objective optimization method is adopted to reduce the unnecessary shift events while minimizing energy consumption. Two driving cycles representing typical daily driving conditions are used to demonstrate the effectiveness of the proposed system in terms of energy efficiency and shifting stability

Bifurcation and Chaos in Fractional-Order Systems

This book presents a collection of seven technical papers on fractional-order complex systems, especially chaotic systems with hidden attractors and symmetries, in the research front of the field, which will be beneficial for scientific researchers, graduate students, and technical professionals to study and apply. It is also suitable for teaching lectures and for seminars to use as a reference on related topics

Analysis and Modeling of Quality Improvement on Clinical Fitness Landscapes

Widespread unexplained variations in clinical practices and patient outcomes, together with rapidly growing availability of data, suggest major opportunities for improving the quality of medical care. One way that healthcare practitioners try to do that is by participating in organized healthcare quality improvement collaboratives (QICs). In QICs, teams of practitioners from different hospitals exchange information on clinical practices, with the aim of improving health outcomes at their own institutions. However, what works in one hospital may not work in others with different local contexts, due to non-linear interactions among various demographics, treatments, and practices. I.e., the clinical landscape is a complex socio-technical system that is difficult to search. In this dissertation we develop methods for analysis and modeling of complex systems, and apply them to the problem of healthcare improvement. Searching clinical landscapes is a multi-objective dynamic problem, as hospitals simultaneously optimize for multiple patient outcomes. We first discuss a general method we developed for finding which changes in features may be associated with various changes in outcomes at different points in time with different delays in affect. This method correctly inferred interactions on synthetic data, however the complexity and incompleteness of the real hospital dataset available to us limited the usefulness of this approach. We then discuss an agent-based model (ABM) of QICs to show that teams comprising individuals from similar institutions outperform those from more diverse institutions, under nearly all conditions, and that this advantage increases with the complexity of the landscape and the level of noise in assessing performance. We present data from a network of real hospitals that provides encouraging evidence of a high degree of similarity in clinical practices among hospitals working together in QIC teams. Based on model outcomes, we propose a secure virtual collaboration system that would allow hospitals to efficiently identify potentially better practices in use at other institutions similar to theirs, without any institutions having to sacrifice the privacy of their own data. To model the search for quality improvement in clinical fitness landscapes, we need benchmark landscapes with tunable feature interactions. NK landscapes have been the classic benchmarks for modeling landscapes with epistatic interactions, but the ruggedness is only tunable in discrete jumps. Walsh polynomials are more finely tunable than NK landscapes, but are only defined on binary alphabets and, in general, have unknown global maximum and minimum. We define a different subset of interaction models that we dub as NM landscapes. NM landscapes are shown to have smoothly tunable ruggedness and difficulty and known location and value of global maxima. With additional constraints, we can also determine the location and value of the global minima. The proposed NM landscapes can be used with alphabets of any arity, from binary to real-valued, without changing the complexity of the landscape. NM landscapes are thus useful models for simulating clinical landscapes with binary or real decision variables and varying number of interactions. NM landscapes permit proper normalization of fitnesses so that search results can be fairly averaged over different random landscapes with the same parameters, and fairly compared between landscapes with different parameters. In future work we plan to use NM landscapes as benchmarks for testing various algorithms that can discover epistatic interactions in real world datasets