Modified memoryless spectral-scaling Broyden family on Riemannian manifolds

This paper presents modified memoryless quasi-Newton methods based on the spectral-scaling Broyden family on Riemannian manifolds. The method involves adding one parameter to the search direction of the memoryless self-scaling Broyden family on the manifold. Moreover, it uses a general map instead of vector transport. This idea has already been proposed within a general framework of Riemannian conjugate gradient methods where one can use vector transport, scaled vector transport, or an inverse retraction. We show that the search direction satisfies the sufficient descent condition under some assumptions on the parameters. In addition, we show global convergence of the proposed method under the Wolfe conditions. We numerically compare it with existing methods, including Riemannian conjugate gradient methods and the memoryless spectral-scaling Broyden family. The numerical results indicate that the proposed method with the BFGS formula is suitable for solving an off-diagonal cost function minimization problem on an oblique manifold.Comment: 20 pages, 8 figure

Modelling zinc oxide thin-film growth

Photovoltaics have a significant role in the solution of energy supply and energy security. Research on photovoltaic devices and their production processes has been carried out for decades. The transparent conducting oxide layer, in the photovoltaic solar cell, composed of aluminium doped zinc oxide, is produced through deposition techniques. By modelling these depositions using classical molecular dynamics, a better understanding on the short term kinetics occurring on the growing surface has been achieved. Compared to the molecular dynamics, the employment of the adaptive kinetic Monte Carlo method enabled such surface growth dynamics simulation to reach much longer time scale. Parallelised transition searching was carried out in an on-the-fly manner without lattice approximation or predefined events table. The simulation techniques allowed deposition conditions to be easily changed, such as deposition energy, deposition rate, substrate temperature, plasma pressure, etc. Therefore, in this project three main deposition techniques were modelled including evaporation (thermal and assisted electron beam), reactive magnetron sputtering and pulsed laser depositions. ZnO as a covalent compound with many uses in semiconductors was investigated in its most energy favourable wurtzite configuration. The O-terminated surface was used as the substrate for the growth simulation. Evaporation deposition at room temperature (300 K) with a stoichiometric distribution of deposition species produced incomplete new layers. Holes were observed existing for long times in each layer. Also, stacking faults were formed during the low-energy (1 eV) growth through evaporation. The reactive sputtering depositions were more capable of getting rid of these holes structures and diminished these stacking faults through high energy bombardments but could also break these desirable crystalline structure during the growth. However, single deposition results with high energies showed that the ZnO lattice presented good capacity of self-healing after energetic impacts. Additionally, such self-healing effects were seen for substrate surface during thin film growth by the sputtering depositions. These facts shed some light on that the sputtering technique is the method of choice for ZnO thin film depositions during industrial production. Simulation results of pulsed laser deposition with separated Zn and O species showed the thin films were grown in porous structures as the O-terminated surface could be severely damaged by Zn atoms during the very short pulse window (10 microseconds). An important growth mechanism with ZnO dimer deposited on the O-terminated polar surface was the coupling of these single ZnO dimers, forming highly mobile strings along the surface and thus quenching its dipole moments, whilst the isolated single ZnO dimers were hardly of this mobility. Such strings were the building blocks for the fabrication occurring on the surface resulting in new layers. Last but not least, a reactive force field for modelling Al doped ZnO was fitted. DFT calculations showed that the Al atoms on the surface were likely to replace Zn atoms in their lattice sites for more energy favourable structures. Al on the ZnO surfaces, structures with Al in the bulk as well as configurations with Al interstitials were used to train the force field to reproduce favourable surface binding sites, cohesive energies and lattice dimensions. The combination scheme of MD and the AKMC allowed simulation work to reach over experimentally realistic time scale. Therefore, crucial mechanisms occurring during the growth could be precisely understood and investigated on an atomistic level. It has been shown from the simulation results that certain types of deposition play significant roles in the quality of resultant thin films and surface morphology, thus providing insight to the optimal deposition conditions for growing complete crystalline ZnO layers

Applications of Probabilistic Inference to Planning & Reinforcement Learning

Optimal control is a profound and fascinating subject that regularly attracts interest from numerous scien- tific disciplines, including both pure and applied Mathematics, Computer Science, Artificial Intelligence, Psychology, Neuroscience and Economics. In 1960 Rudolf Kalman discovered that there exists a dual- ity between the problems of filtering and optimal control in linear systems [84]. This is now regarded as a seminal piece of work and it has since motivated a large amount of research into the discovery of similar dualities between optimal control and statistical inference. This is especially true of recent years where there has been much research into recasting problems of optimal control into problems of statis- tical/approximate inference. Broadly speaking this is the perspective that we take in this work and in particular we present various applications of methods from the fields of statistical/approximate inference to optimal control, planning and Reinforcement Learning. Some of the methods would be more accu- rately described to originate from other fields of research, such as the dual decomposition techniques used in chapter(5) which originate from convex optimisation. However, the original motivation for the application of these techniques was from the field of approximate inference. The study of dualities be- tween optimal control and statistical inference has been a subject of research for over 50 years and we do not claim to encompass the entire subject. Instead, we present what we consider to be a range of interesting and novel applications from this field of researc

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

Design and Optimization of Physical Waveform-Diverse and Spatially-Diverse Radar Emissions

With the advancement of arbitrary waveform generation techniques, new radar transmission modes can be designed via precise control of the waveform's time-domain signal structure. The finer degree of emission control for a waveform (or multiple waveforms via a digital array) presents an opportunity to reduce ambiguities in the estimation of parameters within the radar backscatter. While this freedom opens the door to new emission capabilities, one must still consider the practical attributes for radar waveform design. Constraints such as constant amplitude (to maintain sufficient power efficiency) and continuous phase (for spectral containment) are still considered prerequisites for high-powered radar waveforms. These criteria are also applicable to the design of multiple waveforms emitted from an antenna array in a multiple-input multiple-output (MIMO) mode. In this work, three spatially-diverse radar emission design methods are introduced that provide constant amplitude, spectrally-contained waveforms implemented via a digital array radar (DAR). The first design method, denoted as spatial modulation, designs the radar waveforms via a polyphase-coded frequency-modulated (PCFM) framework to steer the coherent mainbeam of the emission within a pulse. The second design method is an iterative scheme to generate waveforms that achieve a desired wideband and/or widebeam radar emission. However, a wideband and widebeam emission can place a portion of the emitted energy into what is known as the `invisible' space of the array, which is related to the storage of reactive power that can damage a radar transmitter. The proposed design method purposefully avoids this space and a quantity denoted as the Fractional Reactive Power (FRP) is defined to assess the quality of the result. The third design method produces simultaneous radar and communications beams in separate spatial directions while maintaining constant modulus by leveraging the orthogonal complement of the emitted directions. This orthogonal energy defines a trade-space between power efficiency gained from constraining waveforms to be constant amplitude and power efficiency lost by emitting energy in undesired directions. The design of FM waveforms via traditional gradient-based optimization methods is also considered. A waveform model is proposed that is a generalization of the PCFM implementation, denoted as coded-FM (CFM), which defines the phase of the waveform via a summation of weighted, predefined basis functions. Therefore, gradient-based methods can be used to minimize a given cost function with respect to a finite set of optimizable parameters. A generalized integrated sidelobe level (GISL) metric is used as the optimization cost function to minimize the correlation range sidelobes of the radar waveform. System specific waveform optimization is explored by incorporating the linear models of three different loopback configurations into the GISL metric to match the optimized waveforms to the particular systems

Efficient inference algorithms for network activities

The real social network and associated communities are often hidden under the declared friend or group lists in social networks. We usually observe the manifestation of these hidden networks and communities in the form of recurrent and time-stamped individuals' activities in the social network. The inference of relationship between users/nodes or groups of users/nodes could be further complicated when activities are interval-censored, that is, when one only observed the number of activities that occurred in certain time windows. The same phenomenon happens in the online advertisement world where the advertisers often offer a set of advertisement impressions and observe a set of conversions (i.e. product/service adoption). In this case, the advertisers desire to know which advertisements best appeal to the customers and most importantly, their rate of conversions. Inspired by these challenges, we investigated inference algorithms that efficiently recover user relationships in both cases: time-stamped data and interval-censored data. In case of time-stamped data, we proposed a novel algorithm called NetCodec, which relies on a Hawkes process that models the intertwine relationship between group participation and between-user influence. Using Bayesian variational principle and optimization techniques, NetCodec could infer both group participation and user influence simultaneously with iteration complexity being O((N+I)G), where N is the number of events, I is the number of users, and G is the number of groups. In case of interval-censored data, we proposed a Monte-Carlo EM inference algorithm where we iteratively impute the time-stamped events using a Poisson process that has intensity function approximates the underlying intensity function. We show that that proposed simulated approach delivers better inference performance than baseline methods. In the advertisement problem, we propose a Click-to-Conversion delay model that uses Hawkes processes to model the advertisement impressions and thinned Poisson processes to model the Click-to-Conversion mechanism. We then derive an efficient Maximum Likelihood Estimator which utilizes the Minorization-Maximization framework. We verify the model against real life online advertisement logs in comparison with recent conversion rate estimation methods. To facilitate reproducible research, we also developed an open-source software package that focuses on various Hawkes processes proposed in the above mentioned works and prior works. We provided efficient parallel (multi-core) implementations of the inference algorithms using the Bayesian variational inference framework. To further speed up these inference algorithms, we also explored distributed optimization techniques for convex optimization under the distributed data situation. We formulate this problem as a consensus-constrained optimization problem and solve it with the alternating direction method for multipliers (ADMM). It turns out that using bipartite graph as communication topology exhibits the fastest convergence.Ph.D

Hierarchical multiscale modeling of materials: an application to microporous systems

Computer modeling techniques have now established as a standard method for the study of chemical and physical phenomena in complex systems. The origin of their success is twofold: from one side, the continuous increase in the computational power, along with the spread of parallel architectures, has made accessible to a wide audience the most accurate methodologies, while, on the other side, the refinement of the techniques themselves has improved the reliability of the results. Nevertheless, when dealing with multiscale systems it is not possible to effectively use just one single simulation technique. This problem is largely due to inherent limitations of the methods, and it depends only slightly on the finite nature of computing resources. The extension of space and time scales can be achieved through two different approaches: one consists in connecting different levels in the hierarchy by a coarse-graining of the information coming from highly detailed levels, and the other consists in overcoming the intrinsic limitations of the single techniques. In this thesis, the possibility of using both approaches for large scale simulations has been investigated. The methods here developed have been applied to the study of microporous systems, being particularly well suited for a multiscale approach. Nonetheless, such methods are completely general, and can be employed for the simulation of a wide class of materials without too much effort.</br

Advances in Probabilistic Modelling: Sparse Gaussian Processes, Autoencoders, and Few-shot Learning

Learning is the ability to generalise beyond training examples; but because many generalisations are consistent with a given set of observations, all machine learning methods rely on inductive biases to select certain generalisations over others. This thesis explores how the model structure and priors affect the inductiven biases of probabilistic models, and our ability to learn and make inferences from data. Specifically we present theoretical analyses alongside algorithmic and modelling advances in three areas of probabilistic machine learning: sparse Gaussian process approximations and invariant covariance functions, learning flexible priors for variational autoencoders, and probabilistic approaches for few-shot learning. As inference is rarely tractable, we discuss variational inference methods as a secondary theme. First, we disentangle the theoretical properties and optimisation behaviour of two widely used sparse Gaussian process approximations. We conclude that a variational free energy approximation is more principled and extensible and should be used in practice despite potential optimisation difficulties. We then discuss how general symmetries and invariances can be integrated into Gaussian process priors and can be learned using the marginal likelihood. To make inference tractable, we develop a variational inference scheme that uses unbiased estimates of intractable covariance functions. We then address the mismatch between aggregate posteriors and priors in variational autoencoders and propose a mechanism to define flexible distributions using a form of rejection sampling. We use this approach to define a more flexible prior distribution on the latent space of a variational autoencoder, which generalises to unseen test data and reduces the number of low quality samples from the model in a practical way. Finally, we propose two probabilistic approaches to few-shot learning that achieve state of the art results on benchmarks, building on multi-task probabilistic models with adaptive classifier heads. Our first approach combines a pre-trained deep feature extractor with a simple probabilistic model for the head, and can be linked to automatically regularised softmax regression. The second employs an amortised head model; it can be viewed to meta-learn probabilistic inference for prediction, and can be generalised to other contexts such as few-shot regression.UK Engineering and Physics Research Council (EPSRC) DTA, Qualcomm Studentship in Technology, Max Planck Societ

Applications of Molecular Dynamics simulations for biomolecular systems and improvements to density-based clustering in the analysis

Molecular Dynamics simulations provide a powerful tool to study biomolecular systems with atomistic detail. The key to better understand the function and behaviour of these molecules can often be found in their structural variability. Simulations can help to expose this information that is otherwise experimentally hard or impossible to attain. This work covers two application examples for which a sampling and a characterisation of the conformational ensemble could reveal the structural basis to answer a topical research question. For the fungal toxin phalloidin—a small bicyclic peptide—observed product ratios in different cyclisation reactions could be rationalised by assessing the conformational pre-organisation of precursor fragments. For the C-type lectin receptor langerin, conformational changes induced by different side-chain protonations could deliver an explanation of the pH-dependency in the protein’s calcium-binding. The investigations were accompanied by the continued development of a density-based clustering protocol into a respective software package, which is generally well applicable for the use case of extracting conformational states from Molecular Dynamics data