Interfacial characterizations and analytical applications of chemically-modified surfaces

This dissertation explores several new strategies and approaches to the surface modifications for applications in environmental monitoring, and the characterizations of interfaces at the microscopic level. The first of the four papers included in this dissertation describes the development of optical pH sensors based on the immobilization of fluoresceinamine at a base-hydrolyzed cellulose acetate film. The advantages of the sensors include a rapid response time (7 pH units), and exceptional long-term stability. The ionic strength and temperature effects, metal-ion interference, and fluorescence properties of the sensors were examined;The second paper demonstrates the in situ monitoring of the base-hydrolysis of a dithio-bis(succinimidylundecanoate) (DSU) monolayer chemisorbed at a Au(111) surface using scanning force microscopy (SFM). The experiment is based on the dependence of the frictional interactions of the chemical functional groups at the outermost few angstroms of the two surfaces that form the microcontact. The conversion of the ester functionality of DSU to a carboxylate functionality results in an increase in the friction at the tip-sample interface. The tip-assisted hydrolysis of DSU monolayer is reported in the third paper. It was found that contact imaging accelerates the base-hydrolysis of the DSU monolayer relative to the surrounding unimaged area. The proposed mechanism and potential implications to nanotechnology are discussed;An electrochemical approach to the minimization of chloride interference in the determination of chemical oxygen demand (COD) is described in the fourth paper. It is based on the electrochemical deposition of Cl- at silver electrodes. The performance of two types of silver electrodes were evaluated and characterized. Chloride removal to levels below 3 ppm with analysis times of \u3c15 min and COD precision \u3c±20% were demonstrated;An overview of the development in the above research areas was given in the General Introduction section, and a summary of the research results and possible future work were included in the General Conclusions

Shapley Value Based Multi-Agent Reinforcement Learning: Theory, Method and Its Application to Energy Network

Multi-agent reinforcement learning is an area of rapid advancement in artificial intelligence and machine learning. One of the important questions to be answered is how to conduct credit assignment in a multi-agent system. There have been many schemes designed to conduct credit assignment by multi-agent reinforcement learning algorithms. Although these credit assignment schemes have been proved useful in improving the performance of multi-agent reinforcement learning, most of them are designed heuristically without a rigorous theoretic basis and therefore infeasible to understand how agents cooperate. In this thesis, we aim at investigating the foundation of credit assignment in multi-agent reinforcement learning via cooperative game theory. We first extend a game model called convex game and a payoff distribution scheme called Shapley value in cooperative game theory to Markov decision process, named as Markov convex game and Markov Shapley value respectively. We represent a global reward game as a Markov convex game under the grand coalition. As a result, Markov Shapley value can be reasonably used as a credit assignment scheme in the global reward game. Markov Shapley value possesses the following virtues: (i) efficiency; (ii) identifiability of dummy agents; (iii) reflecting the contribution and (iv) symmetry, which form the fair credit assignment. Based on Markov Shapley value, we propose three multi-agent reinforcement learning algorithms called SHAQ, SQDDPG and SMFPPO. Furthermore, we extend Markov convex game to partial observability to deal with the partially observable problems, named as partially observable Markov convex game. In application, we evaluate SQDDPG and SMFPPO on the real-world problem in energy networks.Comment: 206 page

Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning

We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions. It simulates the Nos\'e-Hoover dynamics of a continuously-tempered Hamiltonian system built on the distribution of interest. A significant advantage of this method is that it is not only able to efficiently draw representative i.i.d. samples when the distribution contains multiple isolated modes, but capable of adaptively neutralising the noise arising from mini-batches and maintaining accurate sampling. While the properties of this method have been studied using synthetic distributions, experiments on three real datasets also demonstrated the gain of performance over several strong baselines with various types of neural networks plunged in