Statistical Inference on Dynamical Systems

The ordinary differential equation (ODE) is one representative and popular tool in modeling dynamical systems, which are widely implemented in physics, biology, economics, chemistry and biomedical sciences, etc. Because of the importance of dynamical systems in scientific studies, they are the main focuses of my dissertation. The first chapter of the dissertation is introduction and literature review, which mainly focuses on numerical integration algorithms of ODEs that are difficult to solve analytically, as well as derivative-free optimization algorithms for the so-called inverse problem. The second chapter is on the estimation method based on numerical solvers of differential equations. We start by reviewing the state-of-the-art Gauss-Newton algorithm based method, with the derivation of approximate confidence intervals. Furthermore, we propose and illustrate a method using Differential Evolution along with numerical ODE integration algorithms, as well as a hybrid method to improve the convergence issue for Gauss-Newton algorithm. A numerical comparison study shows the hybrid method is more numerically stable than the traditional Gauss-Newton algorithm based estimation method. In Chapter 3 we propose a novel two-step estimation method based on Fourier basis smoothing and pseudo least square estimator. It is less computationally intensive than methods using numerical ODE integration algorithms, and it works better on periodic or near periodic ODE model functions. In Chapter 4 we expand our study to a population-based hierarchical model to study the correlation between individual features and certain parameter values. Both ML and REML estimation are studied, with more emphasis on REML. An iterative estimation method that incorporates numerical ODE solver into the stochastic approximation EM algorithm for the hierarchical model is proposed and illustrated. Several simulation studies are presented, and a parallel version of the algorithm is implemented as well

Deep Extreme Multi-label Learning

Extreme multi-label learning (XML) or classification has been a practical and important problem since the boom of big data. The main challenge lies in the exponential label space which involves $2^L$ possible label sets especially when the label dimension $L$ is huge, e.g., in millions for Wikipedia labels. This paper is motivated to better explore the label space by originally establishing an explicit label graph. In the meanwhile, deep learning has been widely studied and used in various classification problems including multi-label classification, however it has not been properly introduced to XML, where the label space can be as large as in millions. In this paper, we propose a practical deep embedding method for extreme multi-label classification, which harvests the ideas of non-linear embedding and graph priors-based label space modeling simultaneously. Extensive experiments on public datasets for XML show that our method performs competitive against state-of-the-art result