ONLINE DECISION MAKING FOR DYNAMICAL SYSTEMS: MODEL-BASED AND DATA-DRIVEN APPROACHES

The widespread availability of data sources and their increased speed compared to the past decade have created both new opportunities and challenges for developing decision-making algorithms for data streams. The ability to process data streams and make real-time decisions that align with system dynamics is a crucial aspect in the development of online decision-making algorithms. This thesis leverages tools from control theory, optimization, and learning to address the problem of online decision-making for dynamical systems, considering streaming data and dynamically changing information. Two online decision-making frameworks are presented in this thesis, depending on the availability of system dynamic information. In the first scenario, where the system can be represented by ordinary differential equations using a state-space model, a time-varying convex optimization framework is introduced. This framework combines motion planning and control to design control signals that lead the dynamical system to asymptotically track optimal trajectories implicitly defined through constrained time-varying optimization problems. Consequently, the nonlinear dynamical system is effectively transformed into an optimization algorithm that seeks the optimal solution to the optimization problem. Global asymptotic convergence of the optimization dynamics to the minimizer of the time-varying optimization problem is proven under sufficient regularity assumptions. In the second scenario, when system dynamics are not available, a data-driven approach called constrained reinforcement learning is adopted. Constrained reinforcement learning deals with sequential decision-making problems where an agent aims to maximize its expected total reward while interacting with an unknown environment and receiving sequentially available information over time. The constrained reinforcement learning framework further includes safety constraints or conflicting requirements during the learning process through secondary expected cumulative rewards. To address the limitations of the learning process in constrained reinforcement learning problems, a novel first-order stochastic gradient descent-ascent (GDA) algorithm is proposed: the stochastic dissipative GDA algorithm. This algorithm almost surely converges to the optimal occupancy measure and optimal policy, overcoming the issue of policy oscillation and convergence to suboptimal policies often encountered in C-RL problems

Beta Diffusion

We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Using scaled and shifted beta distributions, beta diffusion utilizes multiplicative transitions over time to create both forward and reverse diffusion processes, maintaining beta distributions in both the forward marginals and the reverse conditionals, given the data at any point in time. Unlike traditional diffusion-based generative models relying on additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. We demonstrate that the proposed KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs, which can also be derived as the KLUBs of the same KL divergence with its two arguments swapped. The loss function of beta diffusion, expressed in terms of Bregman divergence, further supports the efficacy of KLUBs for optimization. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data and validate the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them

Feature-Based Matrix Factorization

Recommender system has been more and more popular and widely used in many applications recently. The increasing information available, not only in quantities but also in types, leads to a big challenge for recommender system that how to leverage these rich information to get a better performance. Most traditional approaches try to design a specific model for each scenario, which demands great efforts in developing and modifying models. In this technical report, we describe our implementation of feature-based matrix factorization. This model is an abstract of many variants of matrix factorization models, and new types of information can be utilized by simply defining new features, without modifying any lines of code. Using the toolkit, we built the best single model reported on track 1 of KDDCup'11.Comment: Minor update, add some related work

Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worlds

Zero-Shot Learning (ZSL) focuses on classifying samples of unseen classes with only their side semantic information presented during training. It cannot handle real-life, open-world scenarios where there are test samples of unknown classes for which neither samples (e.g., images) nor their side semantic information is known during training. Open-Set Recognition (OSR) is dedicated to addressing the unknown class issue, but existing OSR methods are not designed to model the semantic information of the unseen classes. To tackle this combined ZSL and OSR problem, we consider the case of "Zero-Shot Open-Set Recognition" (ZS-OSR), where a model is trained under the ZSL setting but it is required to accurately classify samples from the unseen classes while being able to reject samples from the unknown classes during inference. We perform large experiments on combining existing state-of-the-art ZSL and OSR models for the ZS-OSR task on four widely used datasets adapted from the ZSL task, and reveal that ZS-OSR is a non-trivial task as the simply combined solutions perform badly in distinguishing the unseen-class and unknown-class samples. We further introduce a novel approach specifically designed for ZS-OSR, in which our model learns to generate adversarial semantic embeddings of the unknown classes to train an unknowns-informed ZS-OSR classifier. Extensive empirical results show that our method 1) substantially outperforms the combined solutions in detecting the unknown classes while retaining the classification accuracy on the unseen classes and 2) achieves similar superiority under generalized ZS-OSR settings