1,318 research outputs found
Macro action selection with deep reinforcement learning in StarCraft
StarCraft (SC) is one of the most popular and successful Real Time Strategy
(RTS) games. In recent years, SC is also widely accepted as a challenging
testbed for AI research because of its enormous state space, partially observed
information, multi-agent collaboration, and so on. With the help of annual
AIIDE and CIG competitions, a growing number of SC bots are proposed and
continuously improved. However, a large gap remains between the top-level bot
and the professional human player. One vital reason is that current SC bots
mainly rely on predefined rules to select macro actions during their games.
These rules are not scalable and efficient enough to cope with the enormous yet
partially observed state space in the game. In this paper, we propose a deep
reinforcement learning (DRL) framework to improve the selection of macro
actions. Our framework is based on the combination of the Ape-X DQN and the
Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as
LastOrder. Our evaluation, based on training against all bots from the AIIDE
2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning
rate, outperforming 26 bots in total 28 entrants
The origin and properties of massive prolate galaxies in the Illustris simulation
We study galaxy shapes in the Illustris cosmological hydrodynamic simulation.
We find that massive galaxies have a higher probability of being prolate. For
galaxies with stellar mass larger than , 35 out of total
839 galaxies are prolate. For 21 galaxies with stellar mass larger than
, 9 are prolate, 4 are triaxial while the others are
close to being oblate. There are almost no prolate galaxies with stellar mass
smaller than . We check the merger history of the
prolate galaxies, and find that they are formed by major dry mergers. All the
prolate galaxies have at least one such merger, with most having mass ratios
between and . The gas fraction (gas mass to total baryon mass) of
the progenitors is 0-3 percent for nearly all these mergers, except for one
whose second progenitor contains gas mass, while its main
progenitor still contains less than . For the 35 massive prolate galaxies
that we find, 18 of them have minor axis rotation, and their angular momenta
mostly come from the spin angular momenta of the progenitors (usually that of
the main progenitor). We analyse the merger orbits of these prolate galaxies
and find that most of them experienced a nearly radial merger orbit. Oblate
galaxies with major dry mergers can have either radial or circular merger
orbits. We further discuss various properties of these prolate galaxies, such
as spin parameter , spherical anisotropy parameter ,
dark matter fraction, as well as inner density slopes for the stellar, dark
matter and total mass distributions.Comment: Accepted for publication in MNRAS. 24 pages, 14 figure
Efficient Online Learning with Memory via Frank-Wolfe Optimization: Algorithms with Bounded Dynamic Regret and Applications to Control
Projection operations are a typical computation bottleneck in online
learning. In this paper, we enable projection-free online learning within the
framework of Online Convex Optimization with Memory (OCO-M) -- OCO-M captures
how the history of decisions affects the current outcome by allowing the online
learning loss functions to depend on both current and past decisions.
Particularly, we introduce the first projection-free meta-base learning
algorithm with memory that minimizes dynamic regret, i.e., that minimizes the
suboptimality against any sequence of time-varying decisions. We are motivated
by artificial intelligence applications where autonomous agents need to adapt
to time-varying environments in real-time, accounting for how past decisions
affect the present. Examples of such applications are: online control of
dynamical systems; statistical arbitrage; and time series prediction. The
algorithm builds on the Online Frank-Wolfe (OFW) and Hedge algorithms. We
demonstrate how our algorithm can be applied to the online control of linear
time-varying systems in the presence of unpredictable process noise. To this
end, we develop a controller with memory and bounded dynamic regret against any
optimal time-varying linear feedback control policy. We validate our algorithm
in simulated scenarios of online control of linear time-invariant systems.Comment: The version corrects proofs and updates presentatio
Panel Data Visualization in R (panelView) and Stata (panelview)
We develop an R package panelView and a Stata package panelview for panel data visualization. They are designed to assist causal analysis with panel data and have three main functionalities: (1) They plot the treatment status and missing values in a panel dataset; (2) they visualize the temporal dynamics of the main variables of interest; and (3) they depict the bivariate relationships between a treatment variable and an outcome variable either by unit or in aggregate. These tools can help researchers better understand their panel datasets before conducting statistical analysis
Propagation Characteristics of Seismic Wave While Transiting Interlining in Soil Medium
Based on the theory of wave and the Snell theorem, propagation characteristics of seismic wave, three kinds of elastic body waves, is studied when they pass through soil/elastic solid medium including elastic thin plan intercalation in half-space. The analytic solution of the amplitude rate is the rate of the amplitude of the wave behind the intercalation to that of the incident wave. Compared with the field data, the analytic results are beneficial, affected by soft clay stratum, at the Nankai District of Tianjin Municipality caused by the 1976 Tangshan earthquake (had a magnitude of 7.8). In this area, however, damage of the buildings was generally minor, about 50%, with the Heping District near Nankai
Sparse and Deep Representations for Face Recognition and Object Detection
Face recognition and object detection are two very fundamental visual recognition applications in computer vision. How to learn “good” feature representations using machine learning has become the cornerstone of perception-based systems. A good feature representation is often the one that is robust and discriminative to multiple instances of the same category. Starting from features such as intensity, histogram etc. in the image, followed by hand-crafted features, to the most recent sophisticated deep feature representations, we have witnessed the remarkable improvement in the ability of a feature learning algorithm to perform pattern recognition tasks such as face recognition and object detection. One of the conventional feature learning methods, dictionary learning has been proposed to learn discriminative and sparse representations for visual recognition. These dictionary learning methods can learn both representative and discriminative dictionaries, and the associated sparse representations are effective for vision tasks such as face recognition. More recently, deep features have been widely adopted by the computer vision community owing to the powerful deep neural network, which is capable of distilling information from high dimensional input spaces to a low dimensional semantic space. The research problems which comprise this dissertation lie at the cross section of conventional feature and deep feature learning approaches. Thus, in this dissertation, we study both sparse and deep representations for face recognition and object detection.
First, we begin by studying the topic of spare representations. We present a simple thresholded feature learning algorithm under sparse support recovery. We show that under certain conditions, the thresholded feature exactly recovers the nonzero support of the sparse code. Secondly, based on the theoretical guarantees, we derive the model and algorithm named Dictionary Learning for Thresholded Features (DLTF), to learn the dictionary that is optimized for the thresholded feature. The DLTF dictionaries are specifically designed for using the thresholded feature at inference, which prioritize simplicity, efficiency, general usability and theoretical guarantees. Both synthetic simulations and real-data experiments (i.e. image clustering and unsupervised hashing) verify the competitive quantitative results and remarkable efficiency of applying thresholded features with DLTF dictionaries.
Continuing our focus on investigating the sparse representation and its application to computer vision tasks, we address the sparse representations for unconstrained face verification/recognition problem. In the first part, we address the video-based face recognition problem since it brings more challenges due to the fact that the videos are often acquired under significant variations in poses, expressions, lighting conditions and backgrounds. In order to extract representations that are robust to these variations, we propose a structured dictionary learning framework. Specifically, we employ dictionary learning and low-rank approximation methods to preserve the invariant structure of face images in videos. The learned structured dictionary is both discriminative and reconstructive. We demonstrate the effectiveness of our approach through extensive experiments on three video-based face recognition datasets.
Recently, template-based face verification has gained more popularity. Unlike traditional verification tasks, which evaluate on image-to-image or video-to-video pairs, template-based face verification/recognition methods can exploit training and/or gallery data containing a mixture of both images or videos from the person of interest. In the second part, we propose a regularized sparse coding approach for template-based face verification. First, we construct a reference dictionary, which represents the training set. Then we learn the discriminative sparse codes of the templates for verification through the proposed template regularized sparse coding approach. Finally, we measure the similarity between templates.
However, in real world scenarios, training and test data are sampled from different distributions. Therefore, we also extend the dictionary learning techniques to tackle the domain adaptation problem, where the data from the training set (source domain) and test set (target domain) have different underlying distributions (domain shift). We propose a domain-adaptive dictionary learning framework to model the domain shift by generating a set of intermediate domains. These intermediate domains bridge the gap between the source and target domains. Specifically, we not only learn a common dictionary to encode the domain-shared features but also learn a set of domain specific dictionaries to model the domain shift. This separation enables us to learn more compact and reconstructive dictionaries for domain adaptation. The domain-adaptive features for recognition are finally derived by aligning all the recovered feature representations of both source and target along the domain path. We evaluate our approach on both cross-domain face recognition and object classification tasks.
Finally, we study another fundamental problem in computer vision: generic object detection. Object detection has become one of the most valuable pattern recognition tasks, with great benefits in scene understanding, face recognition, action recognition, robotics and self-driving vehicles, etc. We propose a novel object detector named "Deep Regionlets" by blending deep learning and the traditional regionlet method. The proposed framework "Deep Regionlets" is able to address the limitations of traditional regionlet methods, leading to significant precision improvement by exploiting the power of deep convolutional neural networks. Furthermore, we conduct a detailed analysis of our approach to understand its merits and properties. Extensive experiments on two detection benchmark datasets show that the proposed deep regionlet approach outperforms several state-of-the-art competitors
Mass Dependence of Galaxy-Halo Alignment in LOWZ and CMASS
We measure the galaxy-ellipticity (GI) correlations for the Slogan Digital
Sky Survey DR12 LOWZ and CMASS samples with the shape measurements from the
DESI Legacy Imaging Surveys. We model the GI correlations in an N-body
simulation with our recent accurate stellar-halo mass relation from the
Photometric object Around Cosmic webs (PAC) method. The large data set and our
accurate modeling turns out an accurate measurement of the alignment angle
between central galaxies and their host halos. We find that the alignment of
central {\textit {elliptical}} galaxies with their host halos increases
monotonically with galaxy stellar mass or host halo mass, which can be well
described by a power law for the massive galaxies. We also find that central
elliptical galaxies are more aligned with their host halos in LOWZ than in
CMASS, which might indicate an evolution of galaxy-halo alignment, though
future studies are needed to verify this is not induced by the sample
selections. In contrast, central {\textit {disk}} galaxies are aligned with
their host halos about 10 times more weakly in the GI correlation. These
results have important implications for intrinsic alignment (IA) correction in
weak lensing studies, IA cosmology, and theory of massive galaxy formation.Comment: 9 pages, 4 figures. Accepted for publication in Ap
- …