183 research outputs found
Multi-armed Bandit Learning on a Graph
The multi-armed bandit(MAB) problem is a simple yet powerful framework that
has been extensively studied in the context of decision-making under
uncertainty. In many real-world applications, such as robotic applications,
selecting an arm corresponds to a physical action that constrains the choices
of the next available arms (actions). Motivated by this, we study an extension
of MAB called the graph bandit, where an agent travels over a graph trying to
maximize the reward collected from different nodes. The graph defines the
freedom of the agent in selecting the next available nodes at each step. We
assume the graph structure is fully available, but the reward distributions are
unknown. Built upon an offline graph-based planning algorithm and the principle
of optimism, we design an online learning algorithm that balances long-term
exploration-exploitation using the principle of optimism. We show that our
proposed algorithm achieves learning regret,
where is the number of nodes and is the diameter of the graph, which
is superior compared to the best-known reinforcement learning algorithms under
similar settings. Numerical experiments confirm that our algorithm outperforms
several benchmarks. Finally, we present a synthetic robotic application modeled
by the graph bandit framework, where a robot moves on a network of
rural/suburban locations to provide high-speed internet access using our
proposed algorithm
Distributed Information-based Source Seeking
In this paper, we design an information-based multi-robot source seeking
algorithm where a group of mobile sensors localizes and moves close to a single
source using only local range-based measurements. In the algorithm, the mobile
sensors perform source identification/localization to estimate the source
location; meanwhile, they move to new locations to maximize the Fisher
information about the source contained in the sensor measurements. In doing so,
they improve the source location estimate and move closer to the source. Our
algorithm is superior in convergence speed compared with traditional field
climbing algorithms, is flexible in the measurement model and the choice of
information metric, and is robust to measurement model errors. Moreover, we
provide a fully distributed version of our algorithm, where each sensor decides
its own actions and only shares information with its neighbors through a sparse
communication network. We perform intensive simulation experiments to test our
algorithms on large-scale systems and physical experiments on small ground
vehicles with light sensors, demonstrating success in seeking a light source
Non-asymptotic System Identification for Linear Systems with Nonlinear Policies
This paper considers a single-trajectory system identification problem for
linear systems under general nonlinear and/or time-varying policies with i.i.d.
random excitation noises. The problem is motivated by safe learning-based
control for constrained linear systems, where the safe policies during the
learning process are usually nonlinear and time-varying for satisfying the
state and input constraints. In this paper, we provide a non-asymptotic error
bound for least square estimation when the data trajectory is generated by any
nonlinear and/or time-varying policies as long as the generated state and
action trajectories are bounded. This significantly generalizes the existing
non-asymptotic guarantees for linear system identification, which usually
consider i.i.d. random inputs or linear policies. Interestingly, our error
bound is consistent with that for linear policies with respect to the
dependence on the trajectory length, system dimensions, and excitation levels.
Lastly, we demonstrate the applications of our results by safe learning with
robust model predictive control and provide numerical analysis
Regulation of the stability and transcriptional activity of NFATc4 by ubiquitination
AbstractNuclear factor of activated T cells (NFATc4) has been implicated as a critical regulator of the cardiac development and hypertrophy. However, the mechanisms for regulating NFATc4 stability and transactivation remain unclear. We showed that NFATc4 protein was predominantly ubiquitinated through the formation of Lysine 48-linked polyubiquitin chains, and this modification decreased NFATc4 protein levels and its transcriptional activity. Furthermore, activation of GSK3β markedly enhanced NFATc4 ubiquitination and decreased its transactivation, whereas inhibition of GSK3β had opposite effects. Importantly, ubiquitination and phosphorylation induced by GSK3β repressed NFATc4-dependent cardiac-specific gene expression. These results demonstrate that the ubiquitin–proteasome system plays an important role in regulating NFATc4 stability and transactivation.Structured summaryMINT-6798349:NFATc4 (uniprotkb:Q14934) physically interacts (MI:0218) with Ubiquitin (uniprotkb:P62988) by anti bait coimmunoprecipitation (MI:0006)MINT-6798334:NFATc4 (uniprotkb:Q14934) physically interacts (MI:0218) with Ubiquitin (uniprotkb:P62988) by anti tag coimmunoprecipitation (MI:0007)MINT-6798321:Ubiquitin (uniprotkb:P62988) physically interacts (MI:0218) with NFATc4 (uniprotkb:Q14934) by pull down (MI:0096
Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization
We study the multi-agent Bayesian optimization (BO) problem, where multiple
agents maximize a black-box function via iterative queries. We focus on Entropy
Search (ES), a sample-efficient BO algorithm that selects queries to maximize
the mutual information about the maximum of the black-box function. One of the
main challenges of ES is that calculating the mutual information requires
computationally-costly approximation techniques. For multi-agent BO problems,
the computational cost of ES is exponential in the number of agents. To address
this challenge, we propose the Gaussian Max-value Entropy Search, a multi-agent
BO algorithm with favorable sample and computational efficiency. The key to our
idea is to use a normal distribution to approximate the function maximum and
calculate its mutual information accordingly. The resulting approximation
allows queries to be cast as the solution of a closed-form optimization problem
which, in turn, can be solved via a modified gradient ascent algorithm and
scaled to a large number of agents. We demonstrate the effectiveness of
Gaussian max-value Entropy Search through numerical experiments on standard
test functions and real-robot experiments on the source-seeking problem.
Results show that the proposed algorithm outperforms the multi-agent BO
baselines in the numerical experiments and can stably seek the source with a
limited number of noisy observations on real robots.Comment: 10 pages, 9 figure
EA-BEV: Edge-aware Bird' s-Eye-View Projector for 3D Object Detection
In recent years, great progress has been made in the Lift-Splat-Shot-based
(LSS-based) 3D object detection method, which converts features of 2D camera
view and 3D lidar view to Bird's-Eye-View (BEV) for feature fusion. However,
inaccurate depth estimation (e.g. the 'depth jump' problem) is an obstacle to
develop LSS-based methods. To alleviate the 'depth jump' problem, we proposed
Edge-Aware Bird's-Eye-View (EA-BEV) projector. By coupling proposed edge-aware
depth fusion module and depth estimate module, the proposed EA-BEV projector
solves the problem and enforces refined supervision on depth. Besides, we
propose sparse depth supervision and gradient edge depth supervision, for
constraining learning on global depth and local marginal depth information. Our
EA-BEV projector is a plug-and-play module for any LSS-based 3D object
detection models, and effectively improves the baseline performance. We
demonstrate the effectiveness on the nuScenes benchmark. On the nuScenes 3D
object detection validation dataset, our proposed EA-BEV projector can boost
several state-of-the-art LLS-based baselines on nuScenes 3D object detection
benchmark and nuScenes BEV map segmentation benchmark with negligible increment
of inference time
Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution
This paper proposes a novel module called middle spectrum grouped convolution
(MSGC) for efficient deep convolutional neural networks (DCNNs) with the
mechanism of grouped convolution. It explores the broad "middle spectrum" area
between channel pruning and conventional grouped convolution. Compared with
channel pruning, MSGC can retain most of the information from the input feature
maps due to the group mechanism; compared with grouped convolution, MSGC
benefits from the learnability, the core of channel pruning, for constructing
its group topology, leading to better channel division. The middle spectrum
area is unfolded along four dimensions: group-wise, layer-wise, sample-wise,
and attention-wise, making it possible to reveal more powerful and
interpretable structures. As a result, the proposed module acts as a booster
that can reduce the computational cost of the host backbones for general image
recognition with even improved predictive accuracy. For example, in the
experiments on ImageNet dataset for image classification, MSGC can reduce the
multiply-accumulates (MACs) of ResNet-18 and ResNet-50 by half but still
increase the Top-1 accuracy by more than 1%. With 35% reduction of MACs, MSGC
can also increase the Top-1 accuracy of the MobileNetV2 backbone. Results on MS
COCO dataset for object detection show similar observations. Our code and
trained models are available at https://github.com/hellozhuo/msgc.Comment: 13 pages, 11 figures, submitted to IEEEE Transactions on xx
- …