279 research outputs found
Sample-Efficient Reinforcement Learning for Linearly-Parameterized MDPs with a Generative Model
The curse of dimensionality is a widely known issue in reinforcement learning
(RL). In the tabular setting where the state space and the action
space are both finite, to obtain a nearly optimal policy with
sampling access to a generative model, the minimax optimal sample complexity
scales linearly with , which can be
prohibitively large when or is large. This paper
considers a Markov decision process (MDP) that admits a set of state-action
features, which can linearly express (or approximate) its probability
transition kernel. We show that a model-based approach (resp.Q-learning)
provably learns an -optimal policy (resp.Q-function) with high
probability as soon as the sample size exceeds the order of
(resp.), up to some logarithmic
factor. Here is the feature dimension and is the discount
factor of the MDP. Both sample complexity bounds are provably tight, and our
result for the model-based approach matches the minimax lower bound. Our
results show that for arbitrarily large-scale MDP, both the model-based
approach and Q-learning are sample-efficient when is relatively small, and
hence the title of this paper
The Isotonic Mechanism for Exponential Family Estimation
In 2023, the International Conference on Machine Learning (ICML) required
authors with multiple submissions to rank their submissions based on perceived
quality. In this paper, we aim to employ these author-specified rankings to
enhance peer review in machine learning and artificial intelligence conferences
by extending the Isotonic Mechanism (Su, 2021, 2022) to exponential family
distributions. This mechanism generates adjusted scores closely align with the
original scores while adhering to author-specified rankings. Despite its
applicability to a broad spectrum of exponential family distributions, this
mechanism's implementation does not necessitate knowledge of the specific
distribution form. We demonstrate that an author is incentivized to provide
accurate rankings when her utility takes the form of a convex additive function
of the adjusted review scores. For a certain subclass of exponential family
distributions, we prove that the author reports truthfully only if the question
involves only pairwise comparisons between her submissions, thus indicating the
optimality of ranking in truthful information elicitation. Lastly, we show that
the adjusted scores improve dramatically the accuracy of the original scores
and achieve nearly minimax optimality for estimating the true scores with
statistical consistecy when true scores have bounded total variation
Bridging Convex and Nonconvex Optimization in Robust PCA: Noise, Outliers, and Missing Data
This paper delivers improved theoretical guarantees for the convex
programming approach in low-rank matrix estimation, in the presence of (1)
random noise, (2) gross sparse outliers, and (3) missing data. This problem,
often dubbed as robust principal component analysis (robust PCA), finds
applications in various domains. Despite the wide applicability of convex
relaxation, the available statistical support (particularly the stability
analysis vis-a-vis random noise) remains highly suboptimal, which we strengthen
in this paper. When the unknown matrix is well-conditioned, incoherent, and of
constant rank, we demonstrate that a principled convex program achieves
near-optimal statistical accuracy, in terms of both the Euclidean loss and the
loss. All of this happens even when nearly a constant fraction
of observations are corrupted by outliers with arbitrary magnitudes. The key
analysis idea lies in bridging the convex program in use and an auxiliary
nonconvex optimization algorithm, and hence the title of this paper
Inference and Uncertainty Quantification for Noisy Matrix Completion
Noisy matrix completion aims at estimating a low-rank matrix given only
partial and corrupted entries. Despite substantial progress in designing
efficient estimation algorithms, it remains largely unclear how to assess the
uncertainty of the obtained estimates and how to perform statistical inference
on the unknown matrix (e.g.~constructing a valid and short confidence interval
for an unseen entry).
This paper takes a step towards inference and uncertainty quantification for
noisy matrix completion. We develop a simple procedure to compensate for the
bias of the widely used convex and nonconvex estimators. The resulting
de-biased estimators admit nearly precise non-asymptotic distributional
characterizations, which in turn enable optimal construction of confidence
intervals\,/\,regions for, say, the missing entries and the low-rank factors.
Our inferential procedures do not rely on sample splitting, thus avoiding
unnecessary loss of data efficiency. As a byproduct, we obtain a sharp
characterization of the estimation accuracy of our de-biased estimators, which,
to the best of our knowledge, are the first tractable algorithms that provably
achieve full statistical efficiency (including the preconstant). The analysis
herein is built upon the intimate link between convex and nonconvex
optimization --- an appealing feature recently discovered by
\cite{chen2019noisy}.Comment: published at Proceedings of the National Academy of Sciences Nov
2019, 116 (46) 22931-2293
Walking Support and Power Assistance of a Wheelchair Typed Omnidirectional Mobile Robot with Admittance Control
Composition and predictive functional analysis of bacterial communities inhabiting Chinese Cordyceps insight into conserved core microbiome.
BACKGROUND: Over the past few decades, most attention to Chinese Cordyceps-associated endogenous microorganism was focused on the fungal community that creates critical bioactive components. Bacterial community associated with Chinese Cordyceps has been previously described; however, most studies were only presenting direct comparisons in the Chinese Cordyceps and its microenvironments. In the current study, our objectives were to reveal the bacterial community structure composition and predict their function.
RESULTS: We collected samples of Chinese Cordyceps from five sites located in the Qinghai-Tibet Plateau and used a high throughput sequencing method to compare Chinese Cordyceps-associated bacterial community composition and diversity quantitatively across sites. The results indicated that for the Chinese Cordyceps-associated bacterial community there is no single core microbiome, which was dominated by the both Proteobacteria and Actinobacteria. Predictive functional profiling suggested a location specific function pattern for Chinese Cordyceps and bacteria in the external mycelial cortices involved in the biosynthesis of active constituents.
CONCLUSIONS: This study is firstly used high throughput sequencing method to compare the bacterial communities inhabiting Chinese Cordyceps and its microhabitat and to reveal composition functional capabilities of the bacteria, which will accelerate the study of the functions of bacterial communities in the micro-ecological system of Chinese Cordyceps
- …