472 research outputs found
A Corrected Expected Improvement Acquisition Function Under Noisy Observations
Sequential maximization of expected improvement (EI) is one of the most
widely used policies in Bayesian optimization because of its simplicity and
ability to handle noisy observations. In particular, the improvement function
often uses the best posterior mean as the best incumbent in noisy settings.
However, the uncertainty associated with the incumbent solution is often
neglected in many analytic EI-type methods: a closed-form acquisition function
is derived in the noise-free setting, but then applied to the setting with
noisy observations. To address this limitation, we propose a modification of EI
that corrects its closed-form expression by incorporating the covariance
information provided by the Gaussian Process (GP) model. This acquisition
function specializes to the classical noise-free result, and we argue should
replace that formula in Bayesian optimization software packages, tutorials, and
textbooks. This enhanced acquisition provides good generality for noisy and
noiseless settings. We show that our method achieves a sublinear convergence
rate on the cumulative regret bound under heteroscedastic observation noise.
Our empirical results demonstrate that our proposed acquisition function can
outperform EI in the presence of noisy observations on benchmark functions for
black-box optimization, as well as on parameter search for neural network model
compression
Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning
Prompt-based learning has been an effective paradigm for large pretrained
language models (LLM), enabling few-shot or even zero-shot learning. Black-box
prompt search has received growing interest recently for its distinctive
properties of gradient-free optimization, proven particularly useful and
powerful for model-as-a-service usage. However, the discrete nature and the
complexity of combinatorial optimization hinder the efficiency of modern
black-box approaches. Despite extensive research on search algorithms, the
crucial aspect of search space design and optimization has been largely
overlooked. In this paper, we first conduct a sensitivity analysis by prompting
LLM, revealing that only a small number of tokens exert a disproportionate
amount of influence on LLM predictions. Leveraging this insight, we propose the
Clustering and Pruning for Efficient Black-box Prompt Search (ClaPS), a simple
black-box search method that first clusters and prunes the search space to
focus exclusively on influential prompt tokens. By employing even simple search
methods within the pruned search space, ClaPS achieves state-of-the-art
performance across various tasks and LLMs, surpassing the performance of
complex approaches while significantly reducing search costs. Our findings
underscore the critical role of search space design and optimization in
enhancing both the usefulness and the efficiency of black-box prompt-based
learning.Comment: Findings of EMNLP 2023. 10 pages, 5 figures, 4 tables (14 pages, 5
figures, 8 tables including references and appendices
RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models
The emergence of Neural Radiance Fields (NeRF) has promoted the development
of synthesized high-fidelity views of the intricate real world. However, it is
still a very demanding task to repaint the content in NeRF. In this paper, we
propose a novel framework that can take RGB images as input and alter the 3D
content in neural scenes. Our work leverages existing diffusion models to guide
changes in the designated 3D content. Specifically, we semantically select the
target object and a pre-trained diffusion model will guide the NeRF model to
generate new 3D objects, which can improve the editability, diversity, and
application range of NeRF. Experiment results show that our algorithm is
effective for editing 3D objects in NeRF under different text prompts,
including editing appearance, shape, and more. We validate our method on both
real-world datasets and synthetic-world datasets for these editing tasks.
Please visit https://repaintnerf.github.io for a better view of our results.Comment: IJCAI 2023 Accepted (Main Track
An Honest Joker reveals stereotypical beliefs about the face of deception
Research on deception detection has mainly focused on Simple Deception, in which false information is presented as true. Relatively few studies have examined Sophisticated Deception, in which true information is presented as false. Because Sophisticated Deception incentivizes the appearance of dishonesty, it provides a window onto stereotypical beliefs about cues to deception. Here, we adapted the popular Joker Game to elicit spontaneous facial expressions under Simple Deception, Sophisticated Deception, and Plain Truth conditions, comparing facial behaviors in static, dynamic nonspeaking, and dynamic speaking presentations. Facial behaviors were analysed via machine learning using the Facial Action Coding System. Facial activations were more intense and longer lasting in the Sophisticated Deception condition than in the Simple Deception and Plain Truth conditions. More facial action units intensified in the static condition than in the dynamic speaking condition. Simple Deception involved leaked facial behaviors of which deceivers were unaware. In contrast, Sophisticated Deception involved deliberately leaked facial cues, including stereotypical cues to lying (e.g., gaze aversion). These stereotypes were inaccurate in the sense that they diverged from cues in the Simple Deception condition—the actual appearance of deception in this task. Our findings show that different modes of deception can be distinguished via facial action analysis. They also show that stereotypical beliefs concerning cues to deception can inform behavior. To facilitate future research on these topics, the multimodal stimuli developed in this study are available free for scientific use
Cross-Correlation Forecast of CSST Spectroscopic Galaxy and MeerKAT Neutral Hydrogen Intensity Mapping Surveys
Cross-correlating the data of neutral hydrogen (HI) 21cm intensity mapping
with galaxy surveys is an effective method to extract astrophysical and
cosmological information. In this work, we investigate the cross-correlation of
MeerKAT single-dish mode HI intensity mapping and China Space Station Telescope
(CSST) spectroscopic galaxy surveys. We simulate a survey area of
of MeerKAT and CSST surveys at using Multi-Dark N-body
simulation. The PCA algorithm is applied to remove the foregrounds of HI
intensity mapping, and signal compensation is considered to solve the signal
loss problem in the HI-galaxy cross power spectrum caused by the foreground
removal process. We find that from CSST galaxy auto and MeerKAT-CSST cross
power spectra, the constraint accuracy of the parameter product can reach to , which is about one order
of magnitude higher than the current results. After performing the full MeerKAT
HI intensity mapping survey with 5000 deg survey area, the accuracy can be
enhanced to . This implies that the MeerKAT-CSST cross-correlation can
be a powerful tool to probe the cosmic HI property and the evolution of
galaxies and the Universe.Comment: 17 pages, 11 figures, 3 tables. Accepted for publication in RA
- …