A Corrected Expected Improvement Acquisition Function Under Noisy Observations

Sequential maximization of expected improvement (EI) is one of the most widely used policies in Bayesian optimization because of its simplicity and ability to handle noisy observations. In particular, the improvement function often uses the best posterior mean as the best incumbent in noisy settings. However, the uncertainty associated with the incumbent solution is often neglected in many analytic EI-type methods: a closed-form acquisition function is derived in the noise-free setting, but then applied to the setting with noisy observations. To address this limitation, we propose a modification of EI that corrects its closed-form expression by incorporating the covariance information provided by the Gaussian Process (GP) model. This acquisition function specializes to the classical noise-free result, and we argue should replace that formula in Bayesian optimization software packages, tutorials, and textbooks. This enhanced acquisition provides good generality for noisy and noiseless settings. We show that our method achieves a sublinear convergence rate on the cumulative regret bound under heteroscedastic observation noise. Our empirical results demonstrate that our proposed acquisition function can outperform EI in the presence of noisy observations on benchmark functions for black-box optimization, as well as on parameter search for neural network model compression

Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning

Prompt-based learning has been an effective paradigm for large pretrained language models (LLM), enabling few-shot or even zero-shot learning. Black-box prompt search has received growing interest recently for its distinctive properties of gradient-free optimization, proven particularly useful and powerful for model-as-a-service usage. However, the discrete nature and the complexity of combinatorial optimization hinder the efficiency of modern black-box approaches. Despite extensive research on search algorithms, the crucial aspect of search space design and optimization has been largely overlooked. In this paper, we first conduct a sensitivity analysis by prompting LLM, revealing that only a small number of tokens exert a disproportionate amount of influence on LLM predictions. Leveraging this insight, we propose the Clustering and Pruning for Efficient Black-box Prompt Search (ClaPS), a simple black-box search method that first clusters and prunes the search space to focus exclusively on influential prompt tokens. By employing even simple search methods within the pruned search space, ClaPS achieves state-of-the-art performance across various tasks and LLMs, surpassing the performance of complex approaches while significantly reducing search costs. Our findings underscore the critical role of search space design and optimization in enhancing both the usefulness and the efficiency of black-box prompt-based learning.Comment: Findings of EMNLP 2023. 10 pages, 5 figures, 4 tables (14 pages, 5 figures, 8 tables including references and appendices

RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models

The emergence of Neural Radiance Fields (NeRF) has promoted the development of synthesized high-fidelity views of the intricate real world. However, it is still a very demanding task to repaint the content in NeRF. In this paper, we propose a novel framework that can take RGB images as input and alter the 3D content in neural scenes. Our work leverages existing diffusion models to guide changes in the designated 3D content. Specifically, we semantically select the target object and a pre-trained diffusion model will guide the NeRF model to generate new 3D objects, which can improve the editability, diversity, and application range of NeRF. Experiment results show that our algorithm is effective for editing 3D objects in NeRF under different text prompts, including editing appearance, shape, and more. We validate our method on both real-world datasets and synthetic-world datasets for these editing tasks. Please visit https://repaintnerf.github.io for a better view of our results.Comment: IJCAI 2023 Accepted (Main Track

An Honest Joker reveals stereotypical beliefs about the face of deception

Research on deception detection has mainly focused on Simple Deception, in which false information is presented as true. Relatively few studies have examined Sophisticated Deception, in which true information is presented as false. Because Sophisticated Deception incentivizes the appearance of dishonesty, it provides a window onto stereotypical beliefs about cues to deception. Here, we adapted the popular Joker Game to elicit spontaneous facial expressions under Simple Deception, Sophisticated Deception, and Plain Truth conditions, comparing facial behaviors in static, dynamic nonspeaking, and dynamic speaking presentations. Facial behaviors were analysed via machine learning using the Facial Action Coding System. Facial activations were more intense and longer lasting in the Sophisticated Deception condition than in the Simple Deception and Plain Truth conditions. More facial action units intensified in the static condition than in the dynamic speaking condition. Simple Deception involved leaked facial behaviors of which deceivers were unaware. In contrast, Sophisticated Deception involved deliberately leaked facial cues, including stereotypical cues to lying (e.g., gaze aversion). These stereotypes were inaccurate in the sense that they diverged from cues in the Simple Deception condition—the actual appearance of deception in this task. Our findings show that different modes of deception can be distinguished via facial action analysis. They also show that stereotypical beliefs concerning cues to deception can inform behavior. To facilitate future research on these topics, the multimodal stimuli developed in this study are available free for scientific use

Cross-Correlation Forecast of CSST Spectroscopic Galaxy and MeerKAT Neutral Hydrogen Intensity Mapping Surveys

Cross-correlating the data of neutral hydrogen (HI) 21cm intensity mapping with galaxy surveys is an effective method to extract astrophysical and cosmological information. In this work, we investigate the cross-correlation of MeerKAT single-dish mode HI intensity mapping and China Space Station Telescope (CSST) spectroscopic galaxy surveys. We simulate a survey area of $\sim 300$ $\mathrm{deg}^2$ of MeerKAT and CSST surveys at $z=0.5$ using Multi-Dark N-body simulation. The PCA algorithm is applied to remove the foregrounds of HI intensity mapping, and signal compensation is considered to solve the signal loss problem in the HI-galaxy cross power spectrum caused by the foreground removal process. We find that from CSST galaxy auto and MeerKAT-CSST cross power spectra, the constraint accuracy of the parameter product $\Omega_{\rm HI}b_{\rm HI}r_{{\rm HI},g}$ can reach to $\sim1\%$, which is about one order of magnitude higher than the current results. After performing the full MeerKAT HI intensity mapping survey with 5000 deg$^2$ survey area, the accuracy can be enhanced to $<0.3\%$. This implies that the MeerKAT-CSST cross-correlation can be a powerful tool to probe the cosmic HI property and the evolution of galaxies and the Universe.Comment: 17 pages, 11 figures, 3 tables. Accepted for publication in RA