Momentum Contrastive Pre-training for Question Answering

Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-trained models to simple keyword matching. In order to address this problem, we propose a novel Momentum Contrastive pRe-training fOr queStion anSwering (MCROSS) method for extractive QA. Specifically, MCROSS introduces a momentum contrastive learning framework to align the answer probability between cloze-like and natural query-passage sample pairs. Hence, the pre-trained models can better transfer the knowledge learned in cloze-like samples to answering natural questions. Experimental results on three benchmarking QA datasets show that our method achieves noticeable improvement compared with all baselines in both supervised and zero-shot scenarios.Comment: This work has been accepted by EMNLP 2022. Reference to ACL Anthology: https://aclanthology.org/2022.emnlp-main.291.pd

Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching

Powered by large-scale pre-training, vision foundation models exhibit significant potential in open-world image understanding. However, unlike large language models that excel at directly tackling various language tasks, vision foundation models require a task-specific model structure followed by fine-tuning on specific tasks. In this work, we present Matcher, a novel perception paradigm that utilizes off-the-shelf vision foundation models to address various perception tasks. Matcher can segment anything by using an in-context example without training. Additionally, we design three effective components within the Matcher framework to collaborate with these foundation models and unleash their full potential in diverse perception tasks. Matcher demonstrates impressive generalization performance across various segmentation tasks, all without training. For example, it achieves 52.7% mIoU on COCO-20$^i$ with one example, surpassing the state-of-the-art specialist model by 1.6%. In addition, Matcher achieves 33.0% mIoU on the proposed LVIS-92$^i$ for one-shot semantic segmentation, outperforming the state-of-the-art generalist model by 14.4%. Our visualization results further showcase the open-world generality and flexibility of Matcher when applied to images in the wild. Our code can be found at https://github.com/aim-uofa/Matcher.Comment: Accepted to ICLR202

NMA: Neural Multi-slot Auctions with Externalities for Online Advertising

Online advertising driven by auctions brings billions of dollars in revenue for social networking services and e-commerce platforms. GSP auctions, which are simple and easy to understand for advertisers, have almost become the benchmark for ad auction mechanisms in the industry. However, most GSP-based industrial practices assume that the user click only relies on the ad itself, which overlook the effect of external items, referred to as externalities. Recently, DNA has attempted to upgrade GSP with deep neural networks and models local externalities to some extent. However, it only considers set-level contexts from auctions and ignores the order and displayed position of ads, which is still suboptimal. Although VCG-based multi-slot auctions (e.g., VCG, WVCG) make it theoretically possible to model global externalities (e.g., the order and positions of ads and so on), they lack an efficient balance of both revenue and social welfare. In this paper, we propose novel auction mechanisms named Neural Multi-slot Auctions (NMA) to tackle the above-mentioned challenges. Specifically, we model the global externalities effectively with a context-aware list-wise prediction module to achieve better performance. We design a list-wise deep rank module to guarantee incentive compatibility in end-to-end learning. Furthermore, we propose an auxiliary loss for social welfare to effectively reduce the decline of social welfare while maximizing revenue. Experiment results on both offline large-scale datasets and online A/B tests demonstrate that NMA obtains higher revenue with balanced social welfare than other existing auction mechanisms (i.e., GSP, DNA, WVCG) in industrial practice, and we have successfully deployed NMA on Meituan food delivery platform.Comment: 10 pages, 3figure

Context Effects in the Judgment of Visual Relative-Frequency: Trial-by-Trial Adaptation and Non-linear Sequential Effect

Humans' judgment of relative-frequency, similar to their use of probability in decision-making, is often distorted as an inverted-S-shape curve—small relative-frequency overestimated and large relative-frequency underestimated. Here we investigated how the judgment of relative-frequency, despite its natural reference points (0 and 1) and stereotyped distortion, may adapt to the environmental statistics. The task was to report the relative-frequency of black (or white) dots in a visual array of black and white dots. We found that participants' judgment was distorted in the typical inverted-S-shape, but the distortion curve was influenced by both the central tendency and spread of the distribution of objective relative-frequencies: the lower the central tendency, the higher the overall judgment (contrast effect); the higher the spread, the more curved the inverted-S-shape (curvature effect). These context effects are in the spirit of efficient coding but opposite to what would be predicted by Bayesian inference. We further modeled the context effects on the level of individual trials, through which we found not only a trial-by-trial adaptation, but also the non-linear sequential effects that were recently reported mainly in circularly distributed visual stimuli

Improved charge extraction in inverted perovskite solar cells with dual-site-binding ligands

Inverted (pin) perovskite solar cells (PSCs) afford improved operating stability in comparison to their nip counterparts but have lagged in power conversion efficiency (PCE). The energetic losses responsible for this PCE deficit in pin PSCs occur primarily at the interfaces between the perovskite and the charge-transport layers. Additive and surface treatments that use passivating ligands usually bind to a single active binding site: This dense packing of electrically resistive passivants perpendicular to the surface may limit the fill factor in pin PSCs. We identified ligands that bind two neighboring lead(II) ion (Pb2+) defect sites in a planar ligand orientation on the perovskite. We fabricated pin PSCs and report a certified quasi–steady state PCE of 26.15 and 24.74% for 0.05– and 1.04–square centimeter illuminated areas, respectively. The devices retain 95% of their initial PCE after 1200 hours of continuous 1 sun maximum power point operation at 65°C

Phase Change Material Based Ohmic Switches for Reconfigurable RF Applications

This research work is focused on the study and development of chalcogenide phase change materials and their applications in reconfigurable RF modules and systems. Germanium telluride (GeTe), one of the chalcogenide phase change materials, is studied and used in the development process of RF ohmic switches. This thesis presents the study of GeTe and other phase change materials, the design, fabrication and measurements of GeTe phase change material based RF switches, and the performance evaluation as well as the operation and breakdown analysis of the GeTe phase change RF switches. It also discusses the potential applications of GeTe RF switches in reconfigurable RF modules by demonstrating a bandpass filter design. RF switches based on solid-state transistors and diodes, and micro-electromechanical system (MEMS) as well as other technologies have been reported and used in integrated circuits and systems for RF and microwave applications. Each of these technologies for RF switches shows some limitations regarding RF performance, integration compatibility, cost, fabrication yield, or reliability. This thesis presents a novel alternative for RF switch development using GeTe phase change material. The special phase transition properties of phase change materials have drawn attention for decades. Material study and characterization of phase change materials have been performed for a better understanding of their properties. Phase change materials have since been developed for different applications, with non-volatile memory modules being the most successful application. With the success in phase change memory design, we have directed our attention to RF switching applications based on phase change materials. Two main types of GeTe phase change material based RF ohmic switches are developed and the design and fabrication of each is discussed in detail. The RF switches designed using GeTe have shown very competitive performance results compared to other existing RF switch designs. Analysis and modeling of the switches have also been performed for a better understanding of the devices and phase change materials as well as their phase transition process. A reconfigurable bandpass filter using GeTe switches have verified the good functionality of phase change RF switches and their promising potential in reconfigurable RF applications.PHDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/138752/1/wangmz_1.pd

Shared temporal expectation across higher- and lower-level cognition

Temporal expectation for future events allows people to prepare more efficiently for the future. In sensorimotor tasks, it has been considered as an important factor that influences the accuracy and speed of responding to specific sensory events. However, there was no consensus whether the temporal expectation functioning in sensorimotor tasks is simply an emergent property of task-specific, low-level circuits, or an abstract representation shared by higher-level cognition. In four experiments, we asked whether two simultaneously processed tasks—one of lower-level and the other of higher-level cognition—would be influenced by the same temporal expectation. One task was speeded response to a target stimulus, where the target was cancelled on 30% of the trials. The other task was a real-time gambling task, where participants needed to predict from time to time whether the current trial would end up with target or cancellation. Both the target and cancellation latencies followed specific distributions, with the distribution of cancellation latencies varied across blocks. Participants’ choices in gambling provided real-time measures of the updating of temporal expectation over time, which suggest imperfect representation of temporal distributions. Importantly, we found that on a trial when participants predicted an ending of cancellation instead of target, their subsequent response to the target was strikingly slower (up to 1/3 increase in response time). It implies temporal expectation is shared across higher-level and lower-level cognitive tasks