Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions

We consider the paradigm of a black box AI system that makes life-critical decisions. We propose an "arguing machines" framework that pairs the primary AI system with a secondary one that is independently trained to perform the same task. We show that disagreement between the two systems, without any knowledge of underlying system design or operation, is sufficient to arbitrarily improve the accuracy of the overall decision pipeline given human supervision over disagreements. We demonstrate this system in two applications: (1) an illustrative example of image classification and (2) on large-scale real-world semi-autonomous driving data. For the first application, we apply this framework to image classification achieving a reduction from 8.0% to 2.8% top-5 error on ImageNet. For the second application, we apply this framework to Tesla Autopilot and demonstrate the ability to predict 90.4% of system disengagements that were labeled by human annotators as challenging and needing human supervision

How strongly do plumes influence Pacific seamount distribution?

Seamounts are submarine volcanoes postulated to be formed either by hot mantle plumes rising from the deep mantle or by shallow, plate-related processes. However, the relative importance of these two mechanisms has not hitherto been quantified. In this study, applying Gaussian Process regression to reconstruct irregular seamount topography above and under the sedimentary layer, we calculate an accurate map of volcanism distribution within the Pacific plate. We find that previous erupted volumes have been underestimated by 75% on average. Our results show that (1) the total erupted volume postulated to be plume-related makes up only 18% of total Pacific intraplate volcanism, and (2) the volume statistics for plume-related seamounts and those along the Large Low-Shear-Velocity Province margins are nearly indistinguishable from the rest of the intraplate seamounts. We conclude that proposed plumes account for only a minority of the volume of intraplate volcanism in the Pacific plate, implying that shallow rather than deep processes are dominant

CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild

Non-intrusive, real-time analysis of the dynamics of the eye region allows us to monitor humans' visual attention allocation and estimate their mental state during the performance of real-world tasks, which can potentially benefit a wide range of human-computer interaction (HCI) applications. While commercial eye-tracking devices have been frequently employed, the difficulty of customizing these devices places unnecessary constraints on the exploration of more efficient, end-to-end models of eye dynamics. In this work, we propose CLERA, a unified model for Cognitive Load and Eye Region Analysis, which achieves precise keypoint detection and spatiotemporal tracking in a joint-learning framework. Our method demonstrates significant efficiency and outperforms prior work on tasks including cognitive load estimation, eye landmark detection, and blink estimation. We also introduce a large-scale dataset of 30k human faces with joint pupil, eye-openness, and landmark annotation, which aims to support future HCI research on human factors and eye-related analysis.Comment: ACM Transactions on Computer-Human Interactio

Deep Learning for Spatiotemporal Anomaly Forecasting: A Case Study of Marine Heatwaves

Spatiotemporal data have unique properties and require specific considerations. Forecasting spatiotemporal processes is a difficult task because the data are high-dimensional, often are limited in extent, and temporally correlated. Hence, we propose to evaluate several deep learning-based approaches that are relevant to spatiotemporal anomaly forecasting. We will use marine heatwaves as a case study. Those are observed around the world and have strong impacts on marine ecosystems. The evaluated deep learning methods will be integrated for the task of marine heatwave prediction in order to overcome the limitations of spatiotemporal data and improve data-driven seasonal marine heatwave forecasts

Nanorg Microbial Factories: Light-Driven Renewable Biochemical Synthesis Using Quantum Dot-Bacteria Nanobiohybrids

Living cells do not interface naturally with nanoscale materials, although such artificial organisms can have unprecedented multifunctional properties, like wireless activation of enzyme function using electromagnetic stimuli. Realizing such interfacing in a nanobiohybrid organism (or nanorg) requires (1) chemical coupling via affinity binding and self-assembly, (2) the energetic coupling between optoelectronic states of artificial materials with the cellular process, and (3) the design of appropriate interfaces ensuring biocompatibility. Here we show that seven different core−shell quantum dots (QDs), with excitations ranging from ultraviolet to near-infrared energies, couple with targeted enzyme sites in bacteria. When illuminated by light, these QDs drive the renewable production of different biofuels and chemicals using carbon-dioxide (CO2), water, and nitrogen (from air) as substrates. These QDs use their zinc-rich shell facets for affinity attachment to the proteins. Cysteine zwitterion ligands enable uptake through the cell, facilitating cell survival. Together, these nanorgs catalyze light-induced air−water−CO2 reduction with a high turnover number (TON) of ∼106-108 (mols of product per mol of cells) to biofuels like isopropanol (IPA), 2,3-butanediol (BDO), C11−C15 methyl ketones (MKs), and hydrogen (H2); and chemicals such as formic acid (FA), ammonia (NH3), ethylene (C2H4), and degradable bioplastics polyhydroxybutyrate (PHB). Therefore, these resting cells function as nanomicrobial factories powered by light

HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer

Learning expressive representations for high-dimensional yet sparse features has been a longstanding problem in information retrieval. Though recent deep learning methods can partially solve the problem, they often fail to handle the numerous sparse features, particularly those tail feature values with infrequent occurrences in the training data. Worse still, existing methods cannot explicitly leverage the correlations among different instances to help further improve the representation learning on sparse features since such relational prior knowledge is not provided. To address these challenges, in this paper, we tackle the problem of representation learning on feature-sparse data from a graph learning perspective. Specifically, we propose to model the sparse features of different instances using hypergraphs where each node represents a data instance and each hyperedge denotes a distinct feature value. By passing messages on the constructed hypergraphs based on our Hypergraph Transformer (HyperFormer), the learned feature representations capture not only the correlations among different instances but also the correlations among features. Our experiments demonstrate that the proposed approach can effectively improve feature representation learning on sparse features.Comment: Accepted by SIGIR 202