Learning Single-Image Depth from Videos using Quality Assessment Networks

Depth estimation from a single image in the wild remains a challenging problem. One main obstacle is the lack of high-quality training data for images in the wild. In this paper we propose a method to automatically generate such data through Structure-from-Motion (SfM) on Internet videos. The core of this method is a Quality Assessment Network that identifies high-quality reconstructions obtained from SfM. Using this method, we collect single-view depth training data from a large number of YouTube videos and construct a new dataset called YouTube3D. Experiments show that YouTube3D is useful in training depth estimation networks and advances the state of the art of single-view depth estimation in the wild

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation

The images and sounds that we perceive undergo subtle but geometrically consistent changes as we rotate our heads. In this paper, we use these cues to solve a problem we call Sound Localization from Motion (SLfM): jointly estimating camera rotation and localizing sound sources. We learn to solve these tasks solely through self-supervision. A visual model predicts camera rotation from a pair of images, while an audio model predicts the direction of sound sources from binaural sounds. We train these models to generate predictions that agree with one another. At test time, the models can be deployed independently. To obtain a feature representation that is well-suited to solving this challenging problem, we also propose a method for learning an audio-visual representation through cross-view binauralization: estimating binaural sound from one view, given images and sound from another. Our model can successfully estimate accurate rotations on both real and synthetic scenes, and localize sound sources with accuracy competitive with state-of-the-art self-supervised approaches. Project site: https://ificl.github.io/SLfM/Comment: ICCV 2023. Project site: https://ificl.github.io/SLfM

OASIS: A Large-Scale Dataset for Single Image 3D in the Wild

Single-view 3D is the task of recovering 3D properties such as depth and surface normals from a single image. We hypothesize that a major obstacle to single-image 3D is data. We address this issue by presenting Open Annotations of Single Image Surfaces (OASIS), a dataset for single-image 3D in the wild consisting of annotations of detailed 3D geometry for 140,000 images. We train and evaluate leading models on a variety of single-image 3D tasks. We expect OASIS to be a useful resource for 3D vision research. Project site: https://pvl.cs.princeton.edu/OASIS.Comment: Accepted to CVPR 202

A hub gene signature as a therapeutic target and biomarker for sepsis and geriatric sepsis-induced ARDS concomitant with COVID-19 infection

BackgroundCOVID-19 and sepsis represent formidable public health challenges, characterized by incompletely elucidated molecular mechanisms. Elucidating the interplay between COVID-19 and sepsis, particularly in geriatric patients suffering from sepsis-induced acute respiratory distress syndrome (ARDS), is of paramount importance for identifying potential therapeutic interventions to mitigate hospitalization and mortality risks.MethodsWe employed bioinformatics and systems biology approaches to identify hub genes, shared pathways, molecular biomarkers, and candidate therapeutics for managing sepsis and sepsis-induced ARDS in the context of COVID-19 infection, as well as co-existing or sequentially occurring infections. We corroborated these hub genes utilizing murine sepsis-ARDS models and blood samples derived from geriatric patients afflicted by sepsis-induced ARDS.ResultsOur investigation revealed 189 differentially expressed genes (DEGs) shared among COVID-19 and sepsis datasets. We constructed a protein-protein interaction network, unearthing pivotal hub genes and modules. Notably, nine hub genes displayed significant alterations and correlations with critical inflammatory mediators of pulmonary injury in murine septic lungs. Simultaneously, 12 displayed significant changes and correlations with a neutrophil-recruiting chemokine in geriatric patients with sepsis-induced ARDS. Of these, six hub genes (CD247, CD2, CD40LG, KLRB1, LCN2, RETN) showed significant alterations across COVID-19, sepsis, and geriatric sepsis-induced ARDS. Our single-cell RNA sequencing analysis of hub genes across diverse immune cell types furnished insights into disease pathogenesis. Functional analysis underscored the interconnection between sepsis/sepsis-ARDS and COVID-19, enabling us to pinpoint potential therapeutic targets, transcription factor-gene interactions, DEG-microRNA co-regulatory networks, and prospective drug and chemical compound interactions involving hub genes.ConclusionOur investigation offers potential therapeutic targets/biomarkers, sheds light on the immune response in geriatric patients with sepsis-induced ARDS, emphasizes the association between sepsis/sepsis-ARDS and COVID-19, and proposes prospective alternative pathways for targeted therapeutic interventions

Understanding 3D Object Interaction from a Single Image

Humans can easily understand a single image as depicting multiple potential objects permitting interaction. We use this skill to plan our interactions with the world and accelerate understanding new objects without engaging in interaction. In this paper, we would like to endow machines with the similar ability, so that intelligent agents can better explore the 3D scene or manipulate objects. Our approach is a transformer-based model that predicts the 3D location, physical properties and affordance of objects. To power this model, we collect a dataset with Internet videos, egocentric videos and indoor images to train and validate our approach. Our model yields strong performance on our data, and generalizes well to robotics data

Understanding Light Harvesting in Radial Junction Amorphous Silicon Thin Film Solar Cells

International audienc

Boosting light emission from Si-based thin film over Si and SiO_2 nanowires architecture

International audienc

Bi-Sn alloy catalyst for simultaneous morphology and doping control of silicon nanowires in radial junction solar cells

International audienc

DataSheet_1_A hub gene signature as a therapeutic target and biomarker for sepsis and geriatric sepsis-induced ARDS concomitant with COVID-19 infection.docx

BackgroundCOVID-19 and sepsis represent formidable public health challenges, characterized by incompletely elucidated molecular mechanisms. Elucidating the interplay between COVID-19 and sepsis, particularly in geriatric patients suffering from sepsis-induced acute respiratory distress syndrome (ARDS), is of paramount importance for identifying potential therapeutic interventions to mitigate hospitalization and mortality risks.MethodsWe employed bioinformatics and systems biology approaches to identify hub genes, shared pathways, molecular biomarkers, and candidate therapeutics for managing sepsis and sepsis-induced ARDS in the context of COVID-19 infection, as well as co-existing or sequentially occurring infections. We corroborated these hub genes utilizing murine sepsis-ARDS models and blood samples derived from geriatric patients afflicted by sepsis-induced ARDS.ResultsOur investigation revealed 189 differentially expressed genes (DEGs) shared among COVID-19 and sepsis datasets. We constructed a protein-protein interaction network, unearthing pivotal hub genes and modules. Notably, nine hub genes displayed significant alterations and correlations with critical inflammatory mediators of pulmonary injury in murine septic lungs. Simultaneously, 12 displayed significant changes and correlations with a neutrophil-recruiting chemokine in geriatric patients with sepsis-induced ARDS. Of these, six hub genes (CD247, CD2, CD40LG, KLRB1, LCN2, RETN) showed significant alterations across COVID-19, sepsis, and geriatric sepsis-induced ARDS. Our single-cell RNA sequencing analysis of hub genes across diverse immune cell types furnished insights into disease pathogenesis. Functional analysis underscored the interconnection between sepsis/sepsis-ARDS and COVID-19, enabling us to pinpoint potential therapeutic targets, transcription factor-gene interactions, DEG-microRNA co-regulatory networks, and prospective drug and chemical compound interactions involving hub genes.ConclusionOur investigation offers potential therapeutic targets/biomarkers, sheds light on the immune response in geriatric patients with sepsis-induced ARDS, emphasizes the association between sepsis/sepsis-ARDS and COVID-19, and proposes prospective alternative pathways for targeted therapeutic interventions.</p