Sensor fusion for semantic segmentation of urban scenes

Abstract—Semantic understanding of environments is an important problem in robotics in general and intelligent au-tonomous systems in particular. In this paper, we propose a semantic segmentation algorithm which effectively fuses infor-mation from images and 3D point clouds. The proposed method incorporates information from multiple scales in an intuitive and effective manner. A late-fusion architecture is proposed to maximally leverage the training data in each modality. Finally, a pairwise Conditional Random Field (CRF) is used as a post-processing step to enforce spatial consistency in the structured prediction. The proposed algorithm is evaluated on the publicly available KITTI dataset [1] [2], augmented with additional pixel and point-wise semantic labels for building, sky, road, vegetation, sidewalk, car, pedestrian, cyclist, sign/pole, and fence regions. A per-pixel accuracy of 89.3 % and average class accuracy of 65.4 % is achieved, well above current state-of-the-art [3]. I

The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces

Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading process grows. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. The PDF format for sharing research papers is widely used due to its portability, but it has significant downsides including: static content, poor accessibility for low-vision readers, and difficulty reading on mobile devices. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces -- even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed ten research prototype interfaces and conducted usability studies with more than 300 participants and real-world users showing improved reading experiences for scholars. We've also released a production reading interface for research papers that will incorporate the best features as they mature. We structure this paper around challenges scholars and the public face when reading research papers -- Discovery, Efficiency, Comprehension, Synthesis, and Accessibility -- and present an overview of our progress and remaining open challenges

ML-CIRRUS: The Airborne Experiment on Natural Cirrus and Contrail Cirrus with the High-Altitude Long-Range Research Aircraft HALO

The Midlatitude Cirrus experiment (ML-CIRRUS) deployed the High Altitude and Long Range Research Aircraft (HALO) to obtain new insights into nucleation, life cycle, and climate impact of natural cirrus and aircraft-induced contrail cirrus. Direct observations of cirrus properties and their variability are still incomplete, currently limiting our understanding of the clouds’ impact on climate. Also, dynamical effects on clouds and feedbacks are not adequately represented in today’s weather prediction models.Here, we present the rationale, objectives, and selected scientific highlights of ML-CIRRUS using the G-550 aircraft of the German atmospheric science community. The first combined in situ–remote sensing cloud mission with HALO united state-of-the-art cloud probes, a lidar and novel ice residual, aerosol, trace gas, and radiation instrumentation. The aircraft observations were accompanied by remote sensing from satellite and ground and by numerical simulations.In spring 2014, HALO performed 16 flights above Europe with a focus on anthropogenic contrail cirrus and midlatitude cirrus induced by frontal systems including warm conveyor belts and other dynamical regimes (jet streams, mountain waves, and convection). Highlights from ML-CIRRUS include 1) new observations of microphysical and radiative cirrus properties and their variability in meteorological regimes typical for midlatitudes, 2) insights into occurrence of in situ–formed and lifted liquid-origin cirrus, 3) validation of cloud forecasts and satellite products, 4) assessment of contrail predictability, and 5) direct observations of contrail cirrus and their distinction from natural cirrus. Hence, ML-CIRRUS provides a comprehensive dataset on cirrus in the densely populated European midlatitudes with the scope to enhance our understanding of cirrus clouds and their role for climate and weather