Perception-aware Path Planning

In this paper, we give a double twist to the problem of planning under uncertainty. State-of-the-art planners seek to minimize the localization uncertainty by only considering the geometric structure of the scene. In this paper, we argue that motion planning for vision-controlled robots should be perception aware in that the robot should also favor texture-rich areas to minimize the localization uncertainty during a goal-reaching task. Thus, we describe how to optimally incorporate the photometric information (i.e., texture) of the scene, in addition to the the geometric one, to compute the uncertainty of vision-based localization during path planning. To avoid the caveats of feature-based localization systems (i.e., dependence on feature type and user-defined thresholds), we use dense, direct methods. This allows us to compute the localization uncertainty directly from the intensity values of every pixel in the image. We also describe how to compute trajectories online, considering also scenarios with no prior knowledge about the map. The proposed framework is general and can easily be adapted to different robotic platforms and scenarios. The effectiveness of our approach is demonstrated with extensive experiments in both simulated and real-world environments using a vision-controlled micro aerial vehicle.Comment: 16 pages, 20 figures, revised version. Conditionally accepted for IEEE Transactions on Robotic

Active autonomous aerial exploration for ground robot path planning

We address the problem of planning a path for a ground robot through unknown terrain, using observations from a flying robot. In search and rescue missions, which are our target scenarios, the time from arrival at the disaster site to the delivery of aid is critically important. Previous works required exhaustive exploration before path planning, which is time-consuming but eventually leads to an optimal path for the ground robot. Instead, we propose active exploration of the environment, where the flying robot chooses regions to map in a way that optimizes the overall response time of the system, which is the combined time for the air and ground robots to execute their missions. In our approach, we estimate terrain classes throughout our terrain map, and we also add elevation information in areas where the active exploration algorithm has chosen to perform 3-D reconstruction. This terrain information is used to estimate feasible and efficient paths for the ground robot. By exploring the environment actively, we achieve superior response times compared to both exhaustive and greedy exploration strategies. We demonstrate the performance and capabilities of the proposed system in simulated and real-world outdoor experiments. To the best of our knowledge, this is the first work to address ground robot path planning using active aerial exploration

Predicting the Next Best View for 3D Mesh Refinement

3D reconstruction is a core task in many applications such as robot navigation or sites inspections. Finding the best poses to capture part of the scene is one of the most challenging topic that goes under the name of Next Best View. Recently, many volumetric methods have been proposed; they choose the Next Best View by reasoning over a 3D voxelized space and by finding which pose minimizes the uncertainty decoded into the voxels. Such methods are effective, but they do not scale well since the underlaying representation requires a huge amount of memory. In this paper we propose a novel mesh-based approach which focuses on the worst reconstructed region of the environment mesh. We define a photo-consistent index to evaluate the 3D mesh accuracy, and an energy function over the worst regions of the mesh which takes into account the mutual parallax with respect to the previous cameras, the angle of incidence of the viewing ray to the surface and the visibility of the region. We test our approach over a well known dataset and achieve state-of-the-art results.Comment: 13 pages, 5 figures, to be published in IAS-1

Active Autonomous Aerial Exploration for Ground Robot Path Planning

We address the problem of planning a path for a ground robot through unknown terrain, using observations from a flying robot. In search and rescue missions, which are our target scenarios, the time from arrival at the disaster site to the delivery of aid is critically important. Previous works required exhaustive exploration before path planning, which is time-consuming but eventually leads to an optimal path for the ground robot. Instead, we propose active exploration of the environment, where the flying robot chooses regions to map in a way that optimizes the overall response time of the system, which is the combined time for the air and ground robots to execute their missions. In our approach, we estimate terrain classes throughout our terrain map, and we also add elevation information in areas where the active exploration algorithm has chosen to perform 3-D reconstruction. This terrain information is used to estimate feasible and efficient paths for the ground robot. By exploring the environment actively, we achieve superior response times compared to both exhaustive and greedy exploration strategies. We demonstrate the performance and capabilities of the proposed system in simulated and real-world outdoor experiments. To the best of our knowledge, this is the first work to address ground robot path planning using active aerial exploration

Equivalencing the Collector System of a Large Wind Power Plant: Preprint

This paper focuses on our effort to develop an equivalent representation of a wind power plant collector system for power system planning studies

Model Validation at the 204 MW New Mexico Wind Energy Center: Preprint

In this paper, we describe methods to derive and validate equivalent models for a large wind farm. FPL Energy's 204-MW New Mexico Wind Energy Center, which is interconnected to the Public Service Company of New Mexico (PNM) transmission system, was used as a case study. The methods described are applicable to any large wind power plant

Model Validation at the 204-MW New Mexico Wind Energy Center

Poster for WindPower 2006 held June 4-7, 2006, in Pittsburgh, PA, describing model validation at the 204-MW New Mexico Wind Energy Center

Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Sounds

Humans can robustly recognize and localize objects by integrating visual and auditory cues. While machines are able to do the same now with images, less work has been done with sounds. This work develops an approach for dense semantic labelling of sound-making objects, purely based on binaural sounds. We propose a novel sensor setup and record a new audio-visual dataset of street scenes with eight professional binaural microphones and a 360 degree camera. The co-existence of visual and audio cues is leveraged for supervision transfer. In particular, we employ a cross-modal distillation framework that consists of a vision `teacher' method and a sound `student' method -- the student method is trained to generate the same results as the teacher method. This way, the auditory system can be trained without using human annotations. We also propose two auxiliary tasks namely, a) a novel task on Spatial Sound Super-resolution to increase the spatial resolution of sounds, and b) dense depth prediction of the scene. We then formulate the three tasks into one end-to-end trainable multi-tasking network aiming to boost the overall performance. Experimental results on the dataset show that 1) our method achieves promising results for semantic prediction and the two auxiliary tasks; and 2) the three tasks are mutually beneficial -- training them together achieves the best performance and 3) the number and orientations of microphones are both important. The data and code will be released to facilitate the research in this new direction.Comment: Project page: https://www.trace.ethz.ch/publications/2020/sound_perception/index.htm

The Lung Screen Uptake Trial (LSUT): protocol for a randomised controlled demonstration lung cancer screening pilot testing a targeted invitation strategy for high risk and ‘hard-to-reach’ patients

Background Participation in low-dose CT (LDCT) lung cancer screening offered in the trial context has been poor, especially among smokers from socioeconomically deprived backgrounds; a group for whom the risk-benefit ratio is improved due to their high risk of lung cancer. Attracting high risk participants is essential to the success and equity of any future screening programme. This study will investigate whether the observed low and biased uptake of screening can be improved using a targeted invitation strategy. Methods/design A randomised controlled trial design will be used to test whether targeted invitation materials are effective at improving engagement with an offer of lung cancer screening for high risk candidates. Two thousand patients aged 60–75 and recorded as a smoker within the last five years by their GP, will be identified from primary care records and individually randomised to receive either intervention invitation materials (which take a targeted, stepped and low burden approach to information provision prior to the appointment) or control invitation materials. The primary outcome is uptake of a nurse-led ‘lung health check’ hospital appointment, during which patients will be offered a spirometry test, an exhaled carbon monoxide (CO) reading, and an LDCT if eligible. Initial data on demographics (i.e. age, sex, ethnicity, deprivation score) and smoking status will be collected in primary care and analysed to explore differences between attenders and non-attenders with respect to invitation group. Those who attend the lung health check will have further data on smoking collected during their appointment (including pack-year history, nicotine dependence and confidence to quit). Secondary outcomes will include willingness to be screened, uptake of LDCT and measures of informed decision-making to ensure the latter is not compromised by either invitation strategy. Discussion If effective at improving informed uptake of screening and reducing bias in participation, this invitation strategy could be adopted by local screening pilots or a national programme. Trial registration This study was registered with the ISRCTN (International Standard Registered Clinical/soCial sTudy Number : ISRCTN21774741) on the 23rd September 2015 and the NIH ClinicalTrials.gov database (NCT0255810) on the 22nd September 2015