Visual Perception For Robotic Spatial Understanding

Humans understand the world through vision without much effort. We perceive the structure, objects, and people in the environment and pay little direct attention to most of it, until it becomes useful. Intelligent systems, especially mobile robots, have no such biologically engineered vision mechanism to take for granted. In contrast, we must devise algorithmic methods of taking raw sensor data and converting it to something useful very quickly. Vision is such a necessary part of building a robot or any intelligent system that is meant to interact with the world that it is somewhat surprising we don\u27t have off-the-shelf libraries for this capability. Why is this? The simple answer is that the problem is extremely difficult. There has been progress, but the current state of the art is impressive and depressing at the same time. We now have neural networks that can recognize many objects in 2D images, in some cases performing better than a human. Some algorithms can also provide bounding boxes or pixel-level masks to localize the object. We have visual odometry and mapping algorithms that can build reasonably detailed maps over long distances with the right hardware and conditions. On the other hand, we have robots with many sensors and no efficient way to compute their relative extrinsic poses for integrating the data in a single frame. The same networks that produce good object segmentations and labels in a controlled benchmark still miss obvious objects in the real world and have no mechanism for learning on the fly while the robot is exploring. Finally, while we can detect pose for very specific objects, we don\u27t yet have a mechanism that detects pose that generalizes well over categories or that can describe new objects efficiently. We contribute algorithms in four of the areas mentioned above. First, we describe a practical and effective system for calibrating many sensors on a robot with up to 3 different modalities. Second, we present our approach to visual odometry and mapping that exploits the unique capabilities of RGB-D sensors to efficiently build detailed representations of an environment. Third, we describe a 3-D over-segmentation technique that utilizes the models and ego-motion output in the previous step to generate temporally consistent segmentations with camera motion. Finally, we develop a synthesized dataset of chair objects with part labels and investigate the influence of parts on RGB-D based object pose recognition using a novel network architecture we call PartNet

Think You Know Ketchup, Think Again

This project was submitted in partial fulfillment of the requirements for the Master of Science in Journalism degree

National Football League Skilled and Unskilled Positions Vary in Opportunity and Yield in Return to Play After an Anterior Cruciate Ligament Injury.

BACKGROUND: Anterior cruciate ligament (ACL) injuries pose a significant risk to the careers of players in the National Football League (NFL). The relationships between draft round and position on return to play (RTP) among NFL players are not well understood, and the ability to return to preinjury performance levels remains unknown for most positions. PURPOSE: To test for differences in RTP rates and changes in performance after an ACL injury by position and draft round. We hypothesized that skilled positions would return at a lower rate compared to unskilled positions. We further hypothesized that early draft-round status would relate to a greater rate of RTP and that skilled positions and a lower draft round would correlate with decreased performance for players who return to sport. STUDY DESIGN: Case-control study; Level of evidence, 3. METHODS: Utilizing a previously established database of publicly available information regarding ACL tears among NFL players, athletes with ACL tears occurring between the 2010 and 2013 seasons were identified. Generalized linear models and Kaplan-Meier time-to-event models were used to test the study hypotheses. RESULTS: The overall RTP rate was 61.7%, with skilled players and unskilled players returning at rates of 64.1% and 60.4%, respectively (P = .74). Early draft-round players and unskilled late draft-round players had greater rates of RTP compared to skilled late draft-round players and both unskilled and skilled undrafted free agents (UDFAs). Skilled early draft-round players constituted the only cohort that played significantly fewer games after an injury. Unskilled UDFAs constituted the only cohort to show a significant increase in the number of games started and ratio of games started to games played, starting more games in which they played, after an injury. CONCLUSION: Early draft-round and unskilled players were more likely to return compared to their later draft-round and skilled peers. Skilled early draft-round players, who displayed relatively high rates of RTP, constituted the only cohort to show a decline in performance. Unskilled UDFAs, who exhibited relatively low rates of RTP, constituted the only cohort to show an increase in performance. The significant effect of draft round and position type on RTP may be caused by a combination of differences in talent levels and in opportunities given to returning to play

Detection of Active Emergency Vehicles using Per-Frame CNNs and Output Smoothing

While inferring common actor states (such as position or velocity) is an important and well-explored task of the perception system aboard a self-driving vehicle (SDV), it may not always provide sufficient information to the SDV. This is especially true in the case of active emergency vehicles (EVs), where light-based signals also need to be captured to provide a full context. We consider this problem and propose a sequential methodology for the detection of active EVs, using an off-the-shelf CNN model operating at a frame level and a downstream smoother that accounts for the temporal aspect of flashing EV lights. We also explore model improvements through data augmentation and training with additional hard samples

Active foundering of a continental arc root beneath the southern Sierra Nevada in California

Seismic data provide images of crust–mantle interactions during ongoing removal of the dense batholithic root beneath the southern Sierra Nevada mountains in California. The removal appears to have initiated between 10 and 3 Myr ago with a Rayleigh–Taylor-type instability, but with a pronounced asymmetric flow into a mantle downwelling (drip) beneath the adjacent Great Valley. A nearly horizontal shear zone accommodated the detachment of the ultramafic root from its granitoid batholith. With continuing flow into the mantle drip, viscous drag at the base of the remaining ~35-km-thick crust has thickened the crust by ~7 km in a narrow welt beneath the western flank of the range. Adjacent to the welt and at the top of the drip, a V-shaped cone of crust is being dragged down tens of kilometres into the core of the mantle drip, causing the disappearance of the Moho in the seismic images. Viscous coupling between the crust and mantle is therefore apparently driving present-day surface subsidence