Shape from inconsistent silhouette: Reconstruction of objects in the presence of segmentation and camera calibration error

Silhouettes are useful features to reconstruct the object shape when the object is textureless or the shape classes of objects are unknown. In this dissertation, we explore the problem of reconstructing the shape of challenging objects from silhouettes under real-world conditions such as the presence of silhouette and camera calibration error. This problem is called the Shape from Inconsistent Silhouettes problem. A psuedo-Boolean cost function is formalized for this problem, which penalizes differences between the reconstruction images and the silhouette images, and the Shape from Inconsistent Silhouette problem is cast as a psuedo-Boolean minimization problem. We propose a memory and time efficient method to find a local minimum solution to the optimization problem, including heuristics that take into account the geometric nature of the problem. Our methods are demonstrated on a variety of challenging objects including humans and large, thin objects. We also compare our methods to the state-of-the-art by generating reconstructions of synthetic objects with induced error. ^ We also propose a method for correcting camera calibration error given silhouettes with segmentation error. Unlike other existing methods, our method allows camera calibration error to be corrected without camera placement constraints and allows for silhouette segmentation error. This is accomplished by a modified Iterative Closest Point algorithm which minimizes the difference between an initial reconstruction and the input silhouettes. We characterize the degree of error that can be corrected with synthetic datasets with increasing error, and demonstrate the ability of the camera calibration correction method in improving the reconstruction quality in several challenging real-world datasets

UAV as a Reliable Wingman: A Flight Demonstration

In this brief, we present the results from a flight experiment demonstrating two significant advances in software enabled control: optimization-based control using real-time trajectory generation and logical programming environments for formal analysis of control software. Our demonstration platform consisted of a human-piloted F-15 jet flying together with an autonomous T-33 jet. We describe the behavior of the system in two scenarios. In the first, nominal state communications were present and the autonomous aircraft maintained formation as the human pilot flew maneuvers. In the second, we imposed the loss of high-rate communications and demonstrated an autonomous safe “lost wingman” procedure to increase separation and reacquire contact. The flight demonstration included both a nominal formation flight component and an execution of the lost wingman scenario

Online learning with stability guarantees: A memory-based real-time model predictive controller

We propose and analyze a real-time model predictive control (MPC) scheme that utilizes stored data to improve its performance by learning the value function online with stability guarantees. For linear and nonlinear systems, a learning method is presented that makes use of basic analytic properties of the cost function and is proven to learn the MPC control law and the value function on the limit set of the closed-loop state trajectory. The main idea is to generate a smart warm start based on historical data that improves future data points and thus future warm starts. We show that these warm starts are asymptotically exact and converge to the solution of the MPC optimization problem. Thereby, the suboptimality of the applied control input resulting from the real-time requirements vanishes over time. Simulative examples show that existing real-time MPC schemes can be improved by storing data and the proposed learning scheme.Comment: This article is an extended version of the paper "Online learning with stability guarantees: A memory-based warm starting for real-time MPC" published in Automatica, Volume 122, 109247, 2020, including all proofs, an application example, and a detailed description of the used algorith

Robust Multi-Person Tracking from Moving Platforms

In this paper, we address the problem of multi-person tracking in busy pedestrian zones, using a stereo rig mounted on a mobile platform. The complexity of the problem calls for an integrated solution, which extracts as much visual information as possible and combines it through cognitive feedback. We propose such an approach, which jointly estimates camera position, stereo depth, object detection, and tracking. We model the interplay between these components using a graphical model. Since the model has to incorporate object-object interactions, and temporal links to past frames, direct inference is intractable. We therefore propose a two-stage procedure: for each frame we first solve a simplified version of the model (disregarding interactions and temporal continuity) to estimate the scene geometry and an overcomplete set of object detections. Conditioned on these results, we then address object interactions, tracking, and prediction in a second step. The approach is experimentally evaluated on several long and difficult video sequences from busy inner-city locations. Our results show that the proposed integration makes it possible to deliver stable tracking performance in scenes of realistic complexity