Modeling and simulation of adaptive multimodal optical sensors for target tracking in the visible to near infrared

This work investigates an integrated aerial remote sensor design approach to address moving target detection and tracking problems within highly cluttered, dynamic ground-based scenes. Sophisticated simulation methodologies and scene phenomenology validations have resulted in advancements in artificial multimodal truth video synthesis. Complex modeling of novel micro-opto-electro-mechanical systems (MOEMS) devices, optical systems, and detector arrays has resulted in a proof of concept for a state-of-the-art imaging spectropolarimeter sensor model that does not suffer from typical multimodal image registration problems. Test methodology developed for this work provides the ability to quantify performance of a target tracking application with varying ground scenery, flight characteristics, or sensor specifications. The culmination of this research is an end-to-end simulated demonstration of multimodal aerial remote sensing and target tracking. Deeply hidden target recognition is shown to be enhanced through the fusing of panchromatic, hyperspectral, and polarimetric image modalities. The Digital Imaging and Remote Sensing Image Generation model was leveraged to synthesize truth spectropolarimetric sensor-reaching radiance image cubes comprised of coregistered Stokes vector bands in the visible to near-infrared. An intricate synthetic urban scene containing numerous moving vehicular targets was imaged from a virtual sensor aboard an aerial platform encircling a stare point. An adaptive sensor model was designed with a superpixel array of MOEMS devices fabricated atop a division of focal plane detector. Degree of linear polarization (DoLP) imagery is acquired by combining three adjacent micropolarizer outputs within each 2x2 superpixel whose respective transmissions vary with wavelength, relative angle of polarization, and wire-grid spacing. A novel micromirror within each superpixel adaptively relays light between a panchromatic imaging channel and a hyperspectral spectrometer channel. All optical and detector sensor effects were radiometrically modeled using MATLAB and optical lens design software. Orthorectification of all sensor outputs yields multimodal pseudonadir observation video at a fixed ground sampled distance across an area of responsibility. A proprietary MATLAB-based target tracker accomplishes change detection between sequential panchromatic or DoLP observation frames, and queries the sensor for hyperspectral pixels to aid in track initialization and maintenance. Image quality, spectral quality, and tracking performance metrics are reported for varying scenario parameters including target occlusions within the scene, declination angle and jitter of the aerial platform, micropolarizer diattenuation, and spectral/spatial resolution of the adaptive sensor outputs. DoLP observations were found to track moving vehicles better than panchromatic observations at high oblique angles when facing the sensor generally toward the sun. Vehicular occlusions due to tree canopies and parallax effects of tall buildings significantly reduced tracking performance as expected. Smaller MOEMS pixel sizes drastically improved track performance, but also generated a significant number of false tracks. Atmospheric haze from urban aerosols eliminated the tracking utility of DoLP observations, while aerial platform jitter without image stabilization eliminated tracking utility in both modalities. Wire-grid micropolarizers with very low VNIR diattenuation were found to still extinguish enough cross-polarized light to successfully distinguish and track moving vehicles from their urban background. Thus, state-of-the-art lithographic techniques to create finer wire-grid spacings that exhibit high VNIR diattenuation may not be required

Radiometer Calibration Using Colocated GPS Radio Occultation Measurements

We present a new high-fidelity method of calibrating a cross-track scanning microwave radiometer using Global Positioning System (GPS) radio occultation (GPSRO) measurements. The radiometer and GPSRO receiver periodically observe the same volume of atmosphere near the Earth's limb, and these overlapping measurements are used to calibrate the radiometer. Performance analyses show that absolute calibration accuracy better than 0.25 K is achievable for temperature sounding channels in the 50-60-GHz band for a total-power radiometer using a weakly coupled noise diode for frequent calibration and proximal GPSRO measurements for infrequent (approximately daily) calibration. The method requires GPSRO penetration depth only down to the stratosphere, thus permitting the use of a relatively small GPS antenna. Furthermore, only coarse spacecraft angular knowledge (approximately one degree rms) is required for the technique, as more precise angular knowledge can be retrieved directly from the combined radiometer and GPSRO data, assuming that the radiometer angular sampling is uniform. These features make the technique particularly well suited for implementation on a low-cost CubeSat hosting both radiometer and GPSRO receiver systems on the same spacecraft. We describe a validation platform for this calibration method, the Microwave Radiometer Technology Acceleration (MiRaTA) CubeSat, currently in development for the National Aeronautics and Space Administration (NASA) Earth Science Technology Office. MiRaTA will fly a multiband radiometer and the Compact TEC/Atmosphere GPS Sensor in 2015.United States. Dept. of Defense. Assistant Secretary of Defense for Research & Engineering (United States. Air Force Contract FA8721-05-C-0002

A landmark recognition and tracking experiment for flight on the Shuttle/Advanced Technology Laboratory (ATL)

The preliminary design of an experiment for landmark recognition and tracking from the Shuttle/Advanced Technology Laboratory is described. It makes use of parallel coherent optical processing to perform correlation tests between landmarks observed passively with a telescope and previously made holographic matched filters. The experimental equipment including the optics, the low power laser, the random access file of matched filters and the electro-optical readout device are described. A real time optically excited liquid crystal device is recommended for performing the input non-coherent optical to coherent optical interface function. A development program leading to a flight experiment in 1981 is outlined

Experimental Analysis of RSSI-based Indoor Location Systems with WL Circularly Polarized Antennas

Circularly polarized antennas are used in 2.4 GHz ZigBee radio modules to evaluate performance improvement of RSSI (Received Signal Strength Indicator) based location techniques, with respect to conventional linearly polarized antennas. Experimental RSSI measurements in an indoor environment clearly show that multipath fading is significantly reduced when CP antennas are used; this determines a more reliable estimation of the field amplitude decay law as a function of the distance of the mobile node from the fixed access point, and then a higher location accuracy. At the best of authors' knowledge, it is the first time that the circular polarization features are applied to RSSI-based radio location techniques

A continuum robotic platform for endoscopic non-contact laser surgery: design, control, and preclinical evaluation

The application of laser technologies in surgical interventions has been accepted in the clinical domain due to their atraumatic properties. In addition to manual application of fibre-guided lasers with tissue contact, non-contact transoral laser microsurgery (TLM) of laryngeal tumours has been prevailed in ENT surgery. However, TLM requires many years of surgical training for tumour resection in order to preserve the function of adjacent organs and thus preserve the patient’s quality of life. The positioning of the microscopic laser applicator outside the patient can also impede a direct line-of-sight to the target area due to anatomical variability and limit the working space. Further clinical challenges include positioning the laser focus on the tissue surface, imaging, planning and performing laser ablation, and motion of the target area during surgery. This dissertation aims to address the limitations of TLM through robotic approaches and intraoperative assistance. Although a trend towards minimally invasive surgery is apparent, no highly integrated platform for endoscopic delivery of focused laser radiation is available to date. Likewise, there are no known devices that incorporate scene information from endoscopic imaging into ablation planning and execution. For focusing of the laser beam close to the target tissue, this work first presents miniaturised focusing optics that can be integrated into endoscopic systems. Experimental trials characterise the optical properties and the ablation performance. A robotic platform is realised for manipulation of the focusing optics. This is based on a variable-length continuum manipulator. The latter enables movements of the endoscopic end effector in five degrees of freedom with a mechatronic actuation unit. The kinematic modelling and control of the robot are integrated into a modular framework that is evaluated experimentally. The manipulation of focused laser radiation also requires precise adjustment of the focal position on the tissue. For this purpose, visual, haptic and visual-haptic assistance functions are presented. These support the operator during teleoperation to set an optimal working distance. Advantages of visual-haptic assistance are demonstrated in a user study. The system performance and usability of the overall robotic system are assessed in an additional user study. Analogous to a clinical scenario, the subjects follow predefined target patterns with a laser spot. The mean positioning accuracy of the spot is 0.5 mm. Finally, methods of image-guided robot control are introduced to automate laser ablation. Experiments confirm a positive effect of proposed automation concepts on non-contact laser surgery.Die Anwendung von Lasertechnologien in chirurgischen Interventionen hat sich aufgrund der atraumatischen Eigenschaften in der Klinik etabliert. Neben manueller Applikation von fasergeführten Lasern mit Gewebekontakt hat sich die kontaktfreie transorale Lasermikrochirurgie (TLM) von Tumoren des Larynx in der HNO-Chirurgie durchgesetzt. Die TLM erfordert zur Tumorresektion jedoch ein langjähriges chirurgisches Training, um die Funktion der angrenzenden Organe zu sichern und damit die Lebensqualität der Patienten zu erhalten. Die Positionierung des mikroskopis chen Laserapplikators außerhalb des Patienten kann zudem die direkte Sicht auf das Zielgebiet durch anatomische Variabilität erschweren und den Arbeitsraum einschränken. Weitere klinische Herausforderungen betreffen die Positionierung des Laserfokus auf der Gewebeoberfläche, die Bildgebung, die Planung und Ausführung der Laserablation sowie intraoperative Bewegungen des Zielgebietes. Die vorliegende Dissertation zielt darauf ab, die Limitierungen der TLM durch robotische Ansätze und intraoperative Assistenz zu adressieren. Obwohl ein Trend zur minimal invasiven Chirurgie besteht, sind bislang keine hochintegrierten Plattformen für die endoskopische Applikation fokussierter Laserstrahlung verfügbar. Ebenfalls sind keine Systeme bekannt, die Szeneninformationen aus der endoskopischen Bildgebung in die Ablationsplanung und -ausführung einbeziehen. Für eine situsnahe Fokussierung des Laserstrahls wird in dieser Arbeit zunächst eine miniaturisierte Fokussieroptik zur Integration in endoskopische Systeme vorgestellt. Experimentelle Versuche charakterisieren die optischen Eigenschaften und das Ablationsverhalten. Zur Manipulation der Fokussieroptik wird eine robotische Plattform realisiert. Diese basiert auf einem längenveränderlichen Kontinuumsmanipulator. Letzterer ermöglicht in Kombination mit einer mechatronischen Aktuierungseinheit Bewegungen des Endoskopkopfes in fünf Freiheitsgraden. Die kinematische Modellierung und Regelung des Systems werden in ein modulares Framework eingebunden und evaluiert. Die Manipulation fokussierter Laserstrahlung erfordert zudem eine präzise Anpassung der Fokuslage auf das Gewebe. Dafür werden visuelle, haptische und visuell haptische Assistenzfunktionen eingeführt. Diese unterstützen den Anwender bei Teleoperation zur Einstellung eines optimalen Arbeitsabstandes. In einer Anwenderstudie werden Vorteile der visuell-haptischen Assistenz nachgewiesen. Die Systemperformanz und Gebrauchstauglichkeit des robotischen Gesamtsystems werden in einer weiteren Anwenderstudie untersucht. Analog zu einem klinischen Einsatz verfolgen die Probanden mit einem Laserspot vorgegebene Sollpfade. Die mittlere Positioniergenauigkeit des Spots beträgt dabei 0,5 mm. Zur Automatisierung der Ablation werden abschließend Methoden der bildgestützten Regelung vorgestellt. Experimente bestätigen einen positiven Effekt der Automationskonzepte für die kontaktfreie Laserchirurgie

Panoramic, large-screen, 3-D flight display system design

The report documents and summarizes the results of the required evaluations specified in the SOW and the design specifications for the selected display system hardware. Also included are the proposed development plan and schedule as well as the estimated rough order of magnitude (ROM) cost to design, fabricate, and demonstrate a flyable prototype research flight display system. The thrust of the effort was development of a complete understanding of the user/system requirements for a panoramic, collimated, 3-D flyable avionic display system and the translation of the requirements into an acceptable system design for fabrication and demonstration of a prototype display in the early 1997 time frame. Eleven display system design concepts were presented to NASA LaRC during the program, one of which was down-selected to a preferred display system concept. A set of preliminary display requirements was formulated. The state of the art in image source technology, 3-D methods, collimation methods, and interaction methods for a panoramic, 3-D flight display system were reviewed in depth and evaluated. Display technology improvements and risk reductions associated with maturity of the technologies for the preferred display system design concept were identified

Metasurface-enhanced Light Detection and Ranging Technology

Deploying advanced imaging solutions to robotic and autonomous systems by mimicking human vision requires simultaneous acquisition of multiple fields of views, named the peripheral and fovea regions. Low-resolution peripheral field provides coarse scene exploration to direct the eye to focus to a highly resolved fovea region for sharp imaging. Among 3D computer vision techniques, Light Detection and Ranging (LiDAR) is currently considered at the industrial level for robotic vision. LiDAR is an imaging technique that monitors pulses of light at optical frequencies to sense the space and to recover three-dimensional ranging information. Notwithstanding the efforts on LiDAR integration and optimization, commercially available devices have slow frame rate and low image resolution, notably limited by the performance of mechanical or slow solid-state deflection systems. Metasurfaces (MS) are versatile optical components that can distribute the optical power in desired regions of space. Here, we report on an advanced LiDAR technology that uses ultrafast low FoV deflectors cascaded with large area metasurfaces to achieve large FoV and simultaneous peripheral and central imaging zones. This technology achieves MHz frame rate for 2D imaging, and up to KHz for 3D imaging, with extremely large FoV (up to 150{\deg}deg. on both vertical and horizontal scanning axes). The use of this disruptive LiDAR technology with advanced learning algorithms offers perspectives to improve further the perception capabilities and decision-making process of autonomous vehicles and robotic systems.Comment: 25pages, 18 figures. Including supplementary material