Address-Event Based Stereo Vision with Bio-Inspired Silicon Retina Imagers

Artificial intelligenc

Arctic Domain Awareness Center DHS Center of Excellence (COE): Project Work Plan

As stated by the DHS Science &Technology Directorate, “The increased and diversified use of maritime spaces in the Arctic - including oil and gas exploration, commercial activities, mineral speculation, and recreational activities (tourism) - is generating new challenges and risks for the U.S. Coast Guard and other DHS maritime missions.” Therefore, DHS will look towards the new ADAC for research to identify better ways to create transparency in the maritime domain along coastal regions and inland waterways, while integrating information and intelligence among stakeholders. DHS expects the ADAC to develop new ideas to address these challenges, provide a scientific basis, and develop new approaches for U.S. Coast Guard and other DHS maritime missions. ADAC will also contribute towards the education of both university students and mid-career professionals engaged in maritime security. The US is an Arctic nation, and the Arctic environment is dynamic. We have less multi-year ice and more open water during the summer causing coastal villages to experience unprecedented storm surges and coastal erosion. Decreasing sea ice is also driving expanded oil exploration, bringing risks of oil spills. Tourism is growing rapidly, and our fishing fleet and commercial shipping activities are increasing as well. There continues to be anticipation of an economic pressure to open up a robust northwest passage for commercial shipping. To add to the stresses of these changes is the fact that these many varied activities are spread over an immense area with little connecting infrastructure. The related maritime security issues are many, and solutions demand increasing maritime situational awareness and improved crisis response capabilities, which are the focuses of our Work Plan. UAA understands the needs and concerns of the Arctic community. It is situated on Alaska’s Southcentral coast with the port facility through which 90% of goods for Alaska arrive. It is one of nineteen US National Strategic Seaports for the US DOD, and its airport is among the top five in the world for cargo throughput. However, maritime security is a national concern and although our focus is on the Arctic environment, we will expand our scope to include other areas in the Lower 48 states. In particular, we will develop sensor systems, decision support tools, ice and oil spill models that include oil in ice, and educational programs that are applicable to the Arctic as well as to the Great Lakes and Northeast. The planned work as detailed in this document addresses the DHS mission as detailed in the National Strategy for Maritime Security, in particular, the mission to Maximize Domain Awareness (pages 16 and 17.) This COE will produce systems to aid in accomplishing two of the objectives of this mission. They are: 1) Sensor Technology developing sensor packages for airborne, underwater, shore-based, and offshore platforms, and 2) Automated fusion and real-time simulation and modeling systems for decision support and planning. An integral part of our efforts will be to develop new methods for sharing of data between platforms, sensors, people, and communities.United States Department of Homeland SecurityCOE ADAC Objective/Purpose / Methodology / Center Management Team and Partners / Evaluation and Transition Plans / USCG Stakeholder Engagement / Workforce Development Strategy / Individual Work Plan by Projects Within a Theme / Appendix A / Appendix B / Appendix

Neuromorphic stereo vision: A survey of bio-inspired sensors and algorithms

Any visual sensor, whether artificial or biological, maps the 3D-world on a 2D-representation. The missing dimension is depth and most species use stereo vision to recover it. Stereo vision implies multiple perspectives and matching, hence it obtains depth from a pair of images. Algorithms for stereo vision are also used prosperously in robotics. Although, biological systems seem to compute disparities effortless, artificial methods suffer from high energy demands and latency. The crucial part is the correspondence problem; finding the matching points of two images. The development of event-based cameras, inspired by the retina, enables the exploitation of an additional physical constraint—time. Due to their asynchronous course of operation, considering the precise occurrence of spikes, Spiking Neural Networks take advantage of this constraint. In this work, we investigate sensors and algorithms for event-based stereo vision leading to more biologically plausible robots. Hereby, we focus mainly on binocular stereo vision

Event-Driven Technologies for Reactive Motion Planning: Neuromorphic Stereo Vision and Robot Path Planning and Their Application on Parallel Hardware

Die Robotik wird immer mehr zu einem Schlüsselfaktor des technischen Aufschwungs. Trotz beeindruckender Fortschritte in den letzten Jahrzehnten, übertreffen Gehirne von Säugetieren in den Bereichen Sehen und Bewegungsplanung noch immer selbst die leistungsfähigsten Maschinen. Industrieroboter sind sehr schnell und präzise, aber ihre Planungsalgorithmen sind in hochdynamischen Umgebungen, wie sie für die Mensch-Roboter-Kollaboration (MRK) erforderlich sind, nicht leistungsfähig genug. Ohne schnelle und adaptive Bewegungsplanung kann sichere MRK nicht garantiert werden. Neuromorphe Technologien, einschließlich visueller Sensoren und Hardware-Chips, arbeiten asynchron und verarbeiten so raum-zeitliche Informationen sehr effizient. Insbesondere ereignisbasierte visuelle Sensoren sind konventionellen, synchronen Kameras bei vielen Anwendungen bereits überlegen. Daher haben ereignisbasierte Methoden ein großes Potenzial, schnellere und energieeffizientere Algorithmen zur Bewegungssteuerung in der MRK zu ermöglichen. In dieser Arbeit wird ein Ansatz zur flexiblen reaktiven Bewegungssteuerung eines Roboterarms vorgestellt. Dabei wird die Exterozeption durch ereignisbasiertes Stereosehen erreicht und die Pfadplanung ist in einer neuronalen Repräsentation des Konfigurationsraums implementiert. Die Multiview-3D-Rekonstruktion wird durch eine qualitative Analyse in Simulation evaluiert und auf ein Stereo-System ereignisbasierter Kameras übertragen. Zur Evaluierung der reaktiven kollisionsfreien Online-Planung wird ein Demonstrator mit einem industriellen Roboter genutzt. Dieser wird auch für eine vergleichende Studie zu sample-basierten Planern verwendet. Ergänzt wird dies durch einen Benchmark von parallelen Hardwarelösungen wozu als Testszenario Bahnplanung in der Robotik gewählt wurde. Die Ergebnisse zeigen, dass die vorgeschlagenen neuronalen Lösungen einen effektiven Weg zur Realisierung einer Robotersteuerung für dynamische Szenarien darstellen. Diese Arbeit schafft eine Grundlage für neuronale Lösungen bei adaptiven Fertigungsprozesse, auch in Zusammenarbeit mit dem Menschen, ohne Einbußen bei Geschwindigkeit und Sicherheit. Damit ebnet sie den Weg für die Integration von dem Gehirn nachempfundener Hardware und Algorithmen in die Industrierobotik und MRK

System management algorithms for distributed vision networks

[no abstract

Representation and recognition of action in interactive spaces

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.Includes bibliographical references (p. 246-258).This thesis presents new theory and technology for the representation and recognition of complex, context-sensitive human actions in interactive spaces. To represent action and interaction a symbolic framework has been developed based on Roger Schank's conceptualizations, augmented by a mechanism to represent the temporal structure of the sub-actions based on Allen's interval algebra networks. To overcome the exponential nature of temporal constraint propagation in such networks, we have developed the PNF propagation algorithm based on the projection of IA-networks into simplified, 3-valued (past, now, future) constraint networks called PNF-networks. The PNF propagation algorithm has been applied to an action recognition vision system that handles actions composed of multiple, parallel threads of sub-actions, in situations that can not be efficiently dealt by the commonly used temporal representation schemes such as finite-state machines and HMMs. The PNF propagation algorithm is also the basis of interval scripts, a scripting paradigm for interactive systems that represents interaction as a set of temporal constraints between the individual components of the interaction. Unlike previously proposed non-procedural scripting methods, we use a strong temporal representation (allowing, for example, mutually exclusive actions) and perform control by propagating the temporal constraints in real-time. These concepts have been tested in the context of four projects involving story-driven interactive spaces. The action representation framework has been used in the Intelligent Studio project to enhance the control of automatic cameras in a TV studio. Interval scripts have been extensively employed in the development of "SingSong ", a short interactive performance that introduced the idea of live interaction with computer graphics characters; in "It/I", a full-length computer theater play; and in "It", an interactive art installation based on the play "It /I" that realizes our concept of immersive stages, that is, interactive spaces that can be used both by performers and public.by Claudio Santos Pinhanez.Ph.D

The role of groups in smart camera networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006.Includes bibliographical references (p. 103-111).Recent research in sensor networks has made it possible to deploy networks of sensors with significant local processing. These sensor networks are revolutionising information collection and processing in many different environments. Often the amount of local data produced by these devices, and their sheer number, makes centralised data processing infeasible. Smart camera networks represent a particular challenge in this regard, partly because of the amount of data produced by each camera, but also because many high level vision algorithms require data from more than one camera. Many distributed algorithms exist that work locally to produce results from a collection of nodes, but as this number grows the algorithm's performance is quickly crippled by the resulting exponential increase in communication overhead. This thesis examines the limits this puts on peer-to-peer cooperation between nodes, and demonstrates how for large networks these can only be circumvented by locally formed organisations of nodes. A local group forming protocol is described that provides a method for nodes to create a bottom-up organisation based purely on local conditions. This allows the formation of a dynamic information network of cooperating nodes, in which a distributed algorithm can organise the communications of its nodes using purely local knowledge to maintain its global network performance.(cont.) Building on recent work using SIFT feature detection, this protocol is demonstrated in a network of smart cameras. Local groups with shared views are established, which allow each camera to locally determine their relative position with others in the network. The result partitions the network into groups of cameras with known visual relationships, which can then be used for further analysis.by Jacky Mallett.Ph.D

NASA Tech Briefs, February 1996

Topics covered include: Materials; Computer Programs; Mechanics; Machinery/Automation; Manufacturing/Fabrication; Mathematics and Information Sciences; Life Sciences; Books and Reports