店舗カメラを用いた顧客関心度推定のための人物姿勢検知に関する研究

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 佐藤 洋一, 国立情報学研究所教授 佐藤 真一, 東京大学教授 苗村 健, 東京大学准教授 上條 俊介, 東京大学准教授 山崎 俊彦University of Tokyo(東京大学

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and head-worn wearables, and perform seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences using MAR devices to provide universal access to digital content. Over the past 20 years, several MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discuss the latest studies on MAR through a top-down approach: (1) MAR applications; (2) MAR visualisation techniques adaptive to user mobility and contexts; (3) systematic evaluation of MAR frameworks, including supported platforms and corresponding features such as tracking, feature extraction, and sensing capabilities; and (4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields and the current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike.Peer reviewe

Novel robust computer vision algorithms for micro autonomous systems

People detection and tracking are an essential component of many autonomous platforms, interactive systems and intelligent vehicles used in various search and rescues operations and similar humanitarian applications. Currently, researchers are focusing on the use of vision sensors such as cameras due to their advantages over other sensor types. Cameras are information rich, relatively inexpensive and easily available. Additionally, 3D information is obtained from stereo vision, or by triangulating over several frames in monocular configurations. Another method to obtain 3D data is by using RGB-D sensors (e.g. Kinect) that provide both image and depth data. This method is becoming more attractive over the past few years due to its affordable price and availability for researchers. The aim of this research was to find robust multi-target detection and tracking algorithms for Micro Autonomous Systems (MAS) that incorporate the use of the RGB-D sensor. Contributions include the discovery of novel robust computer vision algorithms. It proposed a new framework for human body detection, from video file, to detect a single person adapted from Viola and Jones framework. The 2D Multi Targets Detection and Tracking (MTDT) algorithm applied the Gaussian Mixture Model (GMM) to reduce noise in the pre-processing stage. Blob analysis was used to detect targets, and Kalman filter was used to track targets. The 3D MTDT extends beyond 2D with the use of depth data from the RGB-D sensor in the pre-processing stage. Bayesian model was employed to provide multiple cues. It includes detection of the upper body, face, skin colour, motion and shape. Kalman filter proved for speed and robustness of the track management. Simultaneous Localisation and Mapping (SLAM) fusing with 3D information was investigated. The new framework introduced front end and back end processing. The front end consists of localisation steps, post refinement and loop closing system. The back-end focus on the post-graph optimisation to eliminate errors.The proposed computer vision algorithms proved for better speed and robustness. The frameworks produced impressive results. New algorithms can be used to improve performances in real time applications including surveillance, vision navigation, environmental perception and vision-based control system on MAS

Probabilistic Models and Inference for Multi-View People Detection in Overlapping Depth Images

Die sensorübergreifende Personendetektion in einem Netzwerk von 3D-Sensoren ist die Grundlage vieler Anwendungen, wie z.B. Personenzählung, digitale Kundenstromanalyse oder öffentliche Sicherheit. Im Gegensatz zu klassischen Verfahren der Videoüberwachung haben 3D-Sensoren dabei im Allgemeinen eine vertikale top-down Sicht auf die Szene, um das Auftreten von Verdeckungen, wie sie z.B. in einer dicht gedrängten Menschenmenge auftreten, zu reduzieren. Aufgrund der vertikalen top-down Perspektive der Sensoren variiert die äußere Erscheinung von Personen sehr stark in Abhängigkeit von deren Position in der Szene. Des Weiteren sind Personen aufgrund von Verdeckungen, Sensorrauschen sowie dem eingeschränkten Sichtfeld der top-down Sensoren häufig nur partiell in einer einzelnen Ansicht sichtbar. Um diese Herausforderungen zu bewältigen, wird in dieser Arbeit untersucht, wie die räumlich-zeitlichen Multi-View-Beobachtungen von mehreren 3D-Sensoren mit sich überlappenden Sichtbereichen effektiv genutzt werden können. Der Fokus liegt insbesondere auf der Verbesserung der Detektionsleistung durch die gemeinsame Betrachtung sowohl der redundanten als auch der komplementären Multi-Sensor-Beobachtungen, einschließlich des zeitlichen Kontextes. In der Arbeit wird das Problem der Personendetektion in einer Sequenz sich überlappender Tiefenbilder als inverses Problem formuliert. In diesem Kontext wird ein probabilistisches Modell zur Personendetektion in mehreren Tiefenbildern eingeführt. Das Modell beinhaltet ein generatives Szenenmodell, um Personen aus beliebigen Blickwinkeln zu erkennen. Basierend auf der vorgeschlagenen probabilistischen Modellierung werden mehrere Inferenzmethoden untersucht, unter anderem Gradienten-basierte kontinuierliche Optimierung, Variational Inference, sowie Convolutional Neural Networks. Dabei liegt der Schwerpunkt der Arbeit auf dem Einsatz von Variationsmethoden wie Mean-Field Variational Inference. In Abgrenzung zu klassischen Verfahren der Literatur wird hier keine Punkt-Schätzung vorgenommen, sondern die a-posteriori Wahrscheinlichkeitsverteilung der in der Szene anwesenden Personen approximiert. Durch den Einsatz des generativen Vorwärtsmodells, welches die Charakteristik der zugrundeliegenden Sensormodalität beinhaltet, ist das vorgeschlagene Verfahren weitestgehend unabhängig von der konkreten Sensormodalität. Die in der Arbeit vorgestellten Methoden werden anhand eines neu eingeführten Datensatzes zur weitflächigen Personendetektion in mehreren sich überlappenden Tiefenbildern evaluiert. Der Datensatz umfasst Bildmaterial von drei passiven Stereo-Sensoren, welche eine top-down Sicht auf eine Bürosituation vorweisen. In der Evaluation konnte nachgewiesen werden, dass die vorgeschlagene Mean-Field Variational Inference Approximation Stand-der-Technik-Resultate erzielt. Während Deep Learnig Verfahren sehr viele annotierte Trainingsdaten benötigen, basiert die in dieser Arbeit vorgeschlagene Methode auf einem expliziten probabilistischen Modell und benötigt keine Trainingsdaten. Ein weiterer Vorteil zu klassischen Verfahren, welche häufig nur eine MAP Punkt-Schätzung vornehmen, besteht in der Approximation der vollständigen Verbund-Wahrscheinlichkeitsverteilung der in der Szene anwesenden Personen

CUSTARD (Cranfield University Space Technology Advanced Research Demonstrator) - A Micro-System Technology Demonstrator Nanosatellite. Summary of the Group Design Project MSc in Astronautics and Space Engineering. 1999-2000, Cranfield University

CUSTARD (Cranfield University Space Technology And Research Demonstrator) was the group design project for students of the MSc in Astronautics and Space Engineering for the Academic Year 1999/2000 at Cranfield University. The project involved the initial design of a nanosatellite to be used as a technology demonstrator for microsystem technology (MST) in space. The students worked together as one group (organised into several subgroups, e.g. system, mechanical), with each student responsible for a set of work packages. The nanosatellite designed had a mass of 4 kg, lifetime of 3 months in low Earth orbit, coarse 3-axis attitude control (no orbit control), and was capable of carrying up to 1 kg of payload. The electrical power available was 18 W (peak). Assuming a single X-band ground station at RAL (UK), a data rate of up to 1 M bit s-1 for about 3000 s per day is possible. The payloads proposed are a microgravity laboratory and a formation flying experiment. The report summarises the results of the project and includes executive summaries from all team members. Further information and summaries of the full reports are available from the College of Aeronautics, Cranfield University

Advanced flight control system study

The architecture, requirements, and system elements of an ultrareliable, advanced flight control system are described. The basic criteria are functional reliability of 10 to the minus 10 power/hour of flight and only 6 month scheduled maintenance. A distributed system architecture is described, including a multiplexed communication system, reliable bus controller, the use of skewed sensor arrays, and actuator interfaces. Test bed and flight evaluation program are proposed

Gaze and Peripheral Vision Analysis for Human-Environment Interaction: Applications in Automotive and Mixed-Reality Scenarios

This thesis studies eye-based user interfaces which integrate information about the user’s perceptual focus-of-attention into multimodal systems to enrich the interaction with the surrounding environment. We examine two new modalities: gaze input and output in the peripheral field of view. All modalities are considered in the whole spectrum of the mixed-reality continuum. We show the added value of these new forms of multimodal interaction in two important application domains: Automotive User Interfaces and Human-Robot Collaboration. We present experiments that analyze gaze under various conditions and help to design a 3D model for peripheral vision. Furthermore, this work presents several new algorithms for eye-based interaction, like deictic reference in mobile scenarios, for non-intrusive user identification, or exploiting the peripheral field view for advanced multimodal presentations. These algorithms have been integrated into a number of software tools for eye-based interaction, which are used to implement 15 use cases for intelligent environment applications. These use cases cover a wide spectrum of applications, from spatial interactions with a rapidly changing environment from within a moving vehicle, to mixed-reality interaction between teams of human and robots.In dieser Arbeit werden blickbasierte Benutzerschnittstellen untersucht, die Infor- mationen ¨uber das Blickfeld des Benutzers in multimodale Systeme integrieren, um neuartige Interaktionen mit der Umgebung zu erm¨oglichen. Wir untersuchen zwei neue Modalit¨aten: Blickeingabe und Ausgaben im peripheren Sichtfeld. Alle Modalit¨aten werden im gesamten Spektrum des Mixed-Reality-Kontinuums betra- chtet. Wir zeigen die Anwendung dieser neuen Formen der multimodalen Interak- tion in zwei wichtigen Dom¨anen auf: Fahrerassistenzsysteme und Werkerassistenz bei Mensch-Roboter-Kollaboration. Wir pr¨asentieren Experimente, die blickbasierte Benutzereingaben unter verschiedenen Bedingungen analysieren und helfen, ein 3D- Modell f¨ur das periphere Sehen zu entwerfen. Dar¨uber hinaus stellt diese Arbeit mehrere neue Algorithmen f¨ur die blickbasierte Interaktion vor, wie die deiktis- che Referenz in mobilen Szenarien, die nicht-intrusive Benutzeridentifikation, oder die Nutzung des peripheren Sichtfeldes f¨ur neuartige multimodale Pr¨asentationen. Diese Algorithmen sind in eine Reihe von Software-Werkzeuge integriert, mit de- nen 15 Anwendungsf¨alle f¨ur intelligente Umgebungen implementiert wurden. Diese Demonstratoren decken ein breites Anwendungsspektrum ab: von der r¨aumlichen In- teraktionen aus einem fahrenden Auto heraus bis hin zu Mixed-Reality-Interaktionen zwischen Mensch-Roboter-Teams

High-Speed Vision and Force Feedback for Motion-Controlled Industrial Manipulators

Over the last decades, both force sensors and cameras have emerged as useful sensors for different applications in robotics. This thesis considers a number of dynamic visual tracking and control problems, as well as the integration of these techniques with contact force control. Different topics ranging from basic theory to system implementation and applications are treated. A new interface developed for external sensor control is presented, designed by making non-intrusive extensions to a standard industrial robot control system. The structure of these extensions are presented, the system properties are modeled and experimentally verified, and results from force-controlled stub grinding and deburring experiments are presented. A novel system for force-controlled drilling using a standard industrial robot is also demonstrated. The solution is based on the use of force feedback to control the contact forces and the sliding motions of the pressure foot, which would otherwise occur during the drilling phase. Basic methods for feature-based tracking and servoing are presented, together with an extension for constrained motion estimation based on a dual quaternion pose parametrization. A method for multi-camera real-time rigid body tracking with time constraints is also presented, based on an optimal selection of the measured features. The developed tracking methods are used as the basis for two different approaches to vision/force control, which are illustrated in experiments. Intensity-based techniques for tracking and vision-based control are also developed. A dynamic visual tracking technique based directly on the image intensity measurements is presented, together with new stability-based methods suitable for dynamic tracking and feedback problems. The stability-based methods outperform the previous methods in many situations, as shown in simulations and experiments