Distant pointing in desktop collaborative virtual environments

Deictic pointing—pointing at things during conversations—is natural and ubiquitous in human communication. Deictic pointing is important in the real world; it is also important in collaborative virtual environments (CVEs) because CVEs are 3D virtual environments that resemble the real world. CVEs connect people from different locations, allowing them to communicate and collaborate remotely. However, the interaction and communication capabilities of CVEs are not as good as those in the real world. In CVEs, people interact with each other using avatars (the visual representations of users). One problem of avatars is that they are not expressive enough when compare to what we can do in the real world. In particular, deictic pointing has many limitations and is not well supported. This dissertation focuses on improving the expressiveness of distant pointing—where referents are out of reach—in desktop CVEs. This is done by developing a framework that guides the design and development of pointing techniques; by identifying important aspects of distant pointing through observation of how people point at distant referents in the real world; by designing, implementing, and evaluating distant-pointing techniques; and by providing a set of guidelines for the design of distant pointing in desktop CVEs. The evaluations of distant-pointing techniques examine whether pointing without extra visual effects (natural pointing) has sufficient accuracy; whether people can control free arm movement (free pointing) along with other avatar actions; and whether free and natural pointing are useful and valuable in desktop CVEs. Overall, this research provides better support for deictic pointing in CVEs by improving the expressiveness of distant pointing. With better pointing support, gestural communication can be more effective and can ultimately enhance the primary function of CVEs—supporting distributed collaboration

Enhanced Virtuality: Increasing the Usability and Productivity of Virtual Environments

Mit stetig steigender Bildschirmauflösung, genauerem Tracking und fallenden Preisen stehen Virtual Reality (VR) Systeme kurz davor sich erfolgreich am Markt zu etablieren. Verschiedene Werkzeuge helfen Entwicklern bei der Erstellung komplexer Interaktionen mit mehreren Benutzern innerhalb adaptiver virtueller Umgebungen. Allerdings entstehen mit der Verbreitung der VR-Systeme auch zusätzliche Herausforderungen: Diverse Eingabegeräte mit ungewohnten Formen und Tastenlayouts verhindern eine intuitive Interaktion. Darüber hinaus zwingt der eingeschränkte Funktionsumfang bestehender Software die Nutzer dazu, auf herkömmliche PC- oder Touch-basierte Systeme zurückzugreifen. Außerdem birgt die Zusammenarbeit mit anderen Anwendern am gleichen Standort Herausforderungen hinsichtlich der Kalibrierung unterschiedlicher Trackingsysteme und der Kollisionsvermeidung. Beim entfernten Zusammenarbeiten wird die Interaktion durch Latenzzeiten und Verbindungsverluste zusätzlich beeinflusst. Schließlich haben die Benutzer unterschiedliche Anforderungen an die Visualisierung von Inhalten, z.B. Größe, Ausrichtung, Farbe oder Kontrast, innerhalb der virtuellen Welten. Eine strikte Nachbildung von realen Umgebungen in VR verschenkt Potential und wird es nicht ermöglichen, die individuellen Bedürfnisse der Benutzer zu berücksichtigen. Um diese Probleme anzugehen, werden in der vorliegenden Arbeit Lösungen in den Bereichen Eingabe, Zusammenarbeit und Erweiterung von virtuellen Welten und Benutzern vorgestellt, die darauf abzielen, die Benutzerfreundlichkeit und Produktivität von VR zu erhöhen. Zunächst werden PC-basierte Hardware und Software in die virtuelle Welt übertragen, um die Vertrautheit und den Funktionsumfang bestehender Anwendungen in VR zu erhalten. Virtuelle Stellvertreter von physischen Geräten, z.B. Tastatur und Tablet, und ein VR-Modus für Anwendungen ermöglichen es dem Benutzer reale Fähigkeiten in die virtuelle Welt zu übertragen. Des Weiteren wird ein Algorithmus vorgestellt, der die Kalibrierung mehrerer ko-lokaler VR-Geräte mit hoher Genauigkeit und geringen Hardwareanforderungen und geringem Aufwand ermöglicht. Da VR-Headsets die reale Umgebung der Benutzer ausblenden, wird die Relevanz einer Ganzkörper-Avatar-Visualisierung für die Kollisionsvermeidung und das entfernte Zusammenarbeiten nachgewiesen. Darüber hinaus werden personalisierte räumliche oder zeitliche Modifikationen vorgestellt, die es erlauben, die Benutzerfreundlichkeit, Arbeitsleistung und soziale Präsenz von Benutzern zu erhöhen. Diskrepanzen zwischen den virtuellen Welten, die durch persönliche Anpassungen entstehen, werden durch Methoden der Avatar-Umlenkung (engl. redirection) kompensiert. Abschließend werden einige der Methoden und Erkenntnisse in eine beispielhafte Anwendung integriert, um deren praktische Anwendbarkeit zu verdeutlichen. Die vorliegende Arbeit zeigt, dass virtuelle Umgebungen auf realen Fähigkeiten und Erfahrungen aufbauen können, um eine vertraute und einfache Interaktion und Zusammenarbeit von Benutzern zu gewährleisten. Darüber hinaus ermöglichen individuelle Erweiterungen des virtuellen Inhalts und der Avatare Einschränkungen der realen Welt zu überwinden und das Erlebnis von VR-Umgebungen zu steigern

Robot Teardown, Stripping Industrial Robots for Good

Building a robot requires a careful selection of components that interact across networks while meeting timing deadlines. Given the complexity associated, as robots get damaged or security compromised, their components will increasingly require updates and replacements. Contrary to the expectations and similar to Ford in the 1920s with cars, most robot manufacturers oppose to this. They employ planned obsolescence practices organizing dealers and system integrators into "private networks", providing repair parts only to "certified" companies to discourage repairs and evade competition. In this article, we introduce and advocate for robot teardown as an approach to study robot hardware architectures and fuel security research. We show how teardown can help understanding the underlying hardware and demonstrate how our approach can help researchers uncovering security vulnerabilities. Our case studies show how robot teardown becomes an essential practice to security in robotics, helping us identify and report a total of 100 security flaws with 17 new CVE IDs over a period of two years. Lastly, we finalize by demonstrating how, through teardown, planned obsolescence hardware limitations can be identified and bypassed obtaining full control of the hardware, which poses both a threat to the robot manufacturers' business model as well as a security threat

Robot Teardown, Stripping Industrial Robots for Good

Building a robot requires a careful selection of components that interact across networks while meeting timing deadlines. Given the complexity associated, as robots get damaged or security compromised, their components will increasingly require updates and replacements. Contrary to the expectations and similar to Ford in the 1920s with cars, most robot manufacturers oppose to this. They employ planned obsolescence practices organizing dealers and system integrators into "private networks", providing repair parts only to "certified" companies to discourage repairs and evade competition. In this article, we introduce and advocate for robot teardown as an approach to study robot hardware architectures and fuel security research. We show how teardown can help understanding the underlying hardware and demonstrate how our approach can help researchers uncovering security vulnerabilities. Our case studies show how robot teardown becomes an essential practice to security in robotics, helping us identify and report a total of 100 security flaws with 17 new CVE IDs over a period of two years. Lastly, we finalize by demonstrating how, through teardown, planned obsolescence hardware limitations can be identified and bypassed obtaining full control of the hardware, which poses both a threat to the robot manufacturers' business model as well as a security threat

DEVELOPMENT AND VALIDATION OF SIMULATORS FOR POWER WHEELCHAIR DRIVING EVALUATIONS

Of all those people with severe physical and cognitive disabilities who are rated as unsafe to drive a power wheelchair and hence denied a wheelchair, a significant number can have positive outcomes by using advanced control interfaces and by getting adequate amount of driving training. This dissertation research presents development and user evaluations with a virtual reality based wheelchair driving simulator system. Using the software systems validated in these research studies clinicians can select and customize joystick interfaces that can optimally use their client’s physical and cognitive capabilities. When people with traumatic brain injury and cerebral palsy used the isometric joystick they committed equivalent or lesser driving errors than when they used the conventional movement sensing joystick to drive a wheelchair. Potential wheelchair users can benefit from such customizable control interfaces to reliably and safely control their power wheelchairs and improve their community participation. An immersive virtual reality simulator was further developed as a driving training and evaluation tool. People with various disabilities completed a clinically validated driving evaluation protocol in real and virtual environments. Their virtual driving performances in the simulator were predictive of their performances in real world. Experienced clinicians showed high inter and intra rater reliabilities in their driving evaluations. Research was also performed to understand the relative contribution of different system components of the simulator system to the overall mental and physical workload of users. This research may assist researchers in selecting simulator system components that best suit the clinical needs of potential users. Clinicians who were trained to evaluate wheelchair driving using this system and wheelchair users who used it gave a general positive feedback that that this simulator has good potential for use in clinical or community settings

Shall I describe it or shall I move closer? Verbal references and locomotion in VR collaborative search tasks

Research in pointing-based communication within immersive collaborative virtual environments (ICVE) remains a compelling area of study. Previous studies explored techniques to improve accuracy and reduce errors when hand-pointing from a distance. In this study, we explore how users adapt their behaviour to cope with the lack of accuracy during pointing. In an ICVE where users can move (i.e., locomotion) when faced with a lack of laser pointers, pointing inaccuracy can be avoided by getting closer to the object of interest. Alternatively, collaborators can enrich the utterances with details to compensate for the lack of pointing precision. Inspired by previous CSCW remote desktop collaboration, we measure visual coordination, the implicitness of deixis’ utterances and the amount of locomotion. We design an experiment that compares the effects of the presence/absence of laser pointers across hard/easy-to-describe referents. Results show that when users face pointing inaccuracy, they prefer to move closer to the referent rather than enrich the verbal reference

Taxonomies for Reasoning About Cyber-physical Attacks in IoT-based Manufacturing Systems

The Internet of Things (IoT) has transformed many aspects of modern manufacturing, from design to production to quality control. In particular, IoT and digital manufacturing technologies have substantially accelerated product development- cycles and manufacturers can now create products of a complexity and precision not heretofore possible. New threats to supply chain security have arisen from connecting machines to the Internet and introducing complex IoT-based systems controlling manufacturing processes. By attacking these IoT-based manufacturing systems and tampering with digital files, attackers can manipulate physical characteristics of parts and change the dimensions, shapes, or mechanical properties of the parts, which can result in parts that fail in the field. These defects increase manufacturing costs and allow silent problems to occur only under certain loads that can threaten safety and/or lives. To understand potential dangers and protect manufacturing system safety, this paper presents two taxonomies: one for classifying cyber-physical attacks against manufacturing processes and another for quality control measures for counteracting these attacks. We systematically identify and classify possible cyber-physical attacks and connect the attacks with variations in manufacturing processes and quality control measures. Our taxonomies also provide a scheme for linking emerging IoT-based manufacturing system vulnerabilities to possible attacks and quality control measures