Towards Everyday Virtual Reality through Eye Tracking

Durch Entwicklungen in den Bereichen Computergrafik, Hardwaretechnologie, Perception Engineering und Mensch-Computer Interaktion, werden Virtual Reality und virtuelle Umgebungen immer mehr in unser tägliches Leben integriert. Head-Mounted Displays werden jedoch im Vergleich zu anderen mobilen Geräten, wie Smartphones und Smartwatches, noch nicht so häufig genutzt. Mit zunehmender Nutzung dieser Technologie und der Gewöhnung von Menschen an virtuelle Anwendungsszenarien ist es wahrscheinlich, dass in naher Zukunft ein alltägliches Virtual-Reality-Paradigma realisiert wird. Im Hinblick auf die Kombination von alltäglicher Virtual Reality und Head-Mounted-Displays, ist Eye Tracking eine neue Technologie, die es ermöglicht, menschliches Verhalten in Echtzeit und nicht-intrusiv zu messen. Bevor diese Technologien in großem Umfang im Alltag eingesetzt werden können, müssen jedoch noch zahlreiche Aspekte genauer erforscht werden. Zunächst sollten Aufmerksamkeits- und Kognitionsmodelle in Alltagsszenarien genau verstanden werden. Des Weiteren sind Maßnahmen zur Wahrung der Privatsphäre notwendig, da die Augen mit visuellen biometrischen Indikatoren assoziiert sind. Zuletzt sollten anstelle von Studien oder Anwendungen, die sich auf eine begrenzte Anzahl menschlicher Teilnehmer mit relativ homogenen Merkmalen stützen, Protokolle und Anwendungsfälle für eine bessere Zugänglichkeit dieser Technologie von wesentlicher Bedeutung sein. In dieser Arbeit wurde unter Berücksichtigung der oben genannten Punkte ein bedeutender wissenschaftlicher Vorstoß mit drei zentralen Forschungsbeiträgen in Richtung alltäglicher Virtual Reality unternommen. Menschliche visuelle Aufmerksamkeit und Kognition innerhalb von Virtual Reality wurden in zwei unterschiedlichen Bereichen, Bildung und Autofahren, erforscht. Die Forschung im Bildungsbereich konzentrierte sich auf die Auswirkungen verschiedener Manipulationen im Klassenraum auf das menschliche Sehverhalten, während die Forschung im Bereich des Autofahrens auf sicherheitsrelevante Fragen und Blickführung abzielte. Die Nutzerstudien in beiden Bereichen zeigen, dass Blickbewegungen signifikante Implikationen für diese alltäglichen Situationen haben. Der zweite wesentliche Beitrag fokussiert sich auf Privatsphäre bewahrendes Eye Tracking für Blickbewegungsdaten von Head-Mounted Displays. Dies beinhaltet Differential Privacy, welche zeitliche Korrelationen von Blickbewegungssignalen berücksichtigt und Privatsphäre wahrende Blickschätzung durch Verwendung eines auf randomisiertem Encoding basierenden Frameworks, welches Augenreferenzunkte verwendet. Die Ergebnisse beider Arbeiten zeigen, dass die Wahrung der Privatsphäre möglich ist und gleichzeitig der Nutzen in einem akzeptablen Bereich bleibt. Wenngleich es bisher nur wenig Forschung zu diesem Aspekt von Eye Tracking gibt, ist weitere Forschung notwendig, um den alltäglichen Gebrauch von Virtual Reality zu ermöglichen. Als letzter signifikanter Beitrag, wurde ein Blockchain- und Smart Contract-basiertes Protokoll zur Eye Tracking Datenerhebung für Virtual Reality vorgeschlagen, um Virtual Reality besser zugänglich zu machen. Die Ergebnisse liefern wertvolle Erkenntnisse für alltägliche Nutzung von Virtual Reality und treiben den aktuellen Stand der Forschung in mehrere Richtungen voran.With developments in computer graphics, hardware technology, perception engineering, and human-computer interaction, virtual reality and virtual environments are becoming more integrated into our daily lives. Head-mounted displays, however, are still not used as frequently as other mobile devices such as smart phones and watches. With increased usage of this technology and the acclimation of humans to virtual application scenarios, it is possible that in the near future an everyday virtual reality paradigm will be realized. When considering the marriage of everyday virtual reality and head-mounted displays, eye tracking is an emerging technology that helps to assess human behaviors in a real time and non-intrusive way. Still, multiple aspects need to be researched before these technologies become widely available in daily life. Firstly, attention and cognition models in everyday scenarios should be thoroughly understood. Secondly, as eyes are related to visual biometrics, privacy preserving methodologies are necessary. Lastly, instead of studies or applications utilizing limited human participants with relatively homogeneous characteristics, protocols and use-cases for making such technology more accessible should be essential. In this work, taking the aforementioned points into account, a significant scientific push towards everyday virtual reality has been completed with three main research contributions. Human visual attention and cognition have been researched in virtual reality in two different domains, including education and driving. Research in the education domain has focused on the effects of different classroom manipulations on human visual behaviors, whereas research in the driving domain has targeted safety related issues and gaze-guidance. The user studies in both domains show that eye movements offer significant implications for these everyday setups. The second substantial contribution focuses on privacy preserving eye tracking for the eye movement data that is gathered from head-mounted displays. This includes differential privacy, taking temporal correlations of eye movement signals into account, and privacy preserving gaze estimation task by utilizing a randomized encoding-based framework that uses eye landmarks. The results of both works have indicated that privacy considerations are possible by keeping utility in a reasonable range. Even though few works have focused on this aspect of eye tracking until now, more research is necessary to support everyday virtual reality. As a final significant contribution, a blockchain- and smart contract-based eye tracking data collection protocol for virtual reality is proposed to make virtual reality more accessible. The findings present valuable insights for everyday virtual reality and advance the state-of-the-art in several directions

Multi-task near-field perception for autonomous driving using surround-view fisheye cameras

Die Bildung der Augen führte zum Urknall der Evolution. Die Dynamik änderte sich von einem primitiven Organismus, der auf den Kontakt mit der Nahrung wartete, zu einem Organismus, der durch visuelle Sensoren gesucht wurde. Das menschliche Auge ist eine der raffiniertesten Entwicklungen der Evolution, aber es hat immer noch Mängel. Der Mensch hat über Millionen von Jahren einen biologischen Wahrnehmungsalgorithmus entwickelt, der in der Lage ist, Autos zu fahren, Maschinen zu bedienen, Flugzeuge zu steuern und Schiffe zu navigieren. Die Automatisierung dieser Fähigkeiten für Computer ist entscheidend für verschiedene Anwendungen, darunter selbstfahrende Autos, Augmented Realität und architektonische Vermessung. Die visuelle Nahfeldwahrnehmung im Kontext von selbstfahrenden Autos kann die Umgebung in einem Bereich von 0 - 10 Metern und 360° Abdeckung um das Fahrzeug herum wahrnehmen. Sie ist eine entscheidende Entscheidungskomponente bei der Entwicklung eines sichereren automatisierten Fahrens. Jüngste Fortschritte im Bereich Computer Vision und Deep Learning in Verbindung mit hochwertigen Sensoren wie Kameras und LiDARs haben ausgereifte Lösungen für die visuelle Wahrnehmung hervorgebracht. Bisher stand die Fernfeldwahrnehmung im Vordergrund. Ein weiteres wichtiges Problem ist die begrenzte Rechenleistung, die für die Entwicklung von Echtzeit-Anwendungen zur Verfügung steht. Aufgrund dieses Engpasses kommt es häufig zu einem Kompromiss zwischen Leistung und Laufzeiteffizienz. Wir konzentrieren uns auf die folgenden Themen, um diese anzugehen: 1) Entwicklung von Nahfeld-Wahrnehmungsalgorithmen mit hoher Leistung und geringer Rechenkomplexität für verschiedene visuelle Wahrnehmungsaufgaben wie geometrische und semantische Aufgaben unter Verwendung von faltbaren neuronalen Netzen. 2) Verwendung von Multi-Task-Learning zur Überwindung von Rechenengpässen durch die gemeinsame Nutzung von initialen Faltungsschichten zwischen den Aufgaben und die Entwicklung von Optimierungsstrategien, die die Aufgaben ausbalancieren.The formation of eyes led to the big bang of evolution. The dynamics changed from a primitive organism waiting for the food to come into contact for eating food being sought after by visual sensors. The human eye is one of the most sophisticated developments of evolution, but it still has defects. Humans have evolved a biological perception algorithm capable of driving cars, operating machinery, piloting aircraft, and navigating ships over millions of years. Automating these capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and architectural surveying. Near-field visual perception in the context of self-driving cars can perceive the environment in a range of 0 - 10 meters and 360° coverage around the vehicle. It is a critical decision-making component in the development of safer automated driving. Recent advances in computer vision and deep learning, in conjunction with high-quality sensors such as cameras and LiDARs, have fueled mature visual perception solutions. Until now, far-field perception has been the primary focus. Another significant issue is the limited processing power available for developing real-time applications. Because of this bottleneck, there is frequently a trade-off between performance and run-time efficiency. We concentrate on the following issues in order to address them: 1) Developing near-field perception algorithms with high performance and low computational complexity for various visual perception tasks such as geometric and semantic tasks using convolutional neural networks. 2) Using Multi-Task Learning to overcome computational bottlenecks by sharing initial convolutional layers between tasks and developing optimization strategies that balance tasks

Recent Developments in Smart Healthcare

Medicine is undergoing a sector-wide transformation thanks to the advances in computing and networking technologies. Healthcare is changing from reactive and hospital-centered to preventive and personalized, from disease focused to well-being centered. In essence, the healthcare systems, as well as fundamental medicine research, are becoming smarter. We anticipate significant improvements in areas ranging from molecular genomics and proteomics to decision support for healthcare professionals through big data analytics, to support behavior changes through technology-enabled self-management, and social and motivational support. Furthermore, with smart technologies, healthcare delivery could also be made more efficient, higher quality, and lower cost. In this special issue, we received a total 45 submissions and accepted 19 outstanding papers that roughly span across several interesting topics on smart healthcare, including public health, health information technology (Health IT), and smart medicine

Penman Contra Patriarch: Reimagining the Central Conflict of Joyce\u27s Finnegans Wake

Senior Project submitted to The Division of Languages and Literature of Bard College

A Geocritical Approach to Coloniality and Aesthetics: Mapping the Spatial Narratives of the Amazon Basin

Coloniality, or the living legacies and practices of the 500 years of European colonization, has produced racial, political, and cultural hierarchies around the colonial difference dividing East from West, center from periphery, civilization from the Global South. This dissertation examines a particular strand of coloniality in the Western narration and aesthetics of the Amazon basin, particularly the consequences of travel writing, science fiction and cinema of Amazon’s tropicality and its enduring effects on spatial cartography. In addressing Western representations of the jungle terrain, this paper focuses on the dichotomous relationship between the metropolitan center and the colonial outer-periphery exemplified by the Amazon basin. I take an alternative approach to understanding spatiality by applying what I call the coloniality of aesthetics to the spatial analysis of tropicality, illuminating the naturalized tendencies that articulate the Amazon as simultaneously a modern physical fantasy perpetually on the verge of colonial conquest and a mythological agent of horror that resists colonial conquest by its continued deferral of meaning production between the antagonism of nature and of civilization. The coloniality of aesthetics elucidates the West’s failure to figuratively conquer the land of the Amazon and suggests that such failures are crafted intentionally to preserve the aesthetics of conquest itself. This paper argues that the ontology of tropicality can be reestablished through a radical territorialization, one that centers the protagonism of the Amazon basin through the revelation of aesthetic modes that focus on the generative qualities of the South American rainforest. In doing so, I hope to expand the role of literary geocriticism through an exercise of spatial decolonization, one that prioritizes the often ignored terrain of the Amazon jungle

Artificial morphogenesis via 3D neural cellular automata

Morphogenesis is the biological process by which an organism grows and takes its form. It is responsible for the growth and shape of all living things and has historically been studied from the perspective of developmental biology. Accordingly, experiments have routinely been performed using real organisms and their cells. However, with continued advancements in computational capabilities, researchers have been able to simulate physical and biological systems with greater fidelity and accuracy. A recent example is the development of two-dimensional neural cellular automata, which have the ability to artificially model morphogenesis. Cellular automata, the antecedent to neural cellular automata, are a type of computational model uniquely suited to model physical systems due to their inherit discretization of space and time, locally rule-based dynamics, and embarrassingly parallel execution. Their ability to exhibit rich and sophisticated results from seemingly trivial implementations has lead to their continued use to study emergent properties found in nature. This thesis expands upon previous works by presenting a novel three-dimensional neural cellular automata which illustrates self-organizing, regenerative, and isotropic properties. Understanding the complexities of morphogenesis could provide beneficial insights for the development of regenerative medicines, self-organizing robots, and other bioengineering endeavors.Computer Scienc

Counter Unmanned Aircraft Systems Technologies and Operations

As the quarter-century mark in the 21st Century nears, new aviation-related equipment has come to the forefront, both to help us and to haunt us. (Coutu, 2020) This is particularly the case with unmanned aerial vehicles (UAVs). These vehicles have grown in popularity and accessible to everyone. Of different shapes and sizes, they are widely available for purchase at relatively low prices. They have moved from the backyard recreation status to important tools for the military, intelligence agencies, and corporate organizations. New practical applications such as military equipment and weaponry are announced on a regular basis – globally. (Coutu, 2020) Every country seems to be announcing steps forward in this bludgeoning field. In our successful 2nd edition of Unmanned Aircraft Systems in the Cyber Domain: Protecting USA’s Advanced Air Assets (Nichols, et al., 2019), the authors addressed three factors influencing UAS phenomena. First, unmanned aircraft technology has seen an economic explosion in production, sales, testing, specialized designs, and friendly / hostile usages of deployed UAS / UAVs / Drones. There is a huge global growing market and entrepreneurs know it. Second, hostile use of UAS is on the forefront of DoD defense and offensive planners. They are especially concerned with SWARM behavior. Movies like “Angel has Fallen,” where drones in a SWARM use facial recognition technology to kill USSS agents protecting POTUS, have built the lore of UAS and brought the problem forefront to DHS. Third, UAS technology was exploding. UAS and Counter- UAS developments in navigation, weapons, surveillance, data transfer, fuel cells, stealth, weight distribution, tactics, GPS / GNSS elements, SCADA protections, privacy invasions, terrorist uses, specialized software, and security protocols has exploded. (Nichols, et al., 2019) Our team has followed / tracked joint ventures between military and corporate entities and specialized labs to build UAS countermeasures. As authors, we felt compelled to address at least the edge of some of the new C-UAS developments. It was clear that we would be lucky if we could cover a few of – the more interesting and priority technology updates – all in the UNCLASSIFIED and OPEN sphere. Counter Unmanned Aircraft Systems: Technologies and Operations is the companion textbook to our 2nd edition. The civilian market is interesting and entrepreneurial, but the military and intelligence markets are of concern because the US does NOT lead the pack in C-UAS technologies. China does. China continues to execute its UAS proliferation along the New Silk Road Sea / Land routes (NSRL). It has maintained a 7% growth in military spending each year to support its buildup. (Nichols, et al., 2019) [Chapter 21]. They continue to innovate and have recently improved a solution for UAS flight endurance issues with the development of advanced hydrogen fuel cell. (Nichols, et al., 2019) Reed and Trubetskoy presented a terrifying map of countries in the Middle East with armed drones and their manufacturing origin. Guess who? China. (A.B. Tabriski & Justin, 2018, December) Our C-UAS textbook has as its primary mission to educate and train resources who will enter the UAS / C-UAS field and trust it will act as a call to arms for military and DHS planners.https://newprairiepress.org/ebooks/1031/thumbnail.jp

The Lumberjack, September 19, 2001

The student newspaper of Humboldt State University.https://digitalcommons.humboldt.edu/studentnewspaper/2377/thumbnail.jp