Programming frameworks for mobile sensing

The proliferation of smart mobile devices in people’s daily lives is making context-aware computing a reality. A plethora of sensors available in these devices can be utilized to understand users’ context better. Apps can provide more relevant data or services to the user based on improved understanding of user’s context. With the advent of cloud-assisted mobile platforms, apps can also perform collaborative computation over the sensing data collected from a group of users. However, there are still two main issues: (1) A lack of simple and effective personal sensing frameworks: existing frameworks do not provide support for real-time fusing of data from motion and visual sensors in a simple manner, and no existing framework collectively utilizes sensors from multiple personal devices and personal IoT sensors, and (2) a lack of collaborative/distributed computing frameworks for mobile users. This dissertation presents solutions for these two issues. The first issue is addressed by TagPix and Sentio, two frameworks for mobile sensing. The second issue is addressed by Moitree, a middleware for mobile distributed computing, and CASINO, a collaborative sensor-driven offloading system. TagPix is a real-time, privacy preserving photo tagging framework, which works locally on the phones and consumes little resources (e.g., battery). It generates relevant tags for landscape photos by utilizing sensors of a mobile device and it does not require any previous training or indexing. When a user aims the mobile camera to a particular landmark, the framework uses accelerometer and geomagnetic field sensor to identify in which direction the user is aiming the camera at. It then uses a landmark database and employs a smart distance estimation algorithm to identify which landmark(s) is targeted by the user. The framework then generates relevant tags for the captured photo using these information. A more versatile sensing framework can be developed using sensors from multiple devices possessed by a user. Sentio is such a framework which enables apps to seamlessly utilize the collective sensing capabilities of the user’s personal devices and of the IoT sensors located in the proximity of the user. With Sentio, an app running on any personal mobile/wearable device can access any sensor of the user in real-time using the same API, can selectively switch to the most suitable sensor of a particular type when multiple sensors of this type are available at different devices, and can build composite sensors. Sentio offers seamless connectivity to sensors even if the sensor-accessing code is offloaded to the cloud. Sentio provides these functionalities with a high-level API and a distributed middleware that handles all low-level communication and sensor management tasks. This dissertation also proposes Moitree, a middleware for the mobile cloud platforms where each mobile device is augmented by an avatar, a per-user always-on software entity that resides in the cloud. Mobile-avatar pairs participate in distributed computing as a unified computing entity. Moitree provides a common programming and execution framework for mobile distributed apps. Moitree allows the components of a distributed app to execute seamlessly over a set of mobile/avatar pairs, with the provision of offloading computation and communication to the cloud. The programming framework has two key features: user collaborations are modeled using group semantics - groups are created dynamically based on context and are hierarchical; data communication among group members is offloaded to the cloud through high-level communication channels. Finally, this dissertation presents and discusses CASINO, a collaborative sensor-driven computation offloading framework which can be used alongside Moitree. This framework includes a new scheduling algorithm which minimizes the total completion time of a collaborative computation that executes over a set of mobile/avatar pairs. Using the CASINO API, the programmers can mark their classes and functions as ”offloadable”. The framework collects profiling information (network, CPU, battery, etc.) from participating users’ mobile devices and avatars, and then schedules ”offloadable” tasks in mobiles and avatars in a way that reduces the total completion time. The scheduling problem is proven to be NP-Hard and there is no polynomial time optimization algorithm for it. The proposed algorithm can generate a schedule in polynomial time using a topological sorting and greedy technique

Photo-shoot localization of a mobile camera based on registered frame data of virtualized reality models

PosterInternational audienceThis paper presents a study of a method for estimating the position and orientation of a photo-shoot in indoor environments for augmented reality applications. Our proposed localization method is based on registered frame data of virtualized reality models, which are photos with known photo-shoot positions and orientations, and depth data. Because registered frame data are secondary product of modeling process, additional works are not necessary to create registered frame data especially for the localization. In the method, a photo taken by a mobile camera is compared to registered frame data for the localization. Since registered frame data are linked with photo-shoot position, orientation, and depth data, 3D coordinates of each pixel on the photo of registered frame data is available. We conducted experiments with employing five techniques of the estimation for comparative evaluations

Mobilizing the Past for a Digital Future : The Potential of Digital Archaeology

Mobilizing the Past is a collection of 20 articles that explore the use and impact of mobile digital technology in archaeological field practice. The detailed case studies present in this volume range from drones in the Andes to iPads at Pompeii, digital workflows in the American Southwest, and examples of how bespoke, DIY, and commercial software provide solutions and craft novel challenges for field archaeologists. The range of projects and contexts ensures that Mobilizing the Past for a Digital Future is far more than a state-of-the-field manual or technical handbook. Instead, the contributors embrace the growing spirit of critique present in digital archaeology. This critical edge, backed by real projects, systems, and experiences, gives the book lasting value as both a glimpse into present practices as well as the anxieties and enthusiasm associated with the most recent generation of mobile digital tools. This book emerged from a workshop funded by the National Endowment for the Humanities held in 2015 at Wentworth Institute of Technology in Boston. The workshop brought together over 20 leading practitioners of digital archaeology in the U.S. for a weekend of conversation. The papers in this volume reflect the discussions at this workshop with significant additional content. Starting with an expansive introduction and concluding with a series of reflective papers, this volume illustrates how tablets, connectivity, sophisticated software, and powerful computers have transformed field practices and offer potential for a radically transformed discipline.https://dc.uwm.edu/arthist_mobilizingthepast/1000/thumbnail.jp

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Animating opacity: race, borders, and biometric surveillance

This project intervenes on the increased policing of borders using digital technologies. It is an autoethnographic practice-led research project that investigates the application of biometric surveillance technologies in identity capture and verification of black migrants. Consequently, it focuses on the racial implications of these new forms of surveillance and the resistances necessary for black migrant survival. This study emphasizes the importance of resistance as black migrants’ movements are increasingly dictated by biometric technologies that transform everyday spaces into the border. Crucial to this study is the connection of the histories of the colonial biometric dissection of the black body to the contemporary inscription of race on the body despite the claim that biometrics are race-neutral. Placed within the connection of modern biometric technologies with their colonial predecessors are black migrants who are disproportionately scrutinized at the border while being subjected to racial bias in moments of biometric data capture, identification, and verification. Animating Opacity, therefore, analyzes these processes of biometric surveillance focusing on autoethnographic accounts and public case studies of the policing of black migrants. The analysis respectively presented within the chapters are: the histories of these biometric technologies that state their links to the colonial dissection, the inscription of race in the act of biometricization, the racial syntax of biometric capture that tags black migrants as other, the affective economy of fear resulting in the boundary maintenance of the black body, and the new spatializing practices engendered by the creation of the biometric border. Countering these experiences of surveillance at biometric borders are moments of resistance placed within media art practices. These art practices include installation art, moving images, and video games which assert the right to opacity and geographic agency of black migrants. Therefore, this study centralizes the resistances of black migrants against biometric surveillance. Resistance is framed as ‘the right to opacity,’— the counterpoint to the colonial imperative of transparency—as conceptualized by Édouard Glissant. Through the framing of resistance, Animating Opacity plots the escape from biometric capture, the new forms of languages that exploit the failures of biometric surveillance, and the virtual spaces outside of the surveillant gaze that animates the opacity of black migrant life

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Ecocinema Theory and Practice 2

This second volume builds on the initial groundwork laid by Ecocinema Theory and Practice by examining the ways in which ecocritical cinema studies have matured and proliferated over the last decade, opening whole new areas of study and research. Featuring fourteen new essays organized into three sections around the themes of cinematic materialities, discourses, and communities, the volume explores a variety of topics within ecocinema studies from examining specifc national and indigenous flm contexts to discussing ecojustice, environmental production studies, flm festivals, and political ecology. The breadth of the contributions exemplifes how ecocinema scholars worldwide have sought to overcome the historical legacy of binary thinking and intellectual norms and are working to champion new ecocritical, intersectional, decolonial, queer, feminist, Indigenous, vitalist, and other emergent theories and cinematic prac-tices. The collection also demonstrates the unique ways that cinema studies scholarship is actively addressing environmental injustice and the climate crisis. This book is an invaluable resource for students and scholars of ecocritical flm and media studies, production studies, cultural studies, and environmental studies.https://cupola.gettysburg.edu/books/1181/thumbnail.jp

Ecocinema Theory and Practice 2

This second volume builds on the initial groundwork laid by Ecocinema Theory and Practice by examining the ways in which ecocritical cinema studies have matured and proliferated over the last decade, opening whole new areas of study and research. Featuring fourteen new essays organized into three sections around the themes of cinematic materialities, discourses, and communities, the volume explores a variety of topics within ecocinema studies from examining specific national and indigenous film contexts to discussing ecojustice, environmental production studies, film festivals, and political ecology. The breadth of the contributions exemplifies how ecocinema scholars worldwide have sought to overcome the historical legacy of binary thinking and intellectual norms and are working to champion new ecocritical, intersectional, decolonial, queer, feminist, Indigenous, vitalist, and other emergent theories and cinematic practices. The collection also demonstrates the unique ways that cinema studies scholarship is actively addressing environmental injustice and the climate crisis. This book is an invaluable resource for students and scholars of ecocritical film and media studies, production studies, cultural studies, and environmental studies

Actor & Avatar: A Scientific and Artistic Catalog

What kind of relationship do we have with artificial beings (avatars, puppets, robots, etc.)? What does it mean to mirror ourselves in them, to perform them or to play trial identity games with them? Actor & Avatar addresses these questions from artistic and scholarly angles. Contributions on the making of "technical others" and philosophical reflections on artificial alterity are flanked by neuroscientific studies on different ways of perceiving living persons and artificial counterparts. The contributors have achieved a successful artistic-scientific collaboration with extensive visual material