Energy Harvesting Networked Nodes: Measurements, Algorithms, and Prototyping

Recent advances in ultra-low-power wireless communications and in energy harvesting will soon enable energetically self-sustainable wireless devices. Networks of such devices will serve as building blocks for different Internet of Things (IoT) applications, such as searching for an object on a network of objects and continuous monitoring of object configurations. Yet, numerous challenges need to be addressed for the IoT vision to be fully realized. This thesis considers several challenges related to ultra-low-power energy harvesting networked nodes: energy source characterization, algorithm design, and node design and prototyping. Additionally, the thesis contributes to engineering education, specifically to project-based learning. We summarize our contributions to light and kinetic (motion) energy characterization for energy harvesting nodes. To characterize light energy, we conducted a first-of-its kind 16 month-long indoor light energy measurements campaign. To characterize energy of motion, we collected over 200 hours of human and object motion traces. We also analyzed traces previously collected in a study with over 40 participants. We summarize our insights, including light and motion energy budgets, variability, and influencing factors. These insights are useful for designing energy harvesting nodes and energy harvesting adaptive algorithms. We shared with the community our light energy traces, which can be used as energy inputs to system and algorithm simulators and emulators. We also discuss resource allocation problems we considered for energy harvesting nodes. Inspired by the needs of tracking and monitoring IoT applications, we formulated and studied resource allocation problems aimed at allocating the nodes' time-varying resources in a uniform way with respect to time. We mainly considered deterministic energy profile and stochastic environmental energy models, and focused on single node and link scenarios. We formulated optimization problems using utility maximization and lexicographic maximization frameworks, and introduced algorithms for solving the formulated problems. For several settings, we provided low-complexity solution algorithms. We also examined many simple policies. We demonstrated, analytically and via simulations, that in many settings simple policies perform well. We also summarize our design and prototyping efforts for a new class of ultra-low-power nodes - Energy Harvesting Active Networked Tags (EnHANTs). Future EnHANTs will be wireless nodes that can be attached to commonplace objects (books, furniture, clothing). We describe the EnHANTs prototypes and the EnHANTs testbed that we developed, in collaboration with other research groups, over the last 4 years in 6 integration phases. The prototypes harvest energy of the indoor light, communicate with each other via ultra-low-power transceivers, form small multihop networks, and adapt their communications and networking to their energy harvesting states. The EnHANTs testbed can expose the prototypes to light conditions based on real-world light energy traces. Using the testbed and our light energy traces, we evaluated some of our energy harvesting adaptive policies. Our insights into node design and performance evaluations may apply beyond EnHANTs to networks of various energy harvesting nodes. Finally, we present our contributions to engineering education. Over the last 4 years, we engaged high school, undergraduate, and M.S. students in more than 100 research projects within the EnHANTs project. We summarize our approaches to facilitating student learning, and discuss the results of evaluation surveys that demonstrate the effectiveness of our approaches

Movers and Shakers: Kinetic Energy Harvesting for the Internet of Things

Numerous energy harvesting wireless devices that will serve as building blocks for the Internet of Things (IoT) are currently under development. However, there is still only limited understanding of the properties of various energy sources and their impact on energy harvesting adaptive algorithms. Hence, we focus on characterizing the kinetic (motion) energy that can be harvested by a wireless node with an IoT form factor and on developing energy allocation algorithms for such nodes. In this paper, we describe methods for estimating harvested energy from acceleration traces. To characterize the energy availability associated with specific human activities (e.g., relaxing, walking, cycling), we analyze a motion dataset with over 40 participants. Based on acceleration measurements that we collected for over 200 hours, we study energy generation processes associated with day-long human routines. We also briefly summarize our experiments with moving objects. We develop energy allocation algorithms that take into account practical IoT node design considerations, and evaluate the algorithms using the collected measurements. Our observations provide insights into the design of motion energy harvesters, IoT nodes, and energy harvesting adaptive algorithms.Comment: 15 pages, 11 figure

Ambient Intelligence for Next-Generation AR

Next-generation augmented reality (AR) promises a high degree of context-awareness - a detailed knowledge of the environmental, user, social and system conditions in which an AR experience takes place. This will facilitate both the closer integration of the real and virtual worlds, and the provision of context-specific content or adaptations. However, environmental awareness in particular is challenging to achieve using AR devices alone; not only are these mobile devices' view of an environment spatially and temporally limited, but the data obtained by onboard sensors is frequently inaccurate and incomplete. This, combined with the fact that many aspects of core AR functionality and user experiences are impacted by properties of the real environment, motivates the use of ambient IoT devices, wireless sensors and actuators placed in the surrounding environment, for the measurement and optimization of environment properties. In this book chapter we categorize and examine the wide variety of ways in which these IoT sensors and actuators can support or enhance AR experiences, including quantitative insights and proof-of-concept systems that will inform the development of future solutions. We outline the challenges and opportunities associated with several important research directions which must be addressed to realize the full potential of next-generation AR.Comment: This is a preprint of a book chapter which will appear in the Springer Handbook of the Metavers

SiTAR: Situated Trajectory Analysis for In-the-Wild Pose Error Estimation

Virtual content instability caused by device pose tracking error remains a prevalent issue in markerless augmented reality (AR), especially on smartphones and tablets. However, when examining environments which will host AR experiences, it is challenging to determine where those instability artifacts will occur; we rarely have access to ground truth pose to measure pose error, and even if pose error is available, traditional visualizations do not connect that data with the real environment, limiting their usefulness. To address these issues we present SiTAR (Situated Trajectory Analysis for Augmented Reality), the first situated trajectory analysis system for AR that incorporates estimates of pose tracking error. We start by developing the first uncertainty-based pose error estimation method for visual-inertial simultaneous localization and mapping (VI-SLAM), which allows us to obtain pose error estimates without ground truth; we achieve an average accuracy of up to 96.1% and an average F1 score of up to 0.77 in our evaluations on four VI-SLAM datasets. Next we present our SiTAR system, implemented for ARCore devices, combining a backend that supplies uncertainty-based pose error estimates with a frontend that generates situated trajectory visualizations. Finally, we evaluate the efficacy of SiTAR in realistic conditions by testing three visualization techniques in an in-the-wild study with 15 users and 13 diverse environments; this study reveals the impact both environment scale and the properties of surfaces present can have on user experience and task performance.Comment: To appear in Proceedings of IEEE ISMAR 202

Here To Stay: A Quantitative Comparison of Virtual Object Stability in Markerless Mobile AR

Mobile augmented reality (AR) has the potential to enable immersive, natural interactions between humans and cyber-physical systems. In particular markerless AR, by not relying on fiducial markers or predefined images, provides great convenience and flexibility for users. However, unwanted virtual object movement frequently occurs in markerless smartphone AR due to inaccurate scene understanding, and resulting errors in device pose tracking. We examine the factors which may affect virtual object stability, design experiments to measure it, and conduct systematic quantitative characterizations across six different user actions and five different smartphone configurations. Our study demonstrates noticeable instances of spatial instability in virtual objects in all but the simplest settings (with position errors of greater than 10cm even on the best-performing smartphones), and underscores the need for further enhancements to pose tracking algorithms for smartphone-based markerless AR.Peer reviewe

Project-based Learning within a Large-Scale Interdisciplinary Research Effort

The modern engineering landscape increasingly requires a range of skills to successfully integrate complex systems. Project-based learning is used to help students build professional skills. However, it is typically applied to small teams and small efforts. This paper describes an experience in engaging a large number of students in research projects within a multi-year interdisciplinary research effort. The projects expose the students to various disciplines in Computer Science (embedded systems, algorithm design, networking), Electrical Engineering (circuit design, wireless communications, hardware prototyping), and Applied Physics (thin-film battery design, solar cell fabrication). While a student project is usually focused on one discipline area, it requires interaction with at least two other areas. Over 5 years, 180 semester-long projects have been completed. The students were a diverse group of high school, undergraduate, and M.S. Computer Science, Computer Engineering, and Electrical Engineering students. Some of the approaches that were taken to facilitate student learning are real-world system development constraints, regular cross-group meetings, and extensive involvement of Ph.D. students in student mentorship and knowledge transfer. To assess the approaches, a survey was conducted among the participating students. The results demonstrate the effectiveness of the approaches. For example, 70% of the students surveyed indicated that working on their research project improved their ability to function on multidisciplinary teams more than coursework, internships, or any other activity

InDepth: Real-time Depth Inpainting for Mobile Augmented Reality

Peer reviewe

Wormhole attack detection in wireless ad hoc networks

This thesis deals with wormhole attack discovery in mobile wireless ad hoc networks. Two separate approaches to wormhole attack discovery are developed in this thesis. One approach -- based on protocol-breaking -- allows detection of wormholes that disrupt network operations by dropping network packets. Another -- a novel frequency-based analysis of periodic network messages -- detects wormholes that do not drop traffic. The developed wormhole attack discovery techniques are local, do not rely on specialized hardware or clock synchronization, and do not require modification to existing ad hoc network routing protocols. In addition, tools that are necessary for ad hoc network attack research are created. Network traffic analyzer modules applicable to ad hoc network research are developed and tested. Also, a realistic implementation of a wormhole attack in the NS-2 network simulator is created