Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies

Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Cyber-Human Systems, Space Technologies, and Threats

CYBER-HUMAN SYSTEMS, SPACE TECHNOLOGIES, AND THREATS is our eighth textbook in a series covering the world of UASs / CUAS/ UUVs / SPACE. Other textbooks in our series are Space Systems Emerging Technologies and Operations; Drone Delivery of CBNRECy – DEW Weapons: Emerging Threats of Mini-Weapons of Mass Destruction and Disruption (WMDD); Disruptive Technologies with applications in Airline, Marine, Defense Industries; Unmanned Vehicle Systems & Operations On Air, Sea, Land; Counter Unmanned Aircraft Systems Technologies and Operations; Unmanned Aircraft Systems in the Cyber Domain: Protecting USA’s Advanced Air Assets, 2nd edition; and Unmanned Aircraft Systems (UAS) in the Cyber Domain Protecting USA’s Advanced Air Assets, 1st edition. Our previous seven titles have received considerable global recognition in the field. (Nichols & Carter, 2022) (Nichols, et al., 2021) (Nichols R. K., et al., 2020) (Nichols R. , et al., 2020) (Nichols R. , et al., 2019) (Nichols R. K., 2018) (Nichols R. K., et al., 2022)https://newprairiepress.org/ebooks/1052/thumbnail.jp

個別ニーズを満たす弾性3Dプリント膝支持装具の体系的な設計と分析方法に関する研究

The focus of this research is on the development of an Orthotic device for the human knee joint by implementing the compliant properties of flexible 3D Printed mechanisms. The main purpose of this study is to address the aspects of customization and especially in the low-budget solutions where currently there is a lack of wearable options due to the cost limitations. Additionally, there is an existing gap for devices with not only personalized flexibility but with adjustable one, controllable during the rehabilitation process of the patients. After the knee injury occurs, a surgical intervention might be needed to replace the damaged ligaments. After that, the patient’s lower limb is fixed, and they are required to move by using crutches. The next phase of the recovery is using orthotic devices. Currently there are solid supportive devices on the market for the initial stage of the healing. Later, when the condition improves, and these supportive devices are being replaced by soft fabric type wearables and supplemented with low impact rehabilitation exercises. The problem with that solution is the lack of a middle supportive devices t that makes the transition during the recovery smoother. The missing transitioning pieces can be serious issue amongst athletes who wish to accelerate their recovery process and start practicing on the field as soon as possible. The knee joint is especially vulnerable during those periods of transition between the hard hinge type devices and the soft sleeves, and usually this is the time when many people get injured again. Therefore, it can be assumed that there is a gap on the current market for an Orthotic device with an adjustable elasticity based on the stage of the rehabilitation process, individual requirements of the patients and their lifestyle. This necessity is even more present among the budget solutions for supportive device realization. In addition, this works is focused on ACL (Anterior Cruciate Ligament) injuries since, as explored in the literature below, they occur 10 times more often than the PCL complications, and in general, from all ligament injuries, half are at the ACL. The ACL usually tears during sports. Some of the main questions this research tries to answer can be formulated as: Is there an actual need for such solution? What are the current solutions and how this problem has been addressed? Who are the targeted groups experiencing this problem? What are the main advantages and disadvantages of the proposed concepts? Lastly, is the implementation on its own a possible task? In this work a more “unconventional” techniques, suitable for the purposes of Orthotic device design and fabrication is introduced and analyzed. The problem has been approached from a different perspective. Instead of looking for ways to improve the existing solutions, the findings of another field – 3D Printing and in particular, soft 3D Printing of compliant mechanisms were implanted and adapted according to the requirements of the above stated problem. At first, a definition of a compliant mechanism is “a flexible mechanism that transmits force and motion through elastic body deformation”. Simply put, a mechanism that relays on the deformation caused by an external loading to perform the motion. From this notion is clear that compliant mechanisms have possible applications for Orthotic purposes. This, combined with the fact that these mechanisms are a good fit for 3D Printing, makes them strong candidate for the development and fabrication of supportive devices designed regarding the individual specificities. In this thesis, several models of compliant flexible 3D Printed mechanisms implemented in Orthotic devices for the knee joint have been presented and investigated. The models of the Orthotic device were verified by non-linear Finite Element Analysis due to the anisotropic nature of the 3D Printing materials and the occurring large deformations. Additionally, some of the proposed models were confirmed in an experimental environment by using a robotic arm and an artificial leg to measure the relationship between the force and the displacements of the samples. A rehabilitation protocol for athletes with ACL complications was developed and proposed. To take full advantage of the capabilities of the additive manufacturing processes and to better optimize the samples, the cellular solid model was implemented to describe the 3D Printing structure from inside and explore the parameters important for controlling the stiffness/flexibility of the mechanisms. Three different infill types and seven infill densities were explored. It was established that flexible 3D Printing can fulfil the criteria of fabricating compliant supportive devices with adjustable resistance and deformation depending on the required parameters. Also was found the relation between the relative density and the moduli of elasticity, and relative density and stress of 3D Printed parts with different infill density and infill geometry. This can be valuable tool throughout the design phase in order to optimize the compliant supportive device, its stiffness and strength to weight ratio. Finally, it can be concluded, 3D Printing by using flexible filaments is suitable manufacturing process for custom-made supportive wearables, especially for low-cost options. It gives extra levels of personalization unavailable to the traditional technologies with removing material. Implementation of compliant mechanisms allows for deeper control on the parameters related with the flexibility, ergonomics, and reliability of the Orthotic device. In the context of the Forth Industrial Revolution, Additive Manufacturing technologies emerge as better and better alternatives as toolless methods with minimal material waste drastically decreasing the prototyping costs and time and only limited by the imagination of the designers and makers, to shape our new perspectives of what is possible.九州工業大学博士学位論文 学位記番号：生工博甲第450号 学位授与年月日：令和4年9月26日1 Introduction|2 Literature Review|3 Methodology|4 Results|5 Discussion|6 Conclusion九州工業大学令和4年

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Energy-Efficient Recurrent Neural Network Accelerators for Real-Time Inference

Over the past decade, Deep Learning (DL) and Deep Neural Network (DNN) have gone through a rapid development. They are now vastly applied to various applications and have profoundly changed the life of hu- man beings. As an essential element of DNN, Recurrent Neural Networks (RNN) are helpful in processing time-sequential data and are widely used in applications such as speech recognition and machine translation. RNNs are difficult to compute because of their massive arithmetic operations and large memory footprint. RNN inference workloads used to be executed on conventional general-purpose processors including Central Processing Units (CPU) and Graphics Processing Units (GPU); however, they have un- necessary hardware blocks for RNN computation such as branch predictor, caching system, making them not optimal for RNN processing. To accelerate RNN computations and outperform the performance of conventional processors, previous work focused on optimization methods on both software and hardware. On the software side, previous works mainly used model compression to reduce the memory footprint and the arithmetic operations of RNNs. On the hardware side, previous works also designed domain-specific hardware accelerators based on Field Pro- grammable Gate Arrays (FPGA) or Application Specific Integrated Circuits (ASIC) with customized hardware pipelines optimized for efficient pro- cessing of RNNs. By following this software-hardware co-design strategy, previous works achieved at least 10X speedup over conventional processors. Many previous works focused on achieving high throughput with a large batch of input streams. However, in real-time applications, such as gaming Artificial Intellegence (AI), dynamical system control, low latency is more critical. Moreover, there is a trend of offloading neural network workloads to edge devices to provide a better user experience and privacy protection. Edge devices, such as mobile phones and wearable devices, are usually resource-constrained with a tight power budget. They require RNN hard- ware that is more energy-efficient to realize both low-latency inference and long battery life. Brain neurons have sparsity in both the spatial domain and time domain. Inspired by this human nature, previous work mainly explored model compression to induce spatial sparsity in RNNs. The delta network algorithm alternatively induces temporal sparsity in RNNs and can save over 10X arithmetic operations in RNNs proven by previous works. In this work, we have proposed customized hardware accelerators to exploit temporal sparsity in Gated Recurrent Unit (GRU)-RNNs and Long Short-Term Memory (LSTM)-RNNs to achieve energy-efficient real-time RNN inference. First, we have proposed DeltaRNN, the first-ever RNN accelerator to exploit temporal sparsity in GRU-RNNs. DeltaRNN has achieved 1.2 TOp/s effective throughput with a batch size of 1, which is 15X higher than its related works. Second, we have designed EdgeDRNN to accelerate GRU-RNN edge inference. Compared to DeltaRNN, EdgeDRNN does not rely on on-chip memory to store RNN weights and focuses on reducing off-chip Dynamic Random Access Memory (DRAM) data traffic using a more scalable architecture. EdgeDRNN have realized real-time inference of large GRU-RNNs with submillisecond latency and only 2.3 W wall plug power consumption, achieving 4X higher energy efficiency than commercial edge AI platforms like NVIDIA Jetson Nano. Third, we have used DeltaRNN to realize the first-ever continuous speech recognition sys- tem with the Dynamic Audio Sensor (DAS) as the front-end. The DAS is a neuromorphic event-driven sensor that produces a stream of asyn- chronous events instead of audio data sampled at a fixed sample rate. We have also showcased how an RNN accelerator can be integrated with an event-driven sensor on the same chip to realize ultra-low-power Keyword Spotting (KWS) on the extreme edge. Fourth, we have used EdgeDRNN to control a powered robotic prosthesis using an RNN controller to replace a conventional proportional–derivative (PD) controller. EdgeDRNN has achieved 21 μs latency of running the RNN controller and could maintain stable control of the prosthesis. We have used DeltaRNN and EdgeDRNN to solve these problems to prove their value in solving real-world problems. Finally, we have applied the delta network algorithm on LSTM-RNNs and have combined it with a customized structured pruning method, called Column-Balanced Targeted Dropout (CBTD), to induce spatio-temporal sparsity in LSTM-RNNs. Then, we have proposed another FPGA-based accelerator called Spartus, the first RNN accelerator that exploits spatio- temporal sparsity. Spartus achieved 9.4 TOp/s effective throughput with a batch size of 1, the highest among present FPGA-based RNN accelerators with a power budget around 10 W. Spartus can complete the inference of an LSTM layer having 5 million parameters within 1 μs