Smart and Secure Augmented Reality for Assisted Living

Augmented reality (AR) is one of the biggest technology trends which enables people to see the real-life surrounding environment with a layer of virtual information overlaid on it. Assistive devices use this match of information to help people better understand the environment and consequently be more efficient. Specially, AR has been extremely useful in the area of Ambient Assisted Living (AAL). AR-based AAL solutions are designed to support people in maintaining their autonomy and compensate for slight physical and mental restrictions by instructing them on everyday tasks. The discovery of visual attention for assistive aims is a big challenge since in dynamic cluttered environments objects are constantly overlapped and partial object occlusion is also frequent. Current solutions use egocentric object recognition techniques. However, the lack of accuracy affects the system's ability to predict users’ needs and consequently provide them with the proper support. Another issue is the manner that sensitive data is treated. This highly private information is crucial for improving the quality of healthcare services. However, current blockchain approaches are used only as a permission management system, while the data is still stored locally. As a result, there is a potential risk of security breaches. Privacy risk in the blockchain domain is also a concern. As major investigation tackles privacy issues based on off-chain approaches, there is a lack of effective solutions for providing on-chain data privacy. Finally, the Blockchain size has been shown to be a limiting factor even for chains that store simple transactional data, much less the massive blocks that would be required for storing medical imaging studies. To tackle the aforementioned major issues, this research proposes a framework to provide a smarter and more secure AR-based solution for AAL. Firstly, a combination of head-worn eye-trackers cameras with egocentric video is designed to improve the accuracy of visual attention object recognition in free-living settings. A heuristic function is designed to generate a probability estimation of visual attention over objects within an egocentric video. Secondly, a novel methodology for the storage of large sensitive AR-based AAL data is introduced in a decentralized fashion. By leveraging the power of the IPFS (InterPlanetary File System) protocol to tackle the lack of storage issue in the Blockchain. Meanwhile, a blockchain solution on the Secret Network blockchain is developed to tackle the existent lack of privacy on smart contracts, which provides data privacy at both transactional and computational levels. In addition, is included a new off-chain solution encapsulates a governing body for permission management purposes to solve the problem of the lost or eventual theft of private keys. Based on the research findings, that visual attention-object detection approach is applicable to cluttered environments which presents a transcend performance compared to the current methods. This study also produced an egocentric indoor dataset annotated with human fixation during natural exploration in a cluttered environment. Comparing to previous works, this dataset is more realistic because it was recorded in real settings with variations in terms of objects overlapping regions and object sizes. With respect to the novel decentralized storage methodology, results indicate that sensitive data can be stored and queried efficiently using the Secret Network blockchain. The proposed approach achieves both computational and transactional privacy with significantly less cost. Additionally, this approach mitigates the risk of permanent loss of access to the patient on-chain data records. The proposed framework can be applied as an assistive technology in a wide range of sectors that requires AR-based solution with high-precision visual-attention object detection, efficient data access, high-integrity data storage and full data privacy and security