Context-awareness for mobile sensing: a survey and future directions

The evolution of smartphones together with increasing computational power have empowered developers to create innovative context-aware applications for recognizing user related social and cognitive activities in any situation and at any location. The existence and awareness of the context provides the capability of being conscious of physical environments or situations around mobile device users. This allows network services to respond proactively and intelligently based on such awareness. The key idea behind context-aware applications is to encourage users to collect, analyze and share local sensory knowledge in the purpose for a large scale community use by creating a smart network. The desired network is capable of making autonomous logical decisions to actuate environmental objects, and also assist individuals. However, many open challenges remain, which are mostly arisen due to the middleware services provided in mobile devices have limited resources in terms of power, memory and bandwidth. Thus, it becomes critically important to study how the drawbacks can be elaborated and resolved, and at the same time better understand the opportunities for the research community to contribute to the context-awareness. To this end, this paper surveys the literature over the period of 1991-2014 from the emerging concepts to applications of context-awareness in mobile platforms by providing up-to-date research and future research directions. Moreover, it points out the challenges faced in this regard and enlighten them by proposing possible solutions

A Novel 6G Conversational Orchestration Framework for Enhancing Performance and Resource Utilization in Autonomous Vehicle Networks

A vision of 6G aims to automate versatile services by eliminating the complexity of human effort for Industry 5.0 applications. This results in an intelligent environment with cognitive and collaborative capabilities of AI conversational orchestration that enable a variety of applications across smart Autonomous Vehicle (AV) networks. In this article, an innovative framework for AI conversational orchestration is proposed by enabling on-the-fly virtual infrastructure service orchestration for Anything-as-a-Service (XaaS) to automate a network service paradigm. The proposed framework will potentially contribute to the growth of 6G conversational orchestration by enabling on-the-fly automation of cloud and network services. The orchestration aspect of the 6G vision is not limited to cognitive collaborative communications, but also extends to context-aware personalized infrastructure for 6G automation. The experimental results of the implemented proof-of-concept framework are presented. These experiments not only affirm the technical capabilities of this framework, but also push into several Industry 5.0 applications

Workload-aware systems and interfaces for cognitive augmentation

In today's society, our cognition is constantly influenced by information intake, attention switching, and task interruptions. This increases the difficulty of a given task, adding to the existing workload and leading to compromised cognitive performances. The human body expresses the use of cognitive resources through physiological responses when confronted with a plethora of cognitive workload. This temporarily mobilizes additional resources to deal with the workload at the cost of accelerated mental exhaustion. We predict that recent developments in physiological sensing will increasingly create user interfaces that are aware of the user’s cognitive capacities, hence able to intervene when high or low states of cognitive workload are detected. In this thesis, we initially focus on determining opportune moments for cognitive assistance. Subsequently, we investigate suitable feedback modalities in a user-centric design process which are desirable for cognitive assistance. We present design requirements for how cognitive augmentation can be achieved using interfaces that sense cognitive workload. We then investigate different physiological sensing modalities to enable suitable real-time assessments of cognitive workload. We provide empirical evidence that the human brain is sensitive to fluctuations in cognitive resting states, hence making cognitive effort measurable. Firstly, we show that electroencephalography is a reliable modality to assess the mental workload generated during the user interface operation. Secondly, we use eye tracking to evaluate changes in eye movements and pupil dilation to quantify different workload states. The combination of machine learning and physiological sensing resulted in suitable real-time assessments of cognitive workload. The use of physiological sensing enables us to derive when cognitive augmentation is suitable. Based on our inquiries, we present applications that regulate cognitive workload in home and work settings. We deployed an assistive system in a field study to investigate the validity of our derived design requirements. Finding that workload is mitigated, we investigated how cognitive workload can be visualized to the user. We present an implementation of a biofeedback visualization that helps to improve the understanding of brain activity. A final study shows how cognitive workload measurements can be used to predict the efficiency of information intake through reading interfaces. Here, we conclude with use cases and applications which benefit from cognitive augmentation. This thesis investigates how assistive systems can be designed to implicitly sense and utilize cognitive workload for input and output. To do so, we measure cognitive workload in real-time by collecting behavioral and physiological data from users and analyze this data to support users through assistive systems that adapt their interface according to the currently measured workload. Our overall goal is to extend new and existing context-aware applications by the factor cognitive workload. We envision Workload-Aware Systems and Workload-Aware Interfaces as an extension in the context-aware paradigm. To this end, we conducted eight research inquiries during this thesis to investigate how to design and create workload-aware systems. Finally, we present our vision of future workload-aware systems and workload-aware interfaces. Due to the scarce availability of open physiological data sets, reference implementations, and methods, previous context-aware systems were limited in their ability to utilize cognitive workload for user interaction. Together with the collected data sets, we expect this thesis to pave the way for methodical and technical tools that integrate workload-awareness as a factor for context-aware systems.Tagtäglich werden unsere kognitiven Fähigkeiten durch die Verarbeitung von unzähligen Informationen in Anspruch genommen. Dies kann die Schwierigkeit einer Aufgabe durch mehr oder weniger Arbeitslast beeinflussen. Der menschliche Körper drückt die Nutzung kognitiver Ressourcen durch physiologische Reaktionen aus, wenn dieser mit kognitiver Arbeitsbelastung konfrontiert oder überfordert wird. Dadurch werden weitere Ressourcen mobilisiert, um die Arbeitsbelastung vorübergehend zu bewältigen. Wir prognostizieren, dass die derzeitige Entwicklung physiologischer Messverfahren kognitive Leistungsmessungen stets möglich machen wird, um die kognitive Arbeitslast des Nutzers jederzeit zu messen. Diese sind in der Lage, einzugreifen wenn eine zu hohe oder zu niedrige kognitive Belastung erkannt wird. Wir konzentrieren uns zunächst auf die Erkennung passender Momente für kognitive Unterstützung welche sich der gegenwärtigen kognitiven Arbeitslast bewusst sind. Anschließend untersuchen wir in einem nutzerzentrierten Designprozess geeignete Feedbackmechanismen, die zur kognitiven Assistenz beitragen. Wir präsentieren Designanforderungen, welche zeigen wie Schnittstellen eine kognitive Augmentierung durch die Messung kognitiver Arbeitslast erreichen können. Anschließend untersuchen wir verschiedene physiologische Messmodalitäten, welche Bewertungen der kognitiven Arbeitsbelastung in Realzeit ermöglichen. Zunächst validieren wir empirisch, dass das menschliche Gehirn auf kognitive Arbeitslast reagiert. Es zeigt sich, dass die Ableitung der kognitiven Arbeitsbelastung über Elektroenzephalographie eine geeignete Methode ist, um den kognitiven Anspruch neuartiger Assistenzsysteme zu evaluieren. Anschließend verwenden wir Eye-Tracking, um Veränderungen in den Augenbewegungen und dem Durchmesser der Pupille unter verschiedenen Intensitäten kognitiver Arbeitslast zu bewerten. Das Anwenden von maschinellem Lernen führt zu zuverlässigen Echtzeit-Bewertungen kognitiver Arbeitsbelastung. Auf der Grundlage der bisherigen Forschungsarbeiten stellen wir Anwendungen vor, welche die Kognition im häuslichen und beruflichen Umfeld unterstützen. Die physiologischen Messungen stellen fest, wann eine kognitive Augmentierung sich als günstig erweist. In einer Feldstudie setzen wir ein Assistenzsystem ein, um die erhobenen Designanforderungen zur Reduktion kognitiver Arbeitslast zu validieren. Unsere Ergebnisse zeigen, dass die Arbeitsbelastung durch den Einsatz von Assistenzsystemen reduziert wird. Im Anschluss untersuchen wir, wie kognitive Arbeitsbelastung visualisiert werden kann. Wir stellen eine Implementierung einer Biofeedback-Visualisierung vor, die das Nutzerverständnis zum Verlauf und zur Entstehung von kognitiver Arbeitslast unterstützt. Eine abschließende Studie zeigt, wie Messungen kognitiver Arbeitslast zur Vorhersage der aktuellen Leseeffizienz benutzt werden können. Wir schließen hierbei mit einer Reihe von Applikationen ab, welche sich kognitive Arbeitslast als Eingabe zunutze machen. Die vorliegende wissenschaftliche Arbeit befasst sich mit dem Design von Assistenzsystemen, welche die kognitive Arbeitslast der Nutzer implizit erfasst und diese bei der Durchführung alltäglicher Aufgaben unterstützt. Dabei werden physiologische Daten erfasst, um Rückschlüsse in Realzeit auf die derzeitige kognitive Arbeitsbelastung zu erlauben. Anschließend werden diese Daten analysiert, um dem Nutzer strategisch zu assistieren. Das Ziel dieser Arbeit ist die Erweiterung neuartiger und bestehender kontextbewusster Benutzerschnittstellen um den Faktor kognitive Arbeitslast. Daher werden in dieser Arbeit arbeitslastbewusste Systeme und arbeitslastbewusste Benutzerschnittstellen als eine zusätzliche Dimension innerhalb des Paradigmas kontextbewusster Systeme präsentiert. Wir stellen acht Forschungsstudien vor, um die Designanforderungen und die Implementierung von kognitiv arbeitslastbewussten Systemen zu untersuchen. Schließlich stellen wir unsere Vision von zukünftigen kognitiven arbeitslastbewussten Systemen und Benutzerschnittstellen vor. Durch die knappe Verfügbarkeit öffentlich zugänglicher Datensätze, Referenzimplementierungen, und Methoden, waren Kontextbewusste Systeme in der Auswertung kognitiver Arbeitslast bezüglich der Nutzerinteraktion limitiert. Ergänzt durch die in dieser Arbeit gesammelten Datensätze erwarten wir, dass diese Arbeit den Weg für methodische und technische Werkzeuge ebnet, welche kognitive Arbeitslast als Faktor in das Kontextbewusstsein von Computersystemen integriert

6G Vision: Towards Future Collaborative Cognitive Communication (3C) Systems

6G networks are expected to have a breakthrough by enabling the emergence of collaborative cognitive communication services over heterogeneous environments for industry 5.0 applications. These applications are required to adapt human-centric approach to make the most of human intuition and intelligence in Industry 4.0 automation.It calls for a transdisciplinarity research domain to investigate innovative systems with overlapping realms of Psychology, Sociology, Communication networks, Artificial Intelligence , Natural Language Processing and Collaborative Computing. The author at the Cognitive Systems Research Centre, London South Bank University has coined the expression “3C Systems" to refer to such artifacts which stands for "Collaborative Cognitive Communication Systems”. In this paper, an innovative framework for 3C Systems is proposed that is able to analyze and predict both the human as well as machine behaviors. It proactively diagnoses issues and recommends solutions without requiring any human intervention. The proposed concept of 3C Systems would potentially contribute towards 6G standardization. The automation and orchestration aspects of this research have variety of applications stretched across city infrastructures, retail, business, tourism, health, law, education and travel. A thorough insight to a broad view of 6G vision has been presented towards envisioned 3C Systems, while covering its enabling technologies. The experimental results for the proof of concept implementation has been presented. Results affirm the technical capabilities of the concept, to contribute to several industry 5.0 applications including, but not limited to holographic communication, self-driving vehicles, context-aware infrastructure and personalized interfaces

Towards a multidisciplinary user-centric design framework for context-aware applications

The primary aim of this article is to review and merge theories of context within linguistics, computer science, and psychology, to propose a multidisciplinary model of context that would facilitate application developers in developing richer descriptions or scenarios of how a context-aware device may be used in various dynamic mobile settings. More specifically, the aim is to:1. Investigate different viewpoints of context within linguistics, computer science, and psychology, to develop summary condensed models for each discipline. 2. Investigate the impact of contrasting viewpoints on the usability of context-aware applications. 3. Investigate the extent to which single-discipline models can be merged and the benefits and insightfulness of a merged model for designing mobile computers. 4. Investigate the extent to which a proposed multidisciplinary modelcan be applied to specific applications of context-aware computing

Interactive guiding and localization platform

Technology is now part of everyone’s life, being widespread and present in the most innocuous objects, making people able to benefit from several technological solutions that only a few years ago were impossible to be implemented. For instance, the smartphone is one example of this type of technology. Usually, this device is equipped with several sensors and a GPS module, which may be used as an orientation system when it carries the right features. The current trend is providing advanced applications to the common public, excluding some fringes of the population, like the people with cognitive disabilities. Thus, everyday technology can be used for helping on several tasks, becoming an interactive terminal, aware of the user conditions. There may be orientation applications that are too complex to be used by people with cognitive disabilities. Therefore, the main goal of this work is to build and implement an orientation method that may be used by people with mild cognitive disabilities and provides several tools to the caregivers. Furthermore, the system is linked to external services like iGenda so the user is stimulated to go outside and increase his/her social participation or not to miss an appointment. The developed system is also a localization system, enabling caregivers to know in real-time the location of the person with cognitive disabilities. Allowing the user to have an independent life.The work of João Ramos was supported by the Portuguese Science and Technology Foundation (FCT - Fundação para a Ciência e a Tecnologia) through the doctoral grant SFRH/BD/- 89530/2012. The work of Angelo Costa through the Project ”AAL4ALL”, co-financed by the European Community Fund FEDER, through COMPETE - Programa Operacional Factores de Competitividade (POFC). The work of Paulo Novais through the project CAMCoF - Context-aware Multimodal Communication Framework funded by ERDF - European Regional Development Fund through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the FCT - Fundac¸ ˜ao para a Ciˆencia e a Tecnologia (Portuguese Foundation for Science and Technology) within project FCOMP-01-0124-FEDER-028980

Toward a multidisciplinary model of context to support context-aware computing

Capturing, defining, and modeling the essence of context are challenging, compelling, and prominent issues for interdisciplinary research and discussion. The roots of its emergence lie in the inconsistencies and ambivalent definitions across and within different research specializations (e.g., philosophy, psychology, pragmatics, linguistics, computer science, and artificial intelligence). Within the area of computer science, the advent of mobile context-aware computing has stimulated broad and contrasting interpretations due to the shift from traditional static desktop computing to heterogeneous mobile environments. This transition poses many challenging, complex, and largely unanswered research issues relating to contextual interactions and usability. To address those issues, many researchers strongly encourage a multidisciplinary approach. The primary aim of this article is to review and unify theories of context within linguistics, computer science, and psychology. Summary models within each discipline are used to propose an outline and detailed multidisciplinary model of context involving (a) the differentiation of focal and contextual aspects of the user and application's world, (b) the separation of meaningful and incidental dimensions, and (c) important user and application processes. The models provide an important foundation in which complex mobile scenarios can be conceptualized and key human and social issues can be identified. The models were then applied to different applications of context-aware computing involving user communities and mobile tourist guides. The authors' future work involves developing a user-centered multidisciplinary design framework (based on their proposed models). This will be used to design a large-scale user study investigating the usability issues of a context-aware mobile computing navigation aid for visually impaired people