A decade of Semantic Web research through the lenses of a mixed methods approach

The identification of research topics and trends is an important scientometric activity, as it can help guide the direction of future research. In the Semantic Web area, initially topic and trend detection was primarily performed through qualitative, top-down style approaches, that rely on expert knowledge. More recently, data-driven, bottom-up approaches have been proposed that offer a quantitative analysis of the evolution of a research domain. In this paper, we aim to provide a broader and more complete picture of Semantic Web topics and trends by adopting a mixed methods methodology, which allows for the combined use of both qualitative and quantitative approaches. Concretely, we build on a qualitative analysis of the main seminal papers, which adopt a top-down approach, and on quantitative results derived with three bottom-up data-driven approaches (Rexplore, Saffron, PoolParty), on a corpus of Semantic Web papers published between 2006 and 2015. In this process, we both use the latter for “fact-checking” on the former and also to derive key findings in relation to the strengths and weaknesses of top-down and bottom up approaches to research topic identification. Although we provide a detailed study on the past decade of Semantic Web research, the findings and the methodology are relevant not only for our community but beyond the area of the Semantic Web to other research fields as well

A Systematic Review of Urban Navigation Systems for Visually Impaired People

Blind and Visually impaired people (BVIP) face a range of practical difficulties when undertaking outdoor journeys as pedestrians. Over the past decade, a variety of assistive devices have been researched and developed to help BVIP navigate more safely and independently. In~addition, research in overlapping domains are addressing the problem of automatic environment interpretation using computer vision and machine learning, particularly deep learning, approaches. Our aim in this article is to present a comprehensive review of research directly in, or relevant to, assistive outdoor navigation for BVIP. We breakdown the navigation area into a series of navigation phases and tasks. We then use this structure for our systematic review of research, analysing articles, methods, datasets and current limitations by task. We also provide an overview of commercial and non-commercial navigation applications targeted at BVIP. Our review contributes to the body of knowledge by providing a comprehensive, structured analysis of work in the domain, including the state of the art, and guidance on future directions. It will support both researchers and other stakeholders in the domain to establish an informed view of research progress

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring

Artificially intelligent perception is increasingly present in the lives of every one of us. Vehicles are no exception, (...) In the near future, pattern recognition will have an even stronger role in vehicles, as self-driving cars will require automated ways to understand what is happening around (and within) them and act accordingly. (...) This doctoral work focused on advancing in-vehicle sensing through the research of novel computer vision and pattern recognition methodologies for both biometrics and wellbeing monitoring. The main focus has been on electrocardiogram (ECG) biometrics, a trait well-known for its potential for seamless driver monitoring. Major efforts were devoted to achieving improved performance in identification and identity verification in off-the-person scenarios, well-known for increased noise and variability. Here, end-to-end deep learning ECG biometric solutions were proposed and important topics were addressed such as cross-database and long-term performance, waveform relevance through explainability, and interlead conversion. Face biometrics, a natural complement to the ECG in seamless unconstrained scenarios, was also studied in this work. The open challenges of masked face recognition and interpretability in biometrics were tackled in an effort to evolve towards algorithms that are more transparent, trustworthy, and robust to significant occlusions. Within the topic of wellbeing monitoring, improved solutions to multimodal emotion recognition in groups of people and activity/violence recognition in in-vehicle scenarios were proposed. At last, we also proposed a novel way to learn template security within end-to-end models, dismissing additional separate encryption processes, and a self-supervised learning approach tailored to sequential data, in order to ensure data security and optimal performance. (...)Comment: Doctoral thesis presented and approved on the 21st of December 2022 to the University of Port

Usable privacy and security in smart homes

Ubiquitous computing devices increasingly dominate our everyday lives, including our most private places: our homes. Homes that are equipped with interconnected, context-aware computing devices, are considered “smart” homes. To provide their functionality and features, these devices are typically equipped with sensors and, thus, are capable of collecting, storing, and processing sensitive user data, such as presence in the home. At the same time, these devices are prone to novel threats, making our homes vulnerable by opening them for attackers from outside, but also from within the home. For instance, remote attackers who digitally gain access to presence data can plan for physical burglary. Attackers who are physically present with access to devices could access associated (sensitive) user data and exploit it for further cyberattacks. As such, users’ privacy and security are at risk in their homes. Even worse, many users are unaware of this and/or have limited means to take action. This raises the need to think about usable mechanisms that can support users in protecting their smart home setups. The design of such mechanisms, however, is challenging due to the variety and heterogeneity of devices available on the consumer market and the complex interplay of user roles within this context. This thesis contributes to usable privacy and security research in the context of smart homes by a) understanding users’ privacy perceptions and requirements for usable mechanisms and b) investigating concepts and prototypes for privacy and security mechanisms. Hereby, the focus is on two specific target groups, that are inhabitants and guests of smart homes. In particular, this thesis targets their awareness of potential privacy and security risks, enables them to take control over their personal privacy and security, and illustrates considerations for usable authentication mechanisms. This thesis provides valuable insights to help researchers and practitioners in designing and evaluating privacy and security mechanisms for future smart devices and homes, particularly targeting awareness, control, and authentication, as well as various roles.Computer und andere „intelligente“, vernetzte Geräte sind allgegenwärtig und machen auch vor unserem privatesten Zufluchtsort keinen Halt: unserem Zuhause. Ein „intelligentes Heim“ verspricht viele Vorteile und nützliche Funktionen. Um diese zu erfüllen, sind die Geräte mit diversen Sensoren ausgestattet – sie können also in unserem Zuhause sensitive Daten sammeln, speichern und verarbeiten (bspw. Anwesenheit). Gleichzeitig sind die Geräte anfällig für (neuartige) Cyberangriffe, gefährden somit unser Zuhause und öffnen es für potenzielle – interne sowie externe – Angreifer. Beispielsweise könnten Angreifer, die digital Zugriff auf sensitive Daten wie Präsenz erhalten, einen physischen Überfall in Abwesenheit der Hausbewohner planen. Angreifer, die physischen Zugriff auf ein Gerät erhalten, könnten auf assoziierte Daten und Accounts zugreifen und diese für weitere Cyberangriffe ausnutzen. Damit werden die Privatsphäre und Sicherheit der Nutzenden in deren eigenem Zuhause gefährdet. Erschwerend kommt hinzu, dass viele Nutzenden sich dessen nicht bewusst sind und/oder nur limitierte Möglichkeiten haben, effiziente Gegenmaßnahmen zu ergreifen. Dies macht es unabdingbar, über benutzbare Mechanismen nachzudenken, die Nutzende beim Schutz ihres intelligenten Zuhauses unterstützen. Die Umsetzung solcher Mechanismen ist allerdings eine große Herausforderung. Das liegt unter anderem an der großen Vielfalt erhältlicher Geräte von verschiedensten Herstellern, was das Finden einer einheitlichen Lösung erschwert. Darüber hinaus interagieren im Heimkontext meist mehrere Nutzende in verschieden Rollen (bspw. Bewohner und Gäste), was die Gestaltung von Mechanismen zusätzlich erschwert. Diese Doktorarbeit trägt dazu bei, benutzbare Privatsphäre- und Sicherheitsmechanismen im Kontext des „intelligenten Zuhauses“ zu entwickeln. Insbesondere werden a) die Wahrnehmung von Privatsphäre sowie Anforderungen an potenzielle Mechanismen untersucht, sowie b) Konzepte und Prototypen für Privatsphäre- und Sicherheitsmechanismen vorgestellt. Der Fokus liegt hierbei auf zwei Zielgruppen, den Bewohnern sowie den Gästen eines intelligenten Zuhauses. Insbesondere werden in dieser Arbeit deren Bewusstsein für potenzielle Privatsphäre- und Sicherheits-Risiken adressiert, ihnen Kontrolle über ihre persönliche Privatsphäre und Sicherheit ermöglicht, sowie Möglichkeiten für benutzbare Authentifizierungsmechanismen für beide Zielgruppen aufgezeigt. Die Ergebnisse dieser Doktorarbeit legen den Grundstein für zukünftige Entwicklung und Evaluierung von benutzbaren Privatsphäre und Sicherheitsmechanismen im intelligenten Zuhause

Face sketch recognition using deep learning

Face sketch recognition refers to automatically identifying a person from a set of facial photos using a face sketch. This thesis focuses on matching facial images between front face photos and front face hand-drawn sketches, and between front face photos and front face composite sketches by software. Because different visual domains, different image forms, and different collection methods exist between the matching image pairs, face sketch recognition is more difficult than traditional facial recognition. In this thesis, three novel deep learning models are presented to increase recognition accuracy on face photo-sketch datasets. An improved Siamese network combined with features extracted from an encoder-decoder network is proposed to extract more correlated features from facial photos and the corresponding face sketches. After that, attention modules are proposed to extract features from the same location in the photos and the sketches. In the third method, in order to reduce the difference between different visual domains, the images are transferred into a graph to increase the relationship for different face attributes and facial landmarks. Meanwhile, the graph neural network is utilized to learn the weights of neighbors adaptively. The first is to fuse more image features from the Siamese network and encoder-decoder network for increased the recognition results. Moreover, the attention modules can fix the similarity positions from different domain images to extract the correlated features. The visualized feature maps exhibit the correlated features which are extracted from the photo and the corresponding face sketch. In addition, a stable deep learning model based on graph structure is introduced to capture the topology of the graph and the relationship after images have been mapped into the graph structure for reducing the gap between face photos and face sketches. The experimental results show that the recognition accuracy of our proposed deep learning models can achieve the state-of-the-art on composite face sketch datasets. Meanwhile, the recognition results on hand-drawn face sketch datasets exceed other deep learning methods

Modelling the use of 3D video on the quality of experience

Последњих година, очигледан је брз развој различитих медија у различитим сферама као што су потрошачка електроника, аутомобилска инфо-забава (енгл. Infotainment), софтверa у сврху здравства итд. Због тога намеће се потреба за иновативним методама процене квалитета доживљаја (енгл. Quality of Experience - QoE) које корисници доживљавају као замену за задовољство потрошача таквих системима и услугама. Емоционално стање корисника игра кључну улогу у области QoE; стога га је неопходно узети у обзир приликом процене корисничког искуства и процеса дизајнирања 3Д видео садржаја. У овој докторској дисертацији представљено је моделовање проценитеља квалитета доживљава заснованог на повратној вишеслојној вештачкој неуронској мрежи као одговарајућој техници машинског учења за процену човековог емоционалног стања током гледања различитих типова 3Д видео садржаја. Циљ је дизајнирање проценитеља емоционалног стања на основу директних психо-физиолошких мерења. Разматрани психо- физиолошки сигнали укључују срчану фреквенцију (HR) израчунату на основу ехо-кардиограма (ECG), електро-дермалну активност (EDA) и активност мозга (BA) у електро-енцефалографским (EEG) сигналима. Експериментални део истраживања постављен је тако да су учесници гледали серију 3Д видео садржаја који се разликују у погледу визуелног квалитета и типа садржаја, док су поменути психо-физиолошки сигнали забележени помоћу специјалних сонди постављених у моменту гледања садржаја, а субјективно проживљене емоције пријављене помоћу упитника за самопроцену (SAM). Добијени резултати показују да је могуће конструисати тако високо прецизан процењивач емоционалних стања.Poslednjih godina, očigledan je brz razvoj različitih medija u različitim sferama kao što su potrošačka elektronika, automobilska info-zabava (engl. Infotainment), softvera u svrhu zdravstva itd. Zbog toga nameće se potreba za inovativnim metodama procene kvaliteta doživljaja (engl. Quality of Experience - QoE) koje korisnici doživljavaju kao zamenu za zadovoljstvo potrošača takvih sistemima i uslugama. Emocionalno stanje korisnika igra ključnu ulogu u oblasti QoE; stoga ga je neophodno uzeti u obzir prilikom procene korisničkog iskustva i procesa dizajniranja 3D video sadržaja. U ovoj doktorskoj disertaciji predstavljeno je modelovanje procenitelja kvaliteta doživljava zasnovanog na povratnoj višeslojnoj veštačkoj neuronskoj mreži kao odgovarajućoj tehnici mašinskog učenja za procenu čovekovog emocionalnog stanja tokom gledanja različitih tipova 3D video sadržaja. Cilj je dizajniranje procenitelja emocionalnog stanja na osnovu direktnih psiho-fizioloških merenja. Razmatrani psiho- fiziološki signali uključuju srčanu frekvenciju (HR) izračunatu na osnovu eho-kardiograma (ECG), elektro-dermalnu aktivnost (EDA) i aktivnost mozga (BA) u elektro-encefalografskim (EEG) signalima. Eksperimentalni deo istraživanja postavljen je tako da su učesnici gledali seriju 3D video sadržaja koji se razlikuju u pogledu vizuelnog kvaliteta i tipa sadržaja, dok su pomenuti psiho-fiziološki signali zabeleženi pomoću specijalnih sondi postavljenih u momentu gledanja sadržaja, a subjektivno proživljene emocije prijavljene pomoću upitnika za samoprocenu (SAM). Dobijeni rezultati pokazuju da je moguće konstruisati tako visoko precizan procenjivač emocionalnih stanja.In recent years, the rapid development of diverse media has been evident in disparate fields such as consumer electronics, automotive infotainment and healthcare software. There is a need for innovative methods to assess user perceived Quality of Experience (QoE), as a proxy for consumer satisfaction with such systems and services. Users emotional state plays a key role in QoE; thus, it is necessary to consider it in user experience evaluation and the design process of stereoscopic 3D video content. In the PhD thesis the use of a specially designed model based on a feedforward Multilayer Perception Artificial Neural Network as an appropriate Machine Learning technique for the estimation of human emotional state while viewing various categories of stereoscopic 3D video content is introduced. The goal is to design an emotional state estimator based on direct psychophysiological measurements. The considered psychophysiological signals include heart rate (HR) calculated from an echocardiogram (ECG), electro-dermal activity (EDA), and brain activity (BA) in EEG signals. In the experimental part of study, participants watched a series of 3D video contents varying in terms of visual quality and type of content, while the mentioned psychophysiological signals were recorded via specific equipment, and self-reported subjectively experienced emotions using a Self-Assessment Manikin (SAM) questionnaire. The obtained results show that it is possible to construct such a highly precise estimator of emotional state

Intelligent Transportation Related Complex Systems and Sensors

Building around innovative services related to different modes of transport and traffic management, intelligent transport systems (ITS) are being widely adopted worldwide to improve the efficiency and safety of the transportation system. They enable users to be better informed and make safer, more coordinated, and smarter decisions on the use of transport networks. Current ITSs are complex systems, made up of several components/sub-systems characterized by time-dependent interactions among themselves. Some examples of these transportation-related complex systems include: road traffic sensors, autonomous/automated cars, smart cities, smart sensors, virtual sensors, traffic control systems, smart roads, logistics systems, smart mobility systems, and many others that are emerging from niche areas. The efficient operation of these complex systems requires: i) efficient solutions to the issues of sensors/actuators used to capture and control the physical parameters of these systems, as well as the quality of data collected from these systems; ii) tackling complexities using simulations and analytical modelling techniques; and iii) applying optimization techniques to improve the performance of these systems. It includes twenty-four papers, which cover scientific concepts, frameworks, architectures and various other ideas on analytics, trends and applications of transportation-related data