Feature Relevance Analysis to Explain Concept Drift -- A Case Study in Human Activity Recognition

This article studies how to detect and explain concept drift. Human activity recognition is used as a case study together with a online batch learning situation where the quality of the labels used in the model updating process starts to decrease. Drift detection is based on identifying a set of features having the largest relevance difference between the drifting model and a model that is known to be accurate and monitoring how the relevance of these features changes over time. As a main result of this article, it is shown that feature relevance analysis cannot only be used to detect the concept drift but also to explain the reason for the drift when a limited number of typical reasons for the concept drift are predefined. To explain the reason for the concept drift, it is studied how these predefined reasons effect to feature relevance. In fact, it is shown that each of these has an unique effect to features relevance and these can be used to explain the reason for concept drift.Comment: Accepted to HASCA 2022 workshop in conjunction with UbiComp/ISWC202

Survey of data mining approaches to user modeling for adaptive hypermedia

The ability of an adaptive hypermedia system to create tailored environments depends mainly on the amount and accuracy of information stored in each user model. Some of the difficulties that user modeling faces are the amount of data available to create user models, the adequacy of the data, the noise within that data, and the necessity of capturing the imprecise nature of human behavior. Data mining and machine learning techniques have the ability to handle large amounts of data and to process uncertainty. These characteristics make these techniques suitable for automatic generation of user models that simulate human decision making. This paper surveys different data mining techniques that can be used to efficiently and accurately capture user behavior. The paper also presents guidelines that show which techniques may be used more efficiently according to the task implemented by the applicatio

Attention-Based Deep Learning Framework for Human Activity Recognition with User Adaptation

Sensor-based human activity recognition (HAR) requires to predict the action of a person based on sensor-generated time series data. HAR has attracted major interest in the past few years, thanks to the large number of applications enabled by modern ubiquitous computing devices. While several techniques based on hand-crafted feature engineering have been proposed, the current state-of-the-art is represented by deep learning architectures that automatically obtain high level representations and that use recurrent neural networks (RNNs) to extract temporal dependencies in the input. RNNs have several limitations, in particular in dealing with long-term dependencies. We propose a novel deep learning framework, \algname, based on a purely attention-based mechanism, that overcomes the limitations of the state-of-the-art. We show that our proposed attention-based architecture is considerably more powerful than previous approaches, with an average increment, of more than $7\%$ on the F1 score over the previous best performing model. Furthermore, we consider the problem of personalizing HAR deep learning models, which is of great importance in several applications. We propose a simple and effective transfer-learning based strategy to adapt a model to a specific user, providing an average increment of $6\%$ on the F1 score on the predictions for that user. Our extensive experimental evaluation proves the significantly superior capabilities of our proposed framework over the current state-of-the-art and the effectiveness of our user adaptation technique.Comment: Accepted for publication on the IEEE Sensors Journa

Affective Music Information Retrieval

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called \emph{acoustic emotion Gaussians} (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a larger-scale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.Comment: 40 pages, 18 figures, 5 tables, author versio

Machine Learning Based Physical Activity Extraction for Unannotated Acceleration Data

Sensor based human activity recognition (HAR) is an emerging and challenging research area. The physical activity of people has been associated with many health benefits and even reducing the risk of different diseases. It is possible to collect sensor data related to physical activities of people with wearable devices and embedded sensors, for example in smartphones and smart environments. HAR has been successful in recognizing physical activities with machine learning methods. However, it is a critical challenge to annotate sensor data in HAR. Most existing approaches use supervised machine learning methods which means that true labels need be given to the data when training a machine learning model. Supervised deep learning methods have outperformed traditional machine learning methods in HAR but they require an even more extensive amount of data and true labels. In this thesis, machine learning methods are used to develop a solution that can recognize physical activity (e.g., walking and sedentary time) from unannotated acceleration data collected using a wearable accelerometer device. It is shown to be beneficial to collect and annotate data from physical activity of only one person. Supervised classifiers can be trained with small, labeled acceleration data and more training data can be obtained in a semi-supervised setting by leveraging knowledge from available unannotated data. The semi-supervised En-Co-Training method is used with the traditional supervised machine learning methods K-nearest Neighbor and Random Forest. Also, intensities of activities are produced by the cut point analysis of the OMGUI software as reference information and used to increase confidence of correctly selecting pseudo-labels that are added to the training data. A new metric is suggested to help to evaluate reliability when no true labels are available. It calculates a fraction of predictions that have a correct intensity out of all the predictions according to the cut point analysis of the OMGUI software. The reliability of the supervised KNN and RF classifiers reaches 88 % accuracy and the C-index value 0,93, while the accuracy of the K-means clustering remains 72 % when testing the models on labeled acceleration data. The initial supervised classifiers and the classifiers retrained in a semi-supervised setting are tested on unlabeled data collected from 12 people and measured with the new metric. The overall results improve from 96-98 % to 98-99 %. The results with more challenging activities to the initial classifiers, taking a walk improve from 55-81 % to 67-81 % and jogging from 0-95 % to 95-98 %. It is shown that the results of the KNN and RF classifiers consistently increase in the semi-supervised setting when tested on unannotated, real-life data of 12 people

Humanoid Robot handling Hand-Signs Recognition

Recent advancements in human-robot interaction have led to tremendous improvement for humanoid robots but still lacks social acceptance among people. Though verbal communication is the primary means of human-robot interaction, non-verbal communication that is proven to be an integral part of the human interactions is not widely used in humanoid robots. This thesis aims to achieve human-robot interaction via non-verbal communication, especially using hand-signs. It presents a prototype system that simulates hand-signs recognition in the NAO humanoid robot, and further an online questionnaire is used to examine people's opinion on the use of non-verbal communication to interact with a humanoid robot. The positive results derived from the study indicates people's willingness to use non-verbal communication as a means to communicate with humanoid robots, thus encouraging robot designers to use non-verbal communications for enhancing human-robot interaction

Trends in human activity recognition using smartphones

AbstractRecognizing human activities and monitoring population behavior are fundamental needs of our society. Population security, crowd surveillance, healthcare support and living assistance, and lifestyle and behavior tracking are some of the main applications that require the recognition of human activities. Over the past few decades, researchers have investigated techniques that can automatically recognize human activities. This line of research is commonly known as Human Activity Recognition (HAR). HAR involves many tasks: from signals acquisition to activity classification. The tasks involved are not simple and often require dedicated hardware, sophisticated engineering, and computational and statistical techniques for data preprocessing and analysis. Over the years, different techniques have been tested and different solutions have been proposed to achieve a classification process that provides reliable results. This survey presents the most recent solutions proposed for each task in the human activity classification process, that is, acquisition, preprocessing, data segmentation, feature extraction, and classification. Solutions are analyzed by emphasizing their strengths and weaknesses. For completeness, the survey also presents the metrics commonly used to evaluate the goodness of a classifier and the datasets of inertial signals from smartphones that are mostly used in the evaluation phase

Artificial intelligence applications in marketing: the chatbot of the Department of Economics and Management "Marco Fanno”

openL'intelligenza artificiale (AI) offre numerose applicazioni nel marketing, ma allo stesso tempo ci sono diverse limitazioni da considerare nella sua adozione. Dopo la prima parte di analisi generale delle applicazioni e degli aspetti negativi dell'AI e dei chatbot, la tesi si concentra sul caso dell'implementazione di un chatbot da parte del Dipartimento di Economia e Management “Marco Fanno” dell'Università di Padova. La domanda di ricerca è volta a capire se il chatbot implementato dal Dipartimento sia stato efficace nell'alleggerire e supportare il lavoro dell'ufficio amministrativo e nel rispondere alle domande degli studenti. A tal fine, il documento analizza se il numero di email è diminuito dopo l'introduzione del chatbot. Inoltre è stato svolto un questionario per valutare l'esperienza che gli studenti del Dipartimento hanno avuto con il chatbot di ateneo. Il sondaggio ha anche chiesto agli studenti quali servizi vorrebbero che il chatbot aggiungesse a quelli attuali. Inoltre, è stata condotta un'analisi economica su benefici e costi per valutare se il chatbot genererà un risultato economico positivo. Questo studio consente di valutare l'impatto che un chatbot potrebbe avere nel campo dell'istruzione. In particolare, può fornire informazioni alle università sul fatto che un chatbot possa migliorare il coinvolgimento con gli studenti, liberare il personale da compiti ripetitivi e generare benefici economici netti nel lungo periodo. Il questionario stesso è stato condotto attraverso un sondaggio web su Google Forms e un sondaggio attraverso un chatbot. In questo modo ho anche analizzato quale dei due metodi sia il più efficace per condurre un'indagine. Alcune prove rivelano come i sondaggi condotti attraverso un chatbot possano portare a risposte più accurate da parte degli intervistati. Confrontando i risultati ottenuti della due modalità di sondaggio ho potuto verificare queste evidenze con un nuovo campione di partecipanti, gli studenti di Economia. I risultati della tesi non hanno mostrato prove chiare del fatto che il chatbot consentisse di ridurre il numero di e-mail. Ma si suggerisce un'indagine su un periodo più lungo. Successivamente i risultati hanno evidenziato un buon apprezzamento degli studenti per il chatbot e hanno suggerito l'introduzione di notifiche push che ricordano delle scadenze universitarie come le tasse. La stima dell'analisi costi-benefici prevedeva un risultato netto positivo su tre anni con un ROI del 29%. Inoltre, il sondaggio chatbot ha parzialmente confermato la tendenza ad ottenere risposte più accurate rispetto ad un classico sondaggio web.Artificial intelligence (AI) offers numerous applications in marketing, but at the same time, there are several limitations to consider in its adoption. After the first part about a general analysis of the applications and negative aspects of AI and chatbots, the thesis focuses on the case of the implementation of a chatbot by the Department of Economics and Management “Marco Fanno” of the University of Padua. The research question turns towards understanding whether the chatbot implemented by the Department was effective in easing and supporting the work of the administrative office and answering students questions. For this purpose, the paper analyses if the number of emails is decreased after the chatbot introduction. In addition, a questionnaire was carried out to evaluate the experience that the students of the Department have had with the university chatbot. The survey also asked students what services they would like the chatbot to add to their current ones. Moreover, an economic analysis on benefits and costs was conducted to estimate whether the chatbot will generate a positive outcome. This study allows evaluating the impact a chatbot could have in the education field. In particular, it can provide insight to universities on whether a chatbot could enhance the engagement with students, offload staff from repetitive tasks and generate net economic benefits in the long period. The questionnaire itself was conducted through a web survey on Google Forms and a chatbot survey. In this way, it could also be verified which of the two methods is the most effective to conduct a survey. Some evidence finds how chatbot surveys can lead to less satisfactory answers by respondents. Comparing the two survey results, I can verify these past findings with a different sample of participants, the students of Economics. The results did not show clear evidence of whether the chatbot allowed reducing the number of emails. But an investigation over a longer period is suggested. Then, findings highlighted a good appreciation of students for the chatbot and suggested the introduction of push notifications that remember university deadlines such as taxes. The estimation of the benefits-cost analysis forecasted a net positive outcome over three years with an ROI of 29%. Also, the chatbot survey partially confirmed the encouraging finding in reducing satisficing by respondents.

DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation [Technical Report]

We design, implement, and evaluate DeepEverest, a system for the efficient execution of interpretation by example queries over the activation values of a deep neural network. DeepEverest consists of an efficient indexing technique and a query execution algorithm with various optimizations. We prove that the proposed query execution algorithm is instance optimal. Experiments with our prototype show that DeepEverest, using less than 20% of the storage of full materialization, significantly accelerates individual queries by up to 63x and consistently outperforms other methods on multi-query workloads that simulate DNN interpretation processes