519 research outputs found
Multi-head attention-based long short-term memory for depression detection from speech.
Depression is a mental disorder that threatens the health and normal life of people. Hence, it is essential to provide an effective way to detect depression. However, research on depression detection mainly focuses on utilizing different parallel features from audio, video, and text for performance enhancement regardless of making full usage of the inherent information from speech. To focus on more emotionally salient regions of depression speech, in this research, we propose a multi-head time-dimension attention-based long short-term memory (LSTM) model. We first extract frame-level features to store the original temporal relationship of a speech sequence and then analyze their difference between speeches of depression and those of health status. Then, we study the performance of various features and use a modified feature set as the input of the LSTM layer. Instead of using the output of the traditional LSTM, multi-head time-dimension attention is employed to obtain more key time information related to depression detection by projecting the output into different subspaces. The experimental results show the proposed model leads to improvements of 2.3 and 10.3% over the LSTM model on the Distress Analysis Interview Corpus-Wizard of Oz (DAIC-WOZ) and the Multi-modal Open Dataset for Mental-disorder Analysis (MODMA) corpus, respectively
Recommended from our members
IMPROVING CREDIT CARD FRAUD DETECTION USING TRANSFER LEARNING AND DATA RESAMPLING TECHNIQUES
This Culminating Experience Project explores the use of machine learning algorithms to detect credit card fraud. The research questions are: Q1. What cross-domain techniques developed in other domains can be effectively adapted and applied to mitigate or eliminate credit card fraud, and how do these techniques compare in terms of fraud detection accuracy and efficiency? Q2. To what extent do synthetic data generation methods effectively mitigate the challenges posed by imbalanced datasets in credit card fraud detection, and how do these methods impact classification performance? Q3. To what extent can the combination of transfer learning and innovative data resampling techniques improve the accuracy and efficiency of credit card fraud detection systems when dealing with imbalanced datasets, and what novel strategies can be developed to address this common challenge?
The main findings are: Q1. Unconventional cross-domain methods improved fraud detection, holding promise for enhanced security. Q2. The problems caused by unbalanced datasets in credit card fraud detection were effectively addressed by the synthetic data generation techniques SMOTE and ADASYN, resulting in a more balanced dataset suitable for fraud classification. Q3. The combination of neural networks and data resampling techniques, such as SMOTE and ADASYN, significantly improved credit card fraud detection accuracy.
The main conclusions are: Q1. Cross-domain methods are useful for credit card fraud detection, especially when it comes to online transactions. Q2. When used with various classifiers, neural networks show remarkable accuracy rates: 97% for unbalanced data, 99.47% for SMOTE, and 99.11% for ADASYN Q3. A fraud recall of 0.99 is obtained by the model evaluation on imbalanced data, with 12,155 right predictions out of 12,336 and 181 incorrect ones. The identified areas for further study encompass the testing of our model on larger datasets and the optimization of hyperparameters for further enhancement
From Private to Public: Benchmarking GANs in the Context of Private Time Series Classification
Deep learning has proven to be successful in various domains and for
different tasks. However, when it comes to private data several restrictions
are making it difficult to use deep learning approaches in these application
fields. Recent approaches try to generate data privately instead of applying a
privacy-preserving mechanism directly, on top of the classifier. The solution
is to create public data from private data in a manner that preserves the
privacy of the data. In this work, two very prominent GAN-based architectures
were evaluated in the context of private time series classification. In
contrast to previous work, mostly limited to the image domain, the scope of
this benchmark was the time series domain. The experiments show that especially
GSWGAN performs well across a variety of public datasets outperforming the
competitor DPWGAN. An analysis of the generated datasets further validates the
superiority of GSWGAN in the context of time series generation.Comment: 18 pages, 6 figures, 6 table
Continual learning from stationary and non-stationary data
Continual learning aims at developing models that are capable of working on constantly evolving problems over a long-time horizon. In such environments, we can distinguish three essential aspects of training and maintaining machine learning models - incorporating new knowledge, retaining it and reacting to changes. Each of them poses its own challenges, constituting a compound problem with multiple goals.
Remembering previously incorporated concepts is the main property of a model that is required when dealing with stationary distributions. In non-stationary environments, models should be capable of selectively forgetting outdated decision boundaries and adapting to new concepts. Finally, a significant difficulty can be found in combining these two abilities within a single learning algorithm, since, in such scenarios, we have to balance remembering and forgetting instead of focusing only on one aspect.
The presented dissertation addressed these problems in an exploratory way. Its main goal was to grasp the continual learning paradigm as a whole, analyze its different branches and tackle identified issues covering various aspects of learning from sequentially incoming data. By doing so, this work not only filled several gaps in the current continual learning research but also emphasized the complexity and diversity of challenges existing in this domain. Comprehensive experiments conducted for all of the presented contributions have demonstrated their effectiveness and substantiated the validity of the stated claims
Towards Foundation Models for Learning on Tabular Data
Learning on tabular data underpins numerous real-world applications. Despite
considerable efforts in developing effective learning models for tabular data,
current transferable tabular models remain in their infancy, limited by either
the lack of support for direct instruction following in new tasks or the
neglect of acquiring foundational knowledge and capabilities from diverse
tabular datasets. In this paper, we propose Tabular Foundation Models (TabFMs)
to overcome these limitations. TabFMs harness the potential of generative
tabular learning, employing a pre-trained large language model (LLM) as the
base model and fine-tuning it using purpose-designed objectives on an extensive
range of tabular datasets. This approach endows TabFMs with a profound
understanding and universal capabilities essential for learning on tabular
data. Our evaluations underscore TabFM's effectiveness: not only does it
significantly excel in instruction-following tasks like zero-shot and
in-context inference, but it also showcases performance that approaches, and in
instances, even transcends, the renowned yet mysterious closed-source LLMs like
GPT-4. Furthermore, when fine-tuning with scarce data, our model achieves
remarkable efficiency and maintains competitive performance with abundant
training data. Finally, while our results are promising, we also delve into
TabFM's limitations and potential opportunities, aiming to stimulate and
expedite future research on developing more potent TabFMs
Mask Off: Analytic-based Malware Detection By Transfer Learning and Model Personalization
The vulnerability of smartphones to cyberattacks has been a severe concern to
users arising from the integrity of installed applications (\textit{apps}).
Although applications are to provide legitimate and diversified on-the-go
services, harmful and dangerous ones have also uncovered the feasible way to
penetrate smartphones for malicious behaviors. Thorough application analysis is
key to revealing malicious intent and providing more insights into the
application behavior for security risk assessments. Such in-depth analysis
motivates employing deep neural networks (DNNs) for a set of features and
patterns extracted from applications to facilitate detecting potentially
dangerous applications independently. This paper presents an Analytic-based
deep neural network, Android Malware detection (ADAM), that employs a
fine-grained set of features to train feature-specific DNNs to have consensus
on the application labels when their ground truth is unknown. In addition, ADAM
leverages the transfer learning technique to obtain its adjustability to new
applications across smartphones for recycling the pre-trained model(s) and
making them more adaptable by model personalization and federated learning
techniques. This adjustability is also assisted by federated learning guards,
which protect ADAM against poisoning attacks through model analysis. ADAM
relies on a diverse dataset containing more than 153000 applications with over
41000 extracted features for DNNs training. The ADAM's feature-specific DNNs,
on average, achieved more than 98% accuracy, resulting in an outstanding
performance against data manipulation attacks
- …