Sigmoid Activation-Based Long Short-Term Memory for Time Series Data Classification

With the enhanced usage of Artificial Intelligence (AI) driven applications, the researchers often face challenges in improving the accuracy of the data classification models, while trading off the complexity. In this paper, we address the classification of time series data using the Long Short-Term Memory (LSTM) network while focusing on the activation functions. While the existing activation functions such as sigmoid and tanh are used as LSTM internal activations, the customizability of these activations stays limited. This motivates us to propose a new family of activation functions, called log-sigmoid, inside the LSTM cell for time series data classification, and analyze its properties. We also present the use of a linear transformation (e.g., log tanh) of the proposed log-sigmoid activation as a replacement of the traditional tanh function in the LSTM cell. Both the cell activation as well as recurrent activation functions inside the LSTM cell are modified with log-sigmoid activation family while tuning the log bases. Further, we report a comparative performance analysis of the LSTM model using the proposed and the state-of-the-art activation functions on multiple public time-series databases

Implementation of recurrent neural network for the forecasting of USD buy rate against IDR

This study implements a recurrent neural network (RNN) by comparing two RNN network structures, namely Elman and Jordan using the backpropagation through time (BPTT) programming algorithm in the training and forecasting process in foreign exchange forecasting cases. The activation functions used are the linear transfer function, the tan-sigmoid transfer function (Tansig), and the log-sigmoid transfer function (Logsig), which are applied to the hidden and output layers. The application of the activation function results in the log-sigmoid transfer function being the most appropriate activation function for the hidden layer, while the linear transfer function is the most appropriate activation function for the output layer. Based on the results of training and forecasting the USD against IDR currency, the Elman BPTT method is better than the Jordan BPTT method, with the best iteration being the 4000th iteration for both. The lowest root mean square error (RMSE) values for training and forecasting produced by Elman BPTT were 0.073477 and 122.15 the following day, while the Jordan backpropagation RNN method yielded 0.130317 and 222.96 also the following day.

THE USE OF NEURAL NETWORKS IN THE OPERATIONAL RISK DATA MODELING

In this article it is presented a proposal of improving the data analysis process of Operational Risk (OpRisk) assessment in the financial institutions, for the Loss Distribution Approach (LDA) method, using the Artificial Intelligence (AI). In the first part of the paper a substitute tool of the traditional model-based Autoregressive Moving Average (ARMA) is described, for analyzing and representing stochastic processes. An Artificial Neural Network (ANN) is particularly suitable for this challenge, especially when dealing with limited data sets. In this case, an ANN is able to operate model-free by extracting the pattern of the training data set and by learning from the data observed during the generalized delta rule back-propagation training. The proposed ANN is a time lagged Feed-Forward Network (FFN) with log-sigmoid activation function.Operational Risk, Advanced Measurement Approach, Loss Distribution Approach, Artificial Neural Networks, Genetic Algorithms

Kecelaruan personaliti antisosial di kalangan pelajar politeknik : satu kajian awal

Kajian ini adalah bertujuan untuk mengenalpasti kecelaruan personalis antisosial (KPA) yang berlaku di kalangan remaja atau muda-mudi terutama di Politeknik Malaysia yang mungkin mengakibatkan berlakunya masalah sosial di kalangan mereka. Kajian ini berbentuk kuantitatif. Sampel kajian telah dipilih di empat buah politeknik. Politeknik yang terlibat adalah politeknik zon selatan. Responden kajian ini terdiri daripada 340 orang pelajar pengambilan bam semester satu yang memasuki institusi berkenaan. Responden juga terdiri daripada pelajar peringkat sijil dan diploma daripada pelbagai pengkhususan. Instrumen yang digunakan adalah borang soal selidik. Data yang telah dikumpulkan dianalisis menggunakan Statistical Package for Social Science (SPSS). Statistik yang digunakan adalah statistik deskriptif. Dapatan kajian menunjukkan di antara 10 jenis kecelaruan, kecelaruan avoidant mencatatkan skor min tertinggi iaitu dengan skor min 3.24 (a = 1.055). Selain itu, pengkaji mendapati personaliti antisosial yang berlaku di kalangan pelajar politeknik adalah pada tahap yang sederhana iaitu skor min 2.35 (a =0.933). Hasil daripada kajian juga mendapati faktor sosial mencatatkan skor min tertinggi iaitu 2.07 (a = 0.851). Faktor keluarga pula hanya mencatatkan skor min 2.03 (g = 0.887). Pengkaji juga mendapati responden lebih gemar kepada konsep keagamaan berbanding konsep-konsep yang lain sekiranya mereka menghadapi masalah. Oleh itu diharapkan kajian ini dapat memberikan penjelasan sedikit sebanyak mengenai kecelaruan personaliti antisosial yang berlaku di kalangan pelajar politeknik di masa kini

Pembangunan kerangka transferable skills bagi perlaksanaan penyelidikan dalam kalangan pelajar pascasiswazah di Malaysia

Malaysia berhasrat menjadi negara maju dan berpendapatan tinggi maka keperluan sumber manusia profesional iaitu graduan pascasiswazah adalah semakin mendesak. Namun demikian, timbul isu tentang tekanan yang dihadapi pelajar dalam menjalankan penyelidikan, seperti putus asa, hilang minat, hilang keyakinan diri, tidak fokus, mengalami tekanan mental, ketandusan idea, tidak mencapai target yang diinginkan, hilang komitmen dan gagal dalam menamatkan pengajian. Terdapat keperluan terhadap peranan transferable skills untuk melakukan pelbagai aktiviti, untuk mencapai sasaran dan menyelesaikan masalah yang timbul sepanjang proses penyelidikan. Oleh itu, kajian ini dilaksanakan untuk membangunkan kerangka transferable skills bagi perlaksanaan penyelidikan dalam kalangan pelajar pascasiswazah di Malaysia. Dalam kajian ini, pengkaji menggunakan reka bentuk penerokaan bercampur berurutan yang melibatkan kajian kualitatif dan kajian kuantitatif. Peserta temu bual iaitu seramai 11 orang pakar dan peserta kajian Fuzzy Delphi iaitu 13 orang pakar, yang telah dipilih menggunakan kaedah persampelan bertujuan. Sampel bagi kajian tinjauan pula iaitu seramai 483 pelajar pascasiswazah dalam bidang sains sosial dan kemanusiaan di universiti awam yang terdapat di Malaysia, telah dipilih menggunakan kaedah pensampelan rawak berlapis mengikut kadar. Dapatan kajian ini menunjukkan bahawa terdapat enam domain transferable skills dan 22 elemen transferable skills. Kajian ini mendapati bahawa pelajar Sarjana dan Doktor Falsafah memberikan tahap persetujuan yang tinggi terhadap enam domain dan 22 elemen transferable skills. Hasil dapatan kajian ini menunjukkan bahawa tidak terdapat perbezaan kesesuaian domain dan elemen transferable skills untuk menjalankan proses penyelidikan berdasarkan pelajar Sarjana dan Doktor Falsafah (PhD). Kajian ini juga mendapati bahawa kerangka transferable skills yang dibangunkan adalah sah dan boleh dipercayai untuk menjadi panduan bagi perlaksanaan penyelidikan dalam kalangan pelajar pascasiswazah di Malaysia. Oleh yang demikian, pengkaji berharap kerangka transferable skills yang dibangunkan melalui kajian ini dapat menjadi panduan bagi pelajar pascasiswazah untuk mencapai target yang diinginkan dan dapat menyelesaikan penyelidikan sebagaimana tempoh yang ditetapkan sehingga berjaya menamatkan pengajian

Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together

Neural networks equipped with self-attention have parallelizable computation, light-weight structure, and the ability to capture both long-range and local dependencies. Further, their expressive power and performance can be boosted by using a vector to measure pairwise dependency, but this requires to expand the alignment matrix to a tensor, which results in memory and computation bottlenecks. In this paper, we propose a novel attention mechanism called "Multi-mask Tensorized Self-Attention" (MTSA), which is as fast and as memory-efficient as a CNN, but significantly outperforms previous CNN-/RNN-/attention-based models. MTSA 1) captures both pairwise (token2token) and global (source2token) dependencies by a novel compatibility function composed of dot-product and additive attentions, 2) uses a tensor to represent the feature-wise alignment scores for better expressive power but only requires parallelizable matrix multiplications, and 3) combines multi-head with multi-dimensional attentions, and applies a distinct positional mask to each head (subspace), so the memory and computation can be distributed to multiple heads, each with sequential information encoded independently. The experiments show that a CNN/RNN-free model based on MTSA achieves state-of-the-art or competitive performance on nine NLP benchmarks with compelling memory- and time-efficiency

Artificial Neural Network Utilization for FSO Link Performance Estimation

This paper describes FSO link performance prediction based on available meteorological data using different Artificial Neural Network (ANN) approaches. Several types of ANNs were compared and their performance were evaluated. The paper introduces an ANN application utilizing real delayed data. This approach has been validated to be more precise than common feed-forward neural networks