38 research outputs found

    Using Long Short-Term Memory Networks to Make and Train Neural Network Based Pseudo Random Number Generator

    Get PDF
    Neural Networks have been used in many decision-making models and been employed in computer vision, and natural language processing. Several works have also used Neural Networks for developing Pseudo-Random Number Generators [2, 4, 5, 7, 8]. However, despite great performance in the National Institute of Standards and Technology (NIST) statistical test suite for randomness, they fail to discuss how the complexity of a neural network affects such statistical results. This work introduces: 1) a series of new Long Short- Term Memory Network (LSTM) based and Fully Connected Neural Network (FCNN – baseline [2] + variations) Pseudo Random Number Generators (PRNG) and 2) an LSTMbased predictor. The thesis also performs adversarial training to determine two things: 1) How the use of sequence models such as LSTMs after adversarial training affects the performance on NIST tests. 2) To study how the complexity of the fully connected network-based generator in [2] and the LSTM-based generator affects NIST results. Experiments were done on four different sets of generators and predictors, i) Fully Connected Neural Network Generator (FC NN Gen) – Convolutional Neural Network Predictor (CNN Pred), ii) FC NN Gen - LSTM Pred, iii) LSTM-based Gen – CNN. Pred, iv) LSTM-based Gen – LSTM Pred, where FC NN Gen and CNN Pred were taken as the baseline from [2] while LSTM-based Gen and LSTM Pred were proposed. Based on the experiments, LSTM Predictor overall gave much consistent and even better results on the NIST test suite than the CNN Predictor from [2]. It was observed that using LSTM generator showed a higher pass rate for NIST test on average when paired with LSTM Predictor but a very low fluctuating trend. On the other hand, an increasing trend was observed for the average NIST test passing rate when the same generator was trained with CNN Predictor in an adversarial environment. The baseline [2] and its variations however only displayed a fluctuating trend, but with better results with the adversarial training with the LSTM-based Predictor than the CNN Predictor

    PARALLEL NUMERICAL COMPUTATION: A COMPARATIVE STUDY ON CPU-GPU PERFORMANCE IN PI DIGITS COMPUTATION

    Get PDF
    As the usage of GPU (Graphical Processing Unit) for non-graphical computation is rising, one important area is to study how the device helps improve numerical calculations. In this work, we present a time performance comparison between purely CPU (serial) and GPU-assisted (parallel) programs in numerical computation. Specifically, we design and implement the calculation of the hexadecimal -digit of the irrational number Pi in two ways: serial and parallel. Both programs are based upon the BBP formula for Pi in the form of infinite series identity. We then provide a detailed time performance analysis of both programs based on the magnitude. Our result shows that the GPU-assisted parallel algorithm ran a hundred times faster than the serial algorithm. To be more precise, we offer that as the value  grows, the ratio between the execution time of the serial and parallel algorithms also increases. Moreover, when  it is large enough, that is This GPU efficiency ratio converges to a constant, showing the GPU's maximally utilized capacity. On the other hand, for sufficiently small enough, the serial algorithm performed solely on the CPU works faster since the GPU's small usage of parallelism does not help much compared to the arithmetic complexity

    Dynamical Systems in Spiking Neuromorphic Hardware

    Get PDF
    Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case

    Hybrid optimisation and formation of index tracking portfolio in TSE

    Get PDF
    Asset allocation and portfolio optimisation are some of the most important steps in an investors decision making process. In order to manage uncertainty and maximise returns, it is assumed that active investment is a zero-sum game. It is possible however, that market inefficiencies could provide the necessary opportunities for investors to beat the market. In this study we examined a core-satellite approach to gain higher returns than that of the market. The core component of the portfolio consists of an index-tracking portfolio which has been formulated using a meta-heuristic genetic algorithm, allowing for the efficient search of the solution space for an optimal (or near-optimal) solution. The satellite component is made up of publicly traded active managed funds and the weights of each component are optimised using mathematical modelling (quadratics) to maximise the returns of the resultant portfolio.In order to address uncertainty within the model variables, robustness is introduced into the objective function of the model in the form of risk tolerance (Degree of uncertainty). The introduction of robustness as a variable allows us to assess the resultant model in worst-case circumstances and determine suitable levels of risk tolerance. Further attempts at implementing additional robustness within the model using an artificial neural network in an LSTM configuration were inconclusive, suggesting that LSTM networks were unable to make informative predictions on the future returns of the index because market efficiencies render historical data irrelevant and market movement is akin to a random walk. A framework is offered for the formation and optimisation of a hybrid multi-stage core-satellite portfolio which manages risk through the implementation of robustness and passive investment, whilst attempting to beat the market in terms of returns. Using daily returns data from the Tehran Stock Exchange for a four-year period, it is shown that the resultant core-satellite portfolio is able to beat the market considerably after training.Results indicate that the tracking ability of the portfolio is affected by the number of its constituents, that there is a specific time frame of 70 days after which the resultant portfolio needs to be re assessed and readjusted and that the implementation of robustness as a degree of uncertainty variable within the objective function increases the correlation coefficient and reduces tracking error.Keywords: Index Funds, Index Tracking, Passive Portfolio Management, Robust Optimisation, Core Satellite Investment, Quadratic Optimisation, Genetic Algorithms, LSTM, Heuristic Neural Networks, Efficient Market Hypothesis, Modern Portfolio Theory, Portfolio optimisatio

    Improving and Scaling Mobile Learning via Emotion and Cognitive-state Aware Interfaces

    Get PDF
    Massive Open Online Courses (MOOCs) provide high-quality learning materials at low cost to millions of learners. Current MOOC designs, however, have minimal learner-instructor communication channels. This limitation restricts MOOCs from addressing major challenges: low retention rates, frequent distractions, and little personalization in instruction. Previous work enriched learner-instructor communication with physiological signals but was not scalable because of the additional hardware requirement. Large MOOC providers, such as Coursera, have released mobile apps providing more flexibility with “on-the-go” learning environments. This thesis reports an iterative process for the design of mobile intelligent interfaces that can run on unmodified smartphones, implicitly sense multiple modalities from learners, infer learner emotions and cognitive states, and intervene to provide gains in learning. The first part of this research explores the usage of photoplethysmogram (PPG) signals collected implicitly on the back-camera of unmodified smartphones. I explore different deep neural networks, DeepHeart, to improve the accuracy (+2.2%) and robustness of heart rate sensing from noisy PPG signals. The second project, AttentiveLearner, infers mind-wandering events via the collected PPG signals at a performance comparable to systems relying on dedicated physiological sensors (Kappa = 0.22). By leveraging the fine-grained cognitive states, the third project, AttentiveReview, achieves significant (+17.4%) learning gains by providing personalized interventions based on learners’ perceived difficulty. The latter part of this research adds real-time facial analysis from the front camera in addition to the PPG sensing from the back camera. AttentiveLearner2 achieves more robust emotion inference (average accuracy = 84.4%) in mobile MOOC learning. According to a longitudinal study with 28 subjects for three weeks, AttentiveReview2, with the multimodal sensing component, improves learning gain by 28.0% with high usability ratings (average System Usability Scale = 80.5). Finally, I show that technologies in this dissertation not only benefit MOOC learning, but also other emerging areas such as computational advertising and behavior targeting. AttentiveVideo, building on top of the sensing architecture in AttentiveLearner2, quantifies emotional responses to mobile video advertisements. In a 24-participant study, AttentiveVideo achieved good accuracy on a wide range of emotional measures (best accuracy = 82.6% across 9 measures)

    Explainable NLP for Human-AI Collaboration

    Get PDF
    With more data and computing resources available these days, we have seen many novel Natural Language Processing (NLP) models breaking one performance record after another. Some of them even outperform human performance in some specific tasks. Meanwhile, many researchers have revealed weaknesses and irrationality of such models, e.g., having biases against some sub-populations, producing inconsistent predictions, and failing to work effectively in the wild due to overfitting. Therefore, in real applications, especially in high-stakes domains, humans cannot rely carelessly on predictions of NLP models, but they need to work closely with the models to ensure that every final decision made is accurate and benevolent. In this thesis, we devise and utilize explainable NLP techniques to support human-AI collaboration using text classification as a target task. Overall, our contributions can be divided into three main parts. First, we study how useful explanations are for humans according to three different purposes: revealing model behavior, justifying model predictions, and helping humans investigate uncertain predictions. Second, we propose a framework that enables humans to debug simple deep text classifiers informed by model explanations. Third, leveraging on computational argumentation, we develop a novel local explanation method for pattern-based logistic regression models that align better with human judgements and effectively assist humans to perform an unfamiliar task in real-time. Altogether, our contributions are paving the way towards the synergy of profound knowledge of human users and the tireless power of AI machines.Open Acces

    Multimodal fake news detection using a Cultural Algorithm with situational and normative knowledge

    Get PDF
    The proliferation of fake news on social media sites is a serious problem with documented negative impacts on individuals and organizations. This makes detection of fake news an extremely important challenge. A fake news item is usually created by manipulating photos, text or videos that indicate the need for multimodal detection. Researchers are building detection algorithms with the aim of high accuracy as this will have a massive impact on the prevailing social and political issues. A shortcoming of existing strategies for identifying fake news is their inability to learn a feature representation of multimodal (textual+visual) information. In this thesis research, we present a novel approach using a Cultural Algorithm with situational and normative knowledge to detect fake news using both text and images. The proposed model’s principal innovation is to use the power of natural language processing like sentiment analysis, segmentation process for feature extraction, and optimizing it with a Cultural algorithm. Then the representations from both modalities are fused, which is ïŹnally used for classiïŹcation. An extensive set of experiments is carried out on real-world multimedia datasets collected from Weibo and Twitter. The proposed method outperforms the state-of-the-art methods for identifying fake new
    corecore