Neural Networks for Complex Data

Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world problems, ranging from time evolving data to sophisticated data structures such as graphs and functions. This paper summarizes advances on those themes from the last decade, with a focus on results obtained by members of the SAMM team of Universit\'e Paris

Graphs in machine learning: an introduction

Graphs are commonly used to characterise interactions between objects of interest. Because they are based on a straightforward formalism, they are used in many scientific fields from computer science to historical sciences. In this paper, we give an introduction to some methods relying on graphs for learning. This includes both unsupervised and supervised methods. Unsupervised learning algorithms usually aim at visualising graphs in latent spaces and/or clustering the nodes. Both focus on extracting knowledge from graph topologies. While most existing techniques are only applicable to static graphs, where edges do not evolve through time, recent developments have shown that they could be extended to deal with evolving networks. In a supervised context, one generally aims at inferring labels or numerical values attached to nodes using both the graph and, when they are available, node characteristics. Balancing the two sources of information can be challenging, especially as they can disagree locally or globally. In both contexts, supervised and un-supervised, data can be relational (augmented with one or several global graphs) as described above, or graph valued. In this latter case, each object of interest is given as a full graph (possibly completed by other characteristics). In this context, natural tasks include graph clustering (as in producing clusters of graphs rather than clusters of nodes in a single graph), graph classification, etc. 1 Real networks One of the first practical studies on graphs can be dated back to the original work of Moreno [51] in the 30s. Since then, there has been a growing interest in graph analysis associated with strong developments in the modelling and the processing of these data. Graphs are now used in many scientific fields. In Biology [54, 2, 7], for instance, metabolic networks can describe pathways of biochemical reactions [41], while in social sciences networks are used to represent relation ties between actors [66, 56, 36, 34]. Other examples include powergrids [71] and the web [75]. Recently, networks have also been considered in other areas such as geography [22] and history [59, 39]. In machine learning, networks are seen as powerful tools to model problems in order to extract information from data and for prediction purposes. This is the object of this paper. For more complete surveys, we refer to [28, 62, 49, 45]. In this section, we introduce notations and highlight properties shared by most real networks. In Section 2, we then consider methods aiming at extracting information from a unique network. We will particularly focus on clustering methods where the goal is to find clusters of vertices. Finally, in Section 3, techniques that take a series of networks into account, where each network i

On-line relational and multiple relational SOM

International audienceIn some applications and in order to address real-world situations better, data may be more complex than simple numerical vectors. In some examples, data can be known only through their pairwise dissimilarities or through multiple dissimilarities, each of them describing a particular feature of the data set. Several variants of the Self Organizing Map (SOM) algorithm were introduced to generalize the original algorithm to the framework of dissimilarity data. Whereas median SOM is based on a rough representation of the prototypes, relational SOM allows representing these prototypes by a virtual linear combination of all elements in the data set, referring to a pseudo-euclidean framework. In the present article, an on-line version of relational SOM is introduced and studied. Similarly to the situation in the Euclidean framework, this on-line algorithm provides a better organization and is much less sensible to prototype initialization than standard (batch) relational SOM. In a more general case, this stochastic version allows us to integrate an additional stochastic gradient descent step in the algorithm which can tune the respective weights of several dissimilarities in an optimal way: the resulting \emph{multiple relational SOM} thus has the ability to integrate several sources of data of different types, or to make a consensus between several dissimilarities describing the same data. The algorithms introduced in this manuscript are tested on several data sets, including categorical data and graphs. On-line relational SOM is currently available in the R package SOMbrero that can be downloaded at http://sombrero.r-forge.r-project.org or directly tested on its Web User Interface at http://shiny.nathalievilla.org/sombrero

Stock market predictions using machine learning

2021 Spring.Includes bibliographical references.In this thesis, an attempt is made to try and establish the impact of news articles and correlated stocks on any one stock. Stock prices are dependent on many factors, some of which are common for most stocks, and some are specific to a type of company. For instance, a product-based company's stocks are dependent on sales and profit, while a research-based company's stocks are based on the progress made in their research over a specified time period. The main idea behind this thesis is that using news articles, we can potentially estimate how much each of these factors can impact the stock prices and how much of it is based on common factors like momentum. This thesis is split into three parts. The first part is finding the correlated stocks for a selected stock ticker. Correlated stocks can have a significant impact on stock prices; having a diverse portfolio of non-correlated stocks is very important for a stock trader, and yet very little research has been done on this part from a computer science point of view. The second part is to use Long-Short Term Memory on a pre-compiled list of news articles for the selected stock ticker; this enables us to understand which articles might have some influence on the stock prices. The third part is to combine the two and compare the result to stock predictions made using the deep neural network on the stock prices during the same period. The selected companies for the experiment are - Microsoft, Google, Netflix, Apple, Nvidia, AMD, Amazon. The companies were selected based on their popularity on the Internet, which makes it easier to get more articles on the companies. If we look at the day to day movement in stock prices, a typical regression approach can give reasonably accurate results on stock prices, but where this method fails is in predicting the significant changes in prices that are not based on trends or momentum. For instance, if a company releases a faulty product but the hype for the product is high prior to the release, the trends would show a positive direction for the stocks and a regression approach would most likely not predict the fall in the prices right after the news of the fault is made public. It will eventually correct itself, but it would not be instantaneous. Using a news-based approach, it is possible to predict the fall in stocks before the change is noticed in the actual stock price. This approach seems to show success to a varying degree with Microsoft showing the best accuracy of 91.46%, and AMD had the lowest at 40.59% on the test dataset. This was probably because of the volatility of AMD's stock prices, and this volatility could be caused by factors other than the news such as the impact of some other third-party companies. While the news articles can help predict specific stock movements, we still need a trend based regression approach for the day to day stock movements. The second part of the thesis is focused on this part of the stock predictions. It incorporates the results from these news articles into another neural network to predict the actual stock prices of each of the companies. The second neural network takes the percentage change in stock price from one day to the next as the input along with the predicted values from the news articles to predict the value of the stock for the next day. This approach seems to produce mixed results. AMD's predicted values seem to be worse when incorporated with only the news articles

Interpretable predictions with Convolutional Neural Networks for complex data

Deep Learning (DL) and Artificial Intelligence (AI) are nowadays one of the most used tools to analyze massive and complex data sets. Despite being very flexible and powerful, Artificial Neural Networks (ANN) are often denoted as "black-box" methods; the causal association between predictions and data is not straightforward and easy to explain. This thesis is focused on three applications with complex data of 1-D Convolutional Neural Networks (1-D CNN), a specific type of ANN. Through Explainable Artificial Intelligence (XAI) algorithms, 1-D CNN-based predictions can be made interpretable. Firstly, we considered the possibility of improving the diagnosis of malignant tumors through the classification of Raman spectra of genomic DNA. Much of the focus is dedicated to discerning different sub-cell lines of the same tumor. Next, 1-D CNN has been implemented to predict El Niño Southen Oscillation (ENSO) from Zebiack-Cane (ZC) simulated data. We tried to understand what 1-D CNN can learn about these events' physical dynamics when trying to distort the parameters ruling the ocean-atmosphere coupling. Last, a joint work with the ICU-Department at UMCU for improving the treatment of ICU patients is presented: 1-D CNN was utilized to predict nosocomial ICU-Acquired Infections (ICU-AI) dynamically. Specifically, 1-D CNN has been trained to score the risk of an ICU-AI onset only by analyzing the massive amount of information available from the ICU monitors. The actual ICU-AI prediction has been provided through Survival Analysis techniques after embedding the 1-D CNN analysis into a wider set of traditional explainable variables

AI - Powered Procedural Content Generation: Enhancing NPC Behaviour for an Immersive Gaming Experience

This paper explores the integration of artificial intelligence (AI) and procedural content generation (PCG) in video game development, with a specific focus on enhancing non-player character (NPC) behaviours. It discusses advancements in PCG driven by deep learning to dynamically create game content, transforming gaming experiences through adaptive, personalised environments. Additionally, it summarises how AI integration has evolved NPC interactions to be more immersive and context aware. Techniques covered include reinforcement learning for strategic decisions, neural networks for complex data processing, and natural language processing for realistic conversations. When evaluating the use of PCG for NPC behaviours, key strengths highlighted are increased diversity and replay ability along with inherent limitations in replicating manually authored complexity. Overall, this research highlights AI's profound impact on gaming by pushing the frontiers of procedural content generation to unlock more captivating, dynamic virtual worlds. Further AI-driven innovations promise to enable the next generation of intuitive game design and interactive experiences