Analysis of binarization techniques and Tsetlin machine architectures targeting image classification

Master's thesis in Information- and communication technology (IKT590)The Tsetlin Machine is a constantly evolving and developing machine learning technique with ever-increasing success. However, for every success, the Tsetlin Machine achieves, a new set of challenges are put ahead. To sufficiently bring the Tsetlin Machine to a broadly used standard, these challenges must be completed. This thesis focuses on the challenge of doing color image classification and will provide an introductory description of how this is possible through the usage of an older technique, namely binarization. A comparison with the various Tsetlin Machine adaptations made public in recent times is also present after the achieved color image classification. The results of both the initial color image classification experiment and the comparison between the varying adaptations show that the Tsetlin Machine, with a little extra work, can achieve high accuracy color image classification without image augmentation or pre-training

Playing endgame chess with the Tsetlin Machine

Master's thesis in Information- and communication technology (IKT590)The report is about training an Artificial Intelligence(AI) that is able to play out endgames, using the existing solved endgame of chess to train the Tsetlin Machine on. This report describes the methods used to train and test a Tsetlin Ma-chine using both the convolutional and multiclass implementation. We have further tested out different methods to handle the data it trains on to investigate what methods work best. Where these methods are; to split the data for two machines for either white or black starting player, transforming the data to only be from one starting players perspective and one splitting based on results by first having one machine looking at win versus draw and loss, then a second machine for looking at draw versus loss. The results showed that some of the methods used, involving only looking at one players perspective, worked well for predicting if the board would lead to a win with perfect play. Since several om the methods achieved over90% accuracy in the testing, while the best achieves an accuracy of 95%. However the playing off the endgame was lacking, as the games mostly ended in draws even when the Tsetlin Machine should have been able to win. Such as only drawing against each other, and only drawing against Monte Carlo Tree Search

TsetlinGo : Solving the game of Go with Tsetlin Machine

Master's thesis in Information- and communication technology (IKT590)The Tsetlin Machine have already shown great promise on pattern recognition and text categorization. The board game GO is a highly complex game, and the Tsetlin Machine have not yet been tested extensively on strategic games like this. This thesis introduces TsetlinGO and aims to Solve the game of Go with Tsetlin Machine. For predicting the next moves a combination of Tsetlin Machine and Tree Search was used. In the thesis a 9x9 board size was used for the game of Go, to prevent the problem from becoming to complex. This thesis goes through hyper-parameter testing for classification of the Go board game. A solution with Tree Search and Tsetlin Machine combined is used to perform self-play and matches between Tsetlin Machines with different hyper-parameters. Based on the empirical results, our conclusion is that the Tsetlin Machine is more than capable for classification of the game of Go at various stages of play. Results from the experiments could be seen to achieve around 90%, while further climbing up to around 95% upon re-training. From examining the clauses, strong patterns was found that gave insight into how the machine works. The Tsetlin Machine was able to play complete games of Go, making connections on the board through use of patterns from the clauses. It was found that the size of the clauses had great impact as clauses with large patterns had trouble getting triggered in early play. The high accuracy from classification was found to not correlate with how strong the Tsetlin Machine would perform during self-play. This may indicate that producing training data directly from self-play may be required to fine tune the assessment of board positions faced during actual play. We can conclude that this thesis provide a benchmark for further research within the field of Tsetlin Machine and the game of Go

Novel Tsetlin Machine Mechanisms for Logic-based Regression and Classification with Support for Continuous Input, Clause Weighting, Confidence Assessment, Deterministic Learning, and Convolution

publishedVersio

Enhancing Attention’s Explanation Using Interpretable Tsetlin Machine

Explainability is one of the key factors in Natural Language Processing (NLP) specially for legal documents, medical diagnosis, and clinical text. Attention mechanism has been a popular choice for such explainability recently by estimating the relative importance of input units. Recent research has revealed, however, that such processes tend to misidentify irrelevant input units when explaining them. This is due to the fact that language representation layers are initialized by pretrained word embedding that is not context-dependent. Such a lack of context-dependent knowledge in the initial layer makes it difficult for the model to concentrate on the important aspects of input. Usually, this does not impact the performance of the model, but the explainability differs from human understanding. Hence, in this paper, we propose an ensemble method to use logic-based information from the Tsetlin Machine to embed it into the initial representation layer in the neural network to enhance the model in terms of explainability. We obtain the global clause score for each word in the vocabulary and feed it into the neural network layer as context-dependent information. Our experiments show that the ensemble method enhances the explainability of the attention layer without sacrificing any performance of the model and even outperforming in some datasets.publishedVersio

Using Tsetlin Machine to discover interpretable rules in natural language processing applications

publishedVersio