An Improved Bees Algorithm for Training Deep Recurrent Networks for Sentiment Classification

Recurrent neural networks (RNNs) are powerful tools for learning information from temporal sequences. Designing an optimum deep RNN is difficult due to configuration and training issues, such as vanishing and exploding gradients. In this paper, a novel metaheuristic optimisation approach is proposed for training deep RNNs for the sentiment classification task. The approach employs an enhanced Ternary Bees Algorithm (BA-3+), which operates for large dataset classification problems by considering only three individual solutions in each iteration. BA-3+ combines the collaborative search of three bees to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. Local learning with exploitative search utilises the greedy selection strategy. Stochastic gradient descent (SGD) learning with singular value decomposition (SVD) aims to handle vanishing and exploding gradients of the decision parameters with the stabilisation strategy of SVD. Global learning with explorative search achieves faster convergence without getting trapped at local optima to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. BA-3+ has been tested on the sentiment classification task to classify symmetric and asymmetric distribution of the datasets from different domains, including Twitter, product reviews, and movie reviews. Comparative results have been obtained for advanced deep language models and Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. BA-3+ converged to the global minimum faster than the DE and PSO algorithms, and it outperformed the SGD, DE, and PSO algorithms for the Turkish and English datasets. The accuracy value and F1 measure have improved at least with a 30–40% improvement than the standard SGD algorithm for all classification datasets. Accuracy rates in the RNN model trained with BA-3+ ranged from 80% to 90%, while the RNN trained with SGD was able to achieve between 50% and 60% for most datasets. The performance of the RNN model with BA-3+ has as good as for Tree-LSTMs and Recursive Neural Tensor Networks (RNTNs) language models, which achieved accuracy results of up to 90% for some datasets. The improved accuracy and convergence results show that BA-3+ is an efficient, stable algorithm for the complex classification task, and it can handle the vanishing and exploding gradients problem of deep RNNs

Production Optimization Indexed to the Market Demand Through Neural Networks

Connectivity, mobility and real-time data analytics are the prerequisites for a new model of intelligent production management that facilitates communication between machines, people and processes and uses technology as the main driver. Many works in the literature treat maintenance and production management in separate approaches, but there is a link between these areas, with maintenance and its actions aimed at ensuring the smooth operation of equipment to avoid unnecessary downtime in production. With the advent of technology, companies are rushing to solve their problems by resorting to technologies in order to fit into the most advanced technological concepts, such as industries 4.0 and 5.0, which are based on the principle of process automation. This approach brings together database technologies, making it possible to monitor the operation of equipment and have the opportunity to study patterns of data behavior that can alert us to possible failures. The present thesis intends to forecast the pulp production indexed to the stock market value.The forecast will be made by means of the pulp production variables of the presses and the stock exchange variables supported by artificial intelligence (AI) technologies, aiming to achieve an effective planning. To support the decision of efficient production management, in this thesis algorithms were developed and validated with from five pulp presses, as well as data from other sources, such as steel production and stock exchange, which were relevant to validate the robustness of the model. This thesis demonstrated the importance of data processing methods and that they have great relevance in the model input since they facilitate the process of training and testing the models. The chosen technologies demonstrated good efficiency and versatility in performing the prediction of the values of the variables of the equipment, also demonstrating robustness and optimization in computational processing. The thesis also presents proposals for future developments, namely in further exploration of these technologies, so that there are market variables that can calibrate production through forecasts supported on these same variables.Conectividade, mobilidade e análise de dados em tempo real são pré-requisitos para um novo modelo de gestão inteligente da produção que facilita a comunicação entre máquinas, pessoas e processos, e usa a tecnologia como motor principal. Muitos trabalhos na literatura tratam a manutenção e a gestão da produção em abordagens separadas, mas existe uma correlação entre estas áreas, sendo que a manutenção e as suas políticas têm como premissa garantir o bom funcionamento dos equipamentos de modo a evitar paragens desnecessárias na linha de produção. Com o advento da tecnologia há uma corrida das empresas para solucionar os seus problemas recorrendo às tecnologias, visando a sua inserção nos conceitos tecnológicos, mais avançados, tais como as indústrias 4.0 e 5.0, as quais têm como princípio a automatização dos processos. Esta abordagem junta as tecnologias de sistema de informação, sendo possível fazer o acompanhamento do funcionamento dos equipamentos e ter a possibilidade de realizar o estudo de padrões de comportamento dos dados que nos possam alertar para possíveis falhas. A presente tese pretende prever a produção da pasta de papel indexada às bolsas de valores. A previsão será feita por via das variáveis da produção da pasta de papel das prensas e das variáveis da bolsa de valores suportadas em tecnologias de artificial intelligence (IA), tendo como objectivo conseguir um planeamento eficaz. Para suportar a decisão de uma gestão da produção eficiente, na presente tese foram desenvolvidos algoritmos, validados em dados de cinco prensas de pasta de papel, bem como dados de outras fontes, tais como, de Produção de Aço e de Bolsas de Valores, os quais se mostraram relevantes para a validação da robustez dos modelos. A presente tese demonstrou a importância dos métodos de tratamento de dados e que os mesmos têm uma grande relevância na entrada do modelo, visto que facilita o processo de treino e testes dos modelos. As tecnologias escolhidas demonstraram uma boa eficiência e versatilidade na realização da previsão dos valores das variáveis dos equipamentos, demonstrando ainda robustez e otimização no processamento computacional. A tese apresenta ainda propostas para futuros desenvolvimentos, designadamente na exploração mais aprofundada destas tecnologias, de modo a que haja variáveis de mercado que possam calibrar a produção através de previsões suportadas nestas mesmas variáveis