Learning Agent for a Heat-Pump Thermostat With a Set-Back Strategy Using Model-Free Reinforcement Learning

The conventional control paradigm for a heat pump with a less efficient auxiliary heating element is to keep its temperature set point constant during the day. This constant temperature set point ensures that the heat pump operates in its more efficient heat-pump mode and minimizes the risk of activating the less efficient auxiliary heating element. As an alternative to a constant set-point strategy, this paper proposes a learning agent for a thermostat with a set-back strategy. This set-back strategy relaxes the set-point temperature during convenient moments, e.g. when the occupants are not at home. Finding an optimal set-back strategy requires solving a sequential decision-making process under uncertainty, which presents two challenges. A first challenge is that for most residential buildings a description of the thermal characteristics of the building is unavailable and challenging to obtain. A second challenge is that the relevant information on the state, i.e. the building envelope, cannot be measured by the learning agent. In order to overcome these two challenges, our paper proposes an auto-encoder coupled with a batch reinforcement learning technique. The proposed approach is validated for two building types with different thermal characteristics for heating in the winter and cooling in the summer. The simulation results indicate that the proposed learning agent can reduce the energy consumption by 4-9% during 100 winter days and by 9-11% during 80 summer days compared to the conventional constant set-point strategyComment: Submitted to Energies - MDPI.co

Using an MBSE approach for automation control system selection in long steel products hot rolling plants

Abstract: Automation systems in long steel products hot rolling plants are prone to performance failures with the potential of serious negative impact on the business. The selection process of these automation systems therefore requires careful consideration of various selection factors to maximize plant performance. The need was therefore identified to investigate the use of a suitable management approach to guide engineering automation teams in the long steel products hot rolling plants in the selection of automation systems. At the core is the need for an in-depth understanding of the issues surrounding distributed and hierarchical automation systems in long steel products plants. This includes identifying the challenges during the selection process, using sound engineering management principles. Current automation selection techniques were investigated through a survey, interviews and a case study. It was then decided to use a Model Based Systems Engineering (MBSE) approach, which utilises systems engineering principles together with digital technology to create models to simplify the understanding of complex problems and relationships. This was then used to develop a management framework for automation systems selection in support of the business case of long steel products hot rolling plants

Voyager: An Open-Ended Embodied Agent with Large Language Models

We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent's abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize. We open-source our full codebase and prompts at https://voyager.minedojo.org/.Comment: Project website and open-source codebase: https://voyager.minedojo.org

Active Refrigerator

It can be desirable to alert a user to expired or spoiled food. For instance, consumption of spoiled food can negatively impact a user’s health. A refrigerator with active capabilities is disclosed herein that can detect a presence of spoiled food and alert a user to the presence of spoiled food

Multimodal forecasting methodology applied to industrial process monitoring

IEEE Industrial process modelling represents a key factor to allow the future generation of industrial manufacturing plants. In this regard, accurate models of critical signals need to be designed in order to forecast process deviations. In this work a novel multimodal forecasting methodology based on adaptive dynamics packaging and codification of the process operation is proposed. First, a target signal is decomposed by means of the Empirical Mode Decomposition in order to identify the characteristics intrinsic mode functions. Second, such dynamics are packaged depending on their significance and modelling complexity. Third, the operating condition of the considered process, reflected by available auxiliary signals, is codified by means of a Self-Organizing Map and presented to the modelling structure. The forecasting structure is supported by a set of ensemble ANFIS based models, each one focused on a different set of signal dynamics. The performance and effectiveness of the proposed method is validated experimentally with industrial data from a copper rod manufacturing plant and performance comparison with classical approaches. The proposed method improves performance and generalization versus classical single model approaches.Peer ReviewedPostprint (author's final draft

Robust classification of advanced power quality disturbances in smart grids

The insertion of new devices, increased data flow, intermittent generation and massive computerization have considerably increased current electrical systems’ complexity. This increase resulted in necessary changes, such as the need for more intelligent electrical net works to adapt to this different reality. Artificial Intelligence (AI) plays an important role in society, especially the techniques based on the learning process, and it is extended to the power systems. In the context of Smart Grids (SG), where the information and innovative solutions in monitoring is a primary concern, those techniques based on AI can present several applications. This dissertation investigates the use of advanced signal processing and ML algorithms to create a Robust Classifier of Advanced Power Quality (PQ) Dis turbances in SG. For this purpose, known models of PQ disturbances were generated with random elements to approach real applications. From these models, thousands of signals were generated with the performance of these disturbances. Signal processing techniques using Discrete Wavelet Transform (DWT) were used to extract the signal’s main charac teristics. This research aims to use ML algorithms to classify these data according to their respective features. ML algorithms were trained, validated, and tested. Also, the accuracy and confusion matrix were analyzed, relating the logic behind the results. The stages of data generation, feature extraction and optimization techniques were performed in the MATLAB software. The Classification Learner toolbox was used for training, validation and testing the 27 different ML algorithms and assess each performance. All stages of the work were previously idealized, enabling their correct development and execution. The results show that the Cubic Support Vector Machine (SVM) classifier achieved the maximum accuracy of all algorithms, indicating the effectiveness of the proposed method for classification. Considerations about the results were interpreted as explaining the per formance of each technique, its relations and their respective justifications.A inserção de novos dispositivos na rede, aumento do fluxo de dados, geração intermitente e a informatização massiva aumentaram consideravelmente a complexidade dos sistemas elétricos atuais. Esse aumento resultou em mudanças necessárias, como a necessidade de redes elétricas mais inteligentes para se adaptarem a essa realidade diferente. A nova ger ação de técnicas de Inteligência Artificial, representada pelo "Big Data", Aprendizado de Máquina ("Machine Learning"), Aprendizagem Profunda e Reconhecimento de Padrões representa uma nova era na sociedade e no desenvolvimento global baseado na infor mação e no conhecimento. Com as mais recentes Redes Inteligentes, o uso de técnicas que utilizem esse tipo de inteligência será ainda mais necessário. Esta dissertação investiga o uso de processamento de sinais avançado e também algoritmos de Aprendizagem de Máquina para desenvolver um classificador robusto de distúrbios de qualidade de energia no contexto das Redes Inteligentes. Para isso, modelos já conhecidos de alguns proble mas de qualidade foram gerados junto com ruídos aleatórios para que o sistema fosse similar a aplicações reais. A partir desses modelos, milhares de sinais foram gerados e a Transformada Wavelet Discreta foi usada para extrair as principais características destas perturbações. Esta dissertação tem como objetivo utilizar algoritmos baseados no con ceito de Aprendizado de Máquina para classificar os dados gerados de acordo com suas classes. Todos estes algoritmos foram treinados, validados e por fim, testados. Além disso, a acurácia e a matriz de confusão de cada um dos modelos foi apresentada e analisada. As etapas de geração de dados, extração das principais características e otimização dos dados foram realizadas no software MATLAB. Uma toolbox específica deste programa foi us ada para treinar, validar e testar os 27 algoritmos diferentes e avaliar cada desempenho. Todas as etapas do trabalho foram previamente idealizadas, possibilitando seu correto desenvolvimento e execução. Os resultados mostram que o classificador "Cubic Support Vector Machine" obteve a máxima precisão entre todos os algoritmos, indicando a eficácia do método proposto para classificação. As considerações sobre os resultados foram inter pretadas, como por exemplo a explicação da performance de cada técnica, suas relações e suas justificativas

SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks

We introduce SwiftSage, a novel agent framework inspired by the dual-process theory of human cognition, designed to excel in action planning for complex interactive reasoning tasks. SwiftSage integrates the strengths of behavior cloning and prompting large language models (LLMs) to enhance task completion performance. The framework comprises two primary modules: the Swift module, representing fast and intuitive thinking, and the Sage module, emulating deliberate thought processes. The Swift module is a small encoder-decoder LM fine-tuned on the oracle agent's action trajectories, while the Sage module employs LLMs such as GPT-4 for subgoal planning and grounding. We develop a heuristic method to harmoniously integrate the two modules, resulting in a more efficient and robust problem-solving process. In 30 tasks from the ScienceWorld benchmark, SwiftSage significantly outperforms other methods such as SayCan, ReAct, and Reflexion, demonstrating its effectiveness in solving complex real-world tasks.Comment: Project website: https://yuchenlin.xyz/swiftsage