Machine Learning for Classical Planning: Neural Network Heuristics, Online Portfolios, and State Space Topologies

State space search solves navigation tasks and many other real world problems. Heuristic search, especially greedy best-first search, is one of the most successful algorithms for state space search. We improve the state of the art in heuristic search in three directions. In Part I, we present methods to train neural networks as powerful heuristics for a given state space. We present a universal approach to generate training data using random walks from a (partial) state. We demonstrate that our heuristics trained for a specific task are often better than heuristics trained for a whole domain. We show that the performance of all trained heuristics is highly complementary. There is no clear pattern, which trained heuristic to prefer for a specific task. In general, model-based planners still outperform planners with trained heuristics. But our approaches exceed the model-based algorithms in the Storage domain. To our knowledge, only once before in the Spanner domain, a learning-based planner exceeded the state-of-the-art model-based planners. A priori, it is unknown whether a heuristic, or in the more general case a planner, performs well on a task. Hence, we trained online portfolios to select the best planner for a task. Today, all online portfolios are based on handcrafted features. In Part II, we present new online portfolios based on neural networks, which receive the complete task as input, and not just a few handcrafted features. Additionally, our portfolios can reconsider their choices. Both extensions greatly improve the state-of-the-art of online portfolios. Finally, we show that explainable machine learning techniques, as the alternative to neural networks, are also good online portfolios. Additionally, we present methods to improve our trust in their predictions. Even if we select the best search algorithm, we cannot solve some tasks in reasonable time. We can speed up the search if we know how it behaves in the future. In Part III, we inspect the behavior of greedy best-first search with a fixed heuristic on simple tasks of a domain to learn its behavior for any task of the same domain. Once greedy best-first search expanded a progress state, it expands only states with lower heuristic values. We learn to identify progress states and present two methods to exploit this knowledge. Building upon this, we extract the bench transition system of a task and generalize it in such a way that we can apply it to any task of the same domain. We can use this generalized bench transition system to split a task into a sequence of simpler searches. In all three research directions, we contribute new approaches and insights to the state of the art, and we indicate interesting topics for future work

Horizontally distributed inference of deep neural networks for AI-enabled IoT

Motivated by the pervasiveness of artificial intelligence (AI) and the Internet of Things (IoT) in the current “smart everything” scenario, this article provides a comprehensive overview of the most recent research at the intersection of both domains, focusing on the design and development of specific mechanisms for enabling a collaborative inference across edge devices towards the in situ execution of highly complex state-of-the-art deep neural networks (DNNs), despite the resource-constrained nature of such infrastructures. In particular, the review discusses the most salient approaches conceived along those lines, elaborating on the specificities of the partitioning schemes and the parallelism paradigms explored, providing an organized and schematic discussion of the underlying workflows and associated communication patterns, as well as the architectural aspects of the DNNs that have driven the design of such techniques, while also highlighting both the primary challenges encountered at the design and operational levels and the specific adjustments or enhancements explored in response to them.Agencia Estatal de Investigación | Ref. DPI2017-87494-RMinisterio de Ciencia e Innovación | Ref. PDC2021-121644-I00Xunta de Galicia | Ref. ED431C 2022/03-GR

Modeling and Communicating Flexibility in Smart Grids Using Artificial Neural Networks as Surrogate Models

Increasing shares of renewable energies and the transition towards electric vehicles pose major challenges to the energy system. In order to tackle these in an economically sensible way, the flexibility of distributed energy resources (DERs), such as battery energy storage systems, combined heat and power plants, and heat pumps, needs to be exploited. Modeling and communicating this flexibility is a fundamental step when trying to achieve control over DERs. The literature proposes and makes use of many different approaches, not only for the exploitation itself, but also in terms of models. In the first step, this thesis presents an extensive literature review and a general framework for classifying exploitation approaches and the communicated models. Often, the employed models only apply to specific types of DERs, or the models are so abstract that they neglect constraints and only roughly outline the true flexibility. Surrogate models, which are learned from data, can pose as generic DER models and may potentially be trained in a fully automated process. In this thesis, the idea of encoding the flexibility of DERs into ANNs is systematically investigated. Based on the presented framework, a set of ANN-based surrogate modeling approaches is derived and outlined, of which some are only applicable for specific use cases. In order to establish a baseline for the approximation quality, one of the most versatile identified approaches is evaluated in order to assess how well a set of reference models is approximated. If this versatile model is able to capture the flexibility well, a more specific model can be expected to do so even better. The results show that simple DERs are very closely approximated, and for more complex DERs and combinations of multiple DERs, a high approximation quality can be achieved by introducing buffers. Additionally, the investigated approach has been tested in scheduling tasks for multiple different DERs, showing that it is indeed possible to use ANN-based surrogates for the flexibility of DERs to derive load schedules. Finally, the computational complexity of utilizing the different approaches for controlling DERs is compared

Efeito do resveratrol sobre parâmetros astrocíticos e permeabilidade de barreira hematoencefálica no modelo animal de autismo induzido por exposição pré-natal ao ácido valproico

O Transtorno do Espectro Autista (TEA) é uma desordem do neurodesenvolvimento de elevada prevalência, afetando 1:59 crianças de até 8 anos de idade nos Estados Unidos. De acordo com a 5ª edição do Manual Diagnóstico e Estatístico de Transtornos Mentais (DSM-5), o diagnóstico do TEA é dado pela díade comportamental composta por prejuízos na comunicação e interação social, bem como por interesses restritos, comportamentos repetitivos e estereotipados. Além das alterações comportamentais, o TEA pode apresentar muitas comorbidades associadas e um importante achado clínico é a presença de maior volume encefálico nos primeiros anos de vida. Embora a etiologia do TEA permaneça desconhecida, sabe-se da importância de fatores genéticos e ambientais, como a exposição pré-natal ao ácido valproico (VPA). O resveratrol (RSV) é uma molécula anti-inflamatória, anti-oxidante e neuroprotetora conhecida por prevenir comportamentos sociais no modelo animal de autismo induzido por exposição pré-natal ao VPA. O objetivo desse trabalho foi avaliar os efeitos do VPA na permeabilidade da barreira hematoencefálica (BHE), nas células GFAP+ e na aquaporina 4 (AQP4) no córtex pré-frontal medial (CPFm) de ratos machos de 30 dias e o possível efeito protetivo do RSV. Ratas Wistar prenhes receberam doses diárias de 3.6 mg/kg de RSV ou DMSO (veículo) via subcutânea do dia embrionário 6.5 (E6.5) até E18.5 e no dia E12.5 foi administrada uma dose intraperitoneal de 600 mg/kg de VPA ou solução salina (0,9%). No dia pós-natal 30 (P30) os animais foram eutanasiados por sobredose anestésica de cetamina (300 mg/kg) e xilazina (40 mg/kg) e destinados a diferentes técnicas experimentais. Os animais avaliados para a permeabilidade da BHE ao corante azul de Evans foram injetados via intraperitoneal com uma solução de azul de Evans 2% diluído em salina (4 mg/kg), anestesiados e submetidos a perfusão transcardíaca com salina e paraformaldeído para posterior remoção e secção do encéfalo e análise de fluorescência. Para a imunofluorescência, os animais foram anestesiados, perfundidos e o encéfalo foi removido e seccionado. Animais do grupo VPA apresentaram maior permeabilidade da BHE ao corante azul de Evans e o RSV preveniu essa alteração. Os grupos que receberam VPA apresentaram aumento no número absoluto de células GFAP+ nas camadas profundas do pré-límbico (PrL), além de um efeito sinérgico do RSV e do VPA no CC (superficial), no PrL (superficial) e no infralímbico (IL – profundo). Na razão de células GFAP+ foi visto um aumento no número relativo nas camadas profundas e uma diminuição nas camadas superficiais do CC e do PrL no grupo VPA e o RSV foi capaz de prevenir essa alteração no PrL. Também foi observado uma diminuição no conteúdo de fluorescência de AQP4 em todas as regiões analisadas do CPFm no grupo VPA. Dessa forma, esse trabalho demonstrou alterações significativas na permeabilidade da BHE, no número e localização de células GFAP+ e no conteúdo de fluorescência de AQP4 no CPFm do modelo VPA, bem como o uso do RSV como uma ferramenta para a investigação de mecanismos envolvidos na fisiopatologia do TEA.Autism spectrum disorder (ASD) is a neurodevelopmental disorder of high prevalence, affecting 1:59 children up to 8 years of age in the United States. According to the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), the ASD diagnosis is given by the behavioral dyad composed by impairments in communication and social interaction, as well as restricted interests, repetitive and stereotyped behaviors. In addition to behavior alterations, ASD may present many associated comorbidities and an important clinical finding in this context is the presence of increased encephalic volume in the first years of life. Although the etiology of ASD remains unclear, the importance of genetic and environmental factors such as prenatal exposure to valproic acid (VPA) are known. Resveratrol (RSV) is an anti-inflammatory, antioxidant and neuroprotective molecule known to prevent social behaviors in the animal model of autism induced by prenatal exposure to VPA. The aim of this study was to evaluate the effects of VPA in blood-brain barrier permeability (BBB), GFAP+ cells and aquaporin-4 (AQP4) in the medial prefrontal cortex (mPFC) of 30-day-old male rats and the possible protective effect of RSV. Pregnant Wistar rats received from embryonic day 6.5 (E6.5) up to E18.5 daily doses of 3.6 mg/kg RSV or DMSO (vehicle) via subcutaneous and on day E12.5 an intraperitoneal dose of 600 mg/kg VPA or saline solution (0.9%). On the postnatal day 30 (P30) the animals were euthanized by anesthetic overdose of ketamine (300 mg/kg) and xylazine (40 mg/kg) and destined to different experimental techniques. The animals evaluated for BBB permeability to the Evans blue dye were injected intraperitoneally with a solution of Evans blue 2% diluted in saline (4 mg/kg), anesthetized and submitted to transcardiac perfusion with saline and paraformaldehyde for posterior removal and section of the brain and fluorescence analysis. For immunofluorescence, the animals were anesthetized, perfused and the brain was removed and sectioned. VPA group animals presented greater BBB permeability to the Evans blue dye and RSV prevented this alteration. The groups that received VPA presented an increase in the absolute number of GFAP+ cells in the deep layers of the prelimbic (PrL), in addition to a synergistic effect of RSV and VPA in CC (upper), PrL (upper) and infralimbic (IL – deeper). In the GFAP+ cell ratio was seen an increase in the relative number in deep layers and a decrease in the upper layers of CC and PrL in the VPA group and RSV was able to prevent this chance in PrL. A decrease in the AQP4 fluorescence content was also observed in all mPFC regions analyzed in the VPA group. Thus, this work demonstrated significant changes in BBB permeability, number and location of GFAP+ cells and in the fluorescent content of AQP4 in the mPFC of the VPA model, as well as the use of RSV as a tool for the investigation of mechanisms involved in ASD pathophysiology

Investigating the processes involved in caring for looked after children

Section A: A systematic literature review which identifies recent research on burnout, secondary trauma and compassion fatigue in foster carers and residential care workers caring for looked after children. The 13 studies are summarised and critically evaluated, and findings are synthesised. Focus is given to exploring the presence of burnout, secondary trauma and compassion fatigue in this population and examining factors that may influence the development of these phenomena. Results are discussed in the context of theory and existing research on social/health care professionals. Clinical and research implications relating to carer wellbeing are discussed. Section B: Uses grounded theory methodology to understand the processes involved in fostering, and the relationship between the roles of parent and professional. Ten foster carers and five social care professionals were interviewed. A preliminary model was developed, which depicts the journey of becoming a ‘professional-parent’. The relationship between the two roles was found to change and blend over time. Training and external support were highlighted as important facilitators. Results are discussed in relation to role theory and existing literature. Clinical implications relating to the support/training of foster carers are considered. Suggestions for future research with more diverse samples are discussed