Application of reinforcement learning with Q-learning for the routing in industrial wireless sensors networks

Industrial Wireless Sensor Networks (IWSN) usually have a centralized management approach, where a device known as Network Manager is responsible for the overall configuration, definition of routes, and allocation of communication resources. The routing algorithms need to ensure path redundancy while reducing latency, power consumption, and resource usage. Graph routing algorithms are used to address these requirements. The dynamicity of wireless networks has been a challenge for tuning and developing routing algorithms, and Machine Learning models such as Reinforcement Learning have been applied in a promising way in Wireless Sensor Networks to select, adapt and optimize routes. The basic concept of Reinforcement Learning is the existence of a learning agent that acts and changes the state of the environment, and receives rewards. However, the existing approaches do not meet some of the requirements of the IWSN standards. In this context, this thesis proposes the Q-Learning Reliable Routing approach, where the Q-Learning model is used to build graph routes. Two approaches are presented: QLRR-WA and QLRR-MA. QLRR-WA uses a learning agent that adjusts the weights of the cost equation of a state-of-the-art routing algorithm to reduce the latency and increase the network lifetime. QLRR-MA uses several learning agents so nodes can choose connections in the graph trying to reduce the latency. Other contributions of this thesis are the performance comparison of the state-of-the-art graph-routing algorithms and the evaluation methodology proposed. The QLRR algorithms were evaluated in a WirelessHART simulator, considering industrial monitoring applications with random topologies. The performance was analyzed considering the average network latency, network lifetime, packet delivery ratio and the reliability of the graphs. The results showed that, when compared to the state of the art, QLRR-WA reduced the average network latency and improved the lifetime while keeping high reliability, while QLRR-MA reduced latency and increased packet delivery ratio with a reduction in the network lifetime. These results indicate that Reinforcement Learning may be helpful to optimize and improve network performance.As Redes Industriais de Sensores Sem Fio (IWSN) geralmente têm uma abordagem de gerenciamento centralizado, onde um dispositivo conhecido como Gerenciador de Rede é responsável pela configuração geral, definição de rotas e alocação de recursos de comunicação. Os algoritmos de roteamento precisam garantir a redundância de caminhos para as mensagens, e também reduzir a latência, o consumo de energia e o uso de recursos. O roteamento por grafos é usado para alcançar estes requisitos. A dinamicidade das redes sem fio tem sido um desafio para o ajuste e o desenvolvimento de algoritmos de roteamento, e modelos de Aprendizado de Máquina como o Aprendizado por Reforço têm sido aplicados de maneira promissora nas Redes de Sensores Sem Fio para selecionar, adaptar e otimizar rotas. O conceito básico do Aprendizado por Reforço envolve a existência de um agente de aprendizado que atua em um ambiente, altera o estado do ambiente e recebe recompensas. No entanto, as abordagens existentes não atendem a alguns dos requisitos dos padrões das IWSN. Nesse contexto, esta tese propõe a abordagem Q-Learning Reliable Routing, onde o modelo Q-Learning é usado para construir os grafos de roteamento. Duas abordagens são propostas: QLRR-WA e QLRR-MA. A abordagem QLRR-WA utiliza um agente de aprendizado que ajusta os pesos da equação de custo de um algoritmo de roteamento de estado da arte, com o objetivo de reduzir a latência e aumentar a vida útil da rede. A abordagem QLRR-MA utiliza diversos agente de aprendizado de forma que cada dispositivo na rede pode escolher suas conexões tentando reduzir a latência. Outras contribuições desta tese são a comparação de desempenho das abordagens com os algoritmos de roteamento de estado da arte e a metodologia de avaliação proposta. As abordagens do QLRR foram avaliadas com um simulador WirelessHART, considerando aplicações de monitoramento industrial com diversas topologias. O desempenho foi analisado considerando a latência média da rede, o tempo de vida esperado da rede, a taxa de entrega de pacotes e a confiabilidade dos grafos. Os resultados mostraram que, quando comparado com o estado da arte, o QLRR-WA reduziu a latência média da rede e melhorou o tempo de vida esperado, mantendo alta confiabilidade, enquanto o QLRR-MA reduziu a latência e aumentou a taxa de entrega de pacotes, ao custo de uma redução no tempo de vida esperado da rede. Esses resultados indicam que o Aprendizado por Reforço pode ser útil para otimizar e melhorar o desempenho destas redes

The Seaman\u27s Personal Injury Action and the Jury Trial

An injured seaman has the availability of a judicially created maritime cause of action for maintenance and cure, or for unseaworthiness, or a cause of action under the Merchant Marine Act (a.k.a. the Jones Act). This article looks at jurisdictional and jury trial issues involving suits by injured seaman. The injured seaman\u27s right to a jury trial differs depending on whether admiralty jurisdiction or jurisdiction in an action at law is invoked in cases where a maritime cause of action for maintenance and cure, or for unseaworthiness, is joined with a cause of action under the Jones Act. In an action at law, the U.S. Supreme Court has held that these types of cases must be submitted to a jury when the causes of action arise out of the same set of facts. On the law side, the courts have pendent jurisdiction over the maritime claims. However, in an action under admiralty, the U.S. Supreme Court has held that an injured seaman has no right to a jury trial in these types of cases. On the admiralty side, unlike maritime claims that can be in rem or in personam, Jones Act claims cannot be in personam. Although case law recounted in the article has clarified certain aspects of injured seaman\u27s rights, the author concludes that many questions remain unanswered

Progress and implementation of team-based care in the United States

Poster presentation from Health Services Research: Evidence-based practice London, UK. 1-3 July 2014

A reforma agraria como politica de geraçao de emprego e renda

Orientador: Nilson Maciel de PaulaMonografia(Graduaçao) - Universidade Federal do Paraná,Setor de Ciencias Sociais Aplicadas, Curso de Ciencias EconômicasResumo: A origem do problema de concentração de terra e renda agrária no Brasil remete à colonização, mas a discussão da necessidade da implantação da reforma agrária no país só ganhou mais destaque a partir da década de 50. Atualmente, o fortalecimento dos complexos industriais convive com o empobrecimento de pequenos produtores, com a fuga do homem do campo que promove "inchaço" nos centros urbanos e com o crescimento da massa de pobres e excluídos que se organizam em movimentos e exigem o acesso a terra e crédito, por meio de uma política de reforma agrária. O objetivo deste trabalho é analisar a discussão que envolve o processo de reforma agrária no Brasil, a contribuição da reforma para o desenvolvimento econômico e seus resultados em termos de geração de emprego, renda e qualidade de vida. Para isto, são expostas diferentes concepções do processo de reforma agrária e como ele se deu em outros países, principalmente na América Latina; e a experiência brasileira da discussão e implantação da reforma agrária. A implantação de uma ampla política de reforma agrária é considerada como um meio de promoção de distribuição de renda, melhoria da qualidade de vida, fortalecimento político dos assentados e fixação do homem na terra. Fatores que, associados, contribuiriam para o desenvolvimento nacional. No entanto, a realidade apresentada por levantamentos feitos em assentamentos é de poucos ganhos em termos de renda e qualidade de vida. Os beneficiários da reforma, porém, mostram-se satisfeitos por terem seu pedaço de terra e garantir alimentos pelo menos para consumo próprio. Isso demonstra que a reforma agrária implantada no Brasil tem ação paliativa, de aquietação política, que traz algum ganho social, mas pouco resultado econômico. Não foi ampla, nem contínua, e só é positiva se comparada a uma realidade de extrema miséria, que é o quadro em que se encontram a maioria dos sem-terra que a reivindicam

Submergence of magnetic flux in interaction of sunspot groups

Submergence of magnetic flux is demonstrated in the process of evolution of sunspot groups NOAA 6850 (26 Sep.-07 Oct. 1991) and 7220/22 (06-17 July 1992). In both cases new magnetic flux emerges immediately behind an existing spotgroup. The new flux does not interact with the old one, as can be seen in YOHKOH X-ray images, so no significant flare activity occurs, although umbrae of different magnetic polarity collide. In both cases the quickly forward moving p-spots of the new flux force the submergence of the f-spots of the older region, these being squeezed from two sides between the old and new p-spots. This leads to the disappearance of about 16 1021 Mx of f-polarity in AR 6850. In the case of this region the submerged flux interacts with newly emerging magnetic fields and in the next rotation continues its life as the complex and eruptive AR 6891. In AR 7220/22, spots of both polarities submerge in the intermediate part of the complex between the old leader of AR 7220 and the quickly growing AR 7222, leaving in the next rotation a bipolar-looking group (AR 7248), consisting of two p-polarity spots. So, submergence of magnetic flux due to interaction of old and new activity may play an important role in the decay of sunspot groups

Moisture control design has to respond to all relevant hygrothermal loads

Moisture related damage is still a formidable cost factor in the building sector. Besides installation deficiencies, moisture control design failures are the most frequent reasons for moisture problems. Therefore, adequate moisture control analysis has become the key for sustainable buildings. However, by focusing on vapour diffusion only other important moisture loads such as driving rain, construction moisture or air infiltration are mostly neglected. Therefore, international moisture control standards often refer to simulation models for more realistic analysis, leaving many practitioners wondering how to handle these tools. To overcome this dilemma, the updated German moisture control standard has introduced a three-pathway approach for design evaluation: 1st deemed to satisfy list, 2nd restricted Glaser calculation and 3rd fully fledged hygrothermal simulation. The third pathway includes the option to account for small leaks or imperfections in building envelope components. Guidelines in other countries are also embracing similar moisture control approaches which gives hope for more durable and sustainable building design. To reach this aim, moisture control should also become an integral part of the design process instead of a secondary chore

Aportes del aprendizaje basado en problemas (ABP) y la neurobiología del aprendizaje a la enseñanza de la medicina hoy

El presente trabajo busca explicar cómo los principios derivados de lo que se ha aprendido sobre la neurobiología del aprendizaje y la memoria pueden ser aplicados para mejorar las estrategias de enseñanza de la medicina como el aprendizaje basado en problemas (ABP). Los autores presentan las bases neurobiológicas del ABP como modelo pedagógico y didáctico, y como utilizarlas mejor para alcanzar un aprendizaje significativo, con el fin de fortalecer y facilitar su aplicación en las instituciones universitarias

Laboratory measurement and boundary conditions for the water vapour resistivity properties of typical Australian impermeable and smart pliable membranes

The duo of better insulated and more air-tight envelopes without appropriate considerationof water vapour diffusion and envelope moisture management has often demonstrated an increased potential of moisture accumulation, interstitial condensation, and mould growth within the building envelope. To inform a resilient, energy efficient, and healthy building design, long-term transient hygrothermal modelling are required. Since 2008, concern has been raised to the Australian building regulators regarding the need to establish the vapour diffusion properties of construction materials, in order to develop a hygrothermal regulatory framework. This paper discusses the results from laboratory testing of the vapour diffusion properties of two common reflective pliable membranes, and one smart pliable membrane. The two reflective pliable membranes are often used within the exterior walls of Australian buildings. The smart pliable membrane is a relatively new, internationally available product. The three membranes were tested as per ISO 12,572 at 23 ◦C and 50% RH. To establish if the vapour resistivity properties were constant, under different relative humidity conditions, the membranes were further tested at 23 ◦C and relative humidity values of 35%, 65%, and 80%. The results of the three pliable membranes show that the vapour resistivity properties varied in a non-linear (dynamic) manner subject to relative humidity. In conclusion, this research demonstrates that regardless of the class, each of the tested membrane types behaved differentlyunder varying relative humidity and pressure gradients within the testing laboratory