Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks

This paper discusses a system that accelerates reinforcement learning by using transfer from related tasks. Without such transfer, even if two tasks are very similar at some abstract level, an extensive re-learning effort is required. The system achieves much of its power by transferring parts of previously learned solutions rather than a single complete solution. The system exploits strong features in the multi-dimensional function produced by reinforcement learning in solving a particular task. These features are stable and easy to recognize early in the learning process. They generate a partitioning of the state space and thus the function. The partition is represented as a graph. This is used to index and compose functions stored in a case base to form a close approximation to the solution of the new task. Experiments demonstrate that function composition often produces more than an order of magnitude increase in learning rate compared to a basic reinforcement learning algorithm

Per-host DDoS mitigation by direct-control reinforcement learning

DDoS attacks plague the availability of online services today, yet like many cybersecurity problems are evolving and non-stationary. Normal and attack patterns shift as new protocols and applications are introduced, further compounded by burstiness and seasonal variation. Accordingly, it is difficult to apply machine learning-based techniques and defences in practice. Reinforcement learning (RL) may overcome this detection problem for DDoS attacks by managing and monitoring consequences; an agent’s role is to learn to optimise performance criteria (which are always available) in an online manner. We advance the state-of-the-art in RL-based DDoS mitigation by introducing two agent classes designed to act on a per-flow basis, in a protocol-agnostic manner for any network topology. This is supported by an in-depth investigation of feature suitability and empirical evaluation. Our results show the existence of flow features with high predictive power for different traffic classes, when used as a basis for feedback-loop-like control. We show that the new RL agent models can offer a significant increase in goodput of legitimate TCP traffic for many choices of host density

A novel approach of creating sustainable urban planning solutions that optimise the local air quality and environmental equity in Helsinki, Finland : The CouSCOUS study protocol

Background Air pollution is one of the major environmental challenges cities worldwide face today. Planning healthy environments for all future populations, whilst considering the ongoing demand for urbanisation and provisions needed to combat climate change, remains a difficult task. Objective To combine artificial intelligence (AI), atmospheric and social sciences to provide urban planning solutions that optimise local air quality by applying novel methods and taking into consideration population structures and traffic flows. Methods We will use high-resolution spatial data and linked electronic population cohort for Helsinki Metropolitan Area (Finland) to model (a) population dynamics and urban inequality related to air pollution; (b) detailed aerosol dynamics, aerosol and gas-phase chemistry together with detailed flow characteristics; (c) high-resolution traffic flow addressing dynamical changes at the city environment, such as accidents, construction work and unexpected congestion. Finally, we will fuse the information resulting from these models into an optimal city planning model balancing air quality, comfort, accessibility and travelling efficiency.Peer reviewe

The 1935 Hsinchu-Taichung Earthquake

The history of natural disasters in Taiwan has frequently been linked to the practice of historical preservation, archival science, oral history, and museum curatorship. All are collectively hallmarks of a broad range of activities that fall under the umbrella of public history. The problem for Taiwan, however, concerns the legitimacy. Taiwan does not have a single national narrative. It has been subjected to waves of colonialism since the seventeenth century and does not presently have a fully post-colonial narrative. The earthquakes discussed in this paper occurred in two different periods of colonisation.  In order to situate the history of earthquakes into a public history discourse, the field of earthquake-based research in Taiwan has to incorporate different audiences and integrate into a much broader understanding. By this, I mean that the present regimental academic disciplines in Taiwan need to be cross disciplinary, especially since public history is by its very nature collaborative. It illuminates a shared authority over a much wider area. It needs to. It is my argument that it is in digital humanities that Taiwanese academics work best in collaboration. Efforts have been made to digitise the personal experiences of those involved in typhoon reconstruction efforts. A natural synergy, therefore, for the understanding of earthquakes, as public history, is to emphasise access and broad participation in the creation of knowledge. Digital humanities enables this. Attention to this is particularly important in historical preservation of particular sites on an island that frequently develops and re-develops brownfield sites

Natural hazards, disasters and local development

International audienceThis publication is aimed at individuals and institutions aware of the need to integrate reconstruction programs as part of a strategy to achieve the sustainable improvement of the living conditions of populations. The objective of this document is to show the interest, relevance and potential of local materials in at-risk situations but also and particularly to demonstrate that ignoring or banning the use of local materials and their associated know-how, as part of reconstruction, risk prevention and management or development projects, inevitably leads to even more vulnerable and dependent affected (or likely to be affected) populations. Beyond housing reconstruction, the idea is to implement sustainable conditions for better prevention and risk preparedness, for and by local populations, by taking into account and valorizing their knowledge and know-how: their own building culture. Through the organization of seminars and international meetings, and with the proliferation of operational partnerships in the field, our team was able to begin structuring the knowledge in the field. This document is therefore intended to present the conclusions of a shared reflection process. The theoretical and methodological elements proposed are illustrated by case studies of several recent initiatives, reflecting the different approaches developed and their results. This document does not provide instructions to be followed, but is intended as a basis for reflection. It opens new doors into more effective programs of reconstruction and prevention in high-risk area

Nursing home admission: Effects, predictors of well-being and implications for counseling

Admission to a nursing home is a stressful life event demanding changes in all aspects of living. Becoming a nursing home resident was perceived by subjects in Gordon\u27s study (1985) to be the third most stressful life event preceded only by death of a spouse and divorce and followed by pregnancy and retirement. Researchers have disputed the direct relationship of relocation trauma to mortality (Borup 1983), but agree that certain factors increase the older person\u27s vulnerability to the stresses of relocation