Managing coupled human and natural systems (CHANS) : the case of water

Many sustainability challenges of the 21st century are the result of poor management of coupled human and natural systems (CHANS). Limited understanding of the mechanisms that give rise to complex dynamics in CHANS has contributed to overexploitation and degradation of water and other natural resources around the globe, leading to unintended consequences of well-intentioned policies. This raises the question of whether the tools and methods currently used in environmental management and policy design meet the requirements of complex dynamic systems. In this thesis, qualitative and quantitative research approaches from the fields of systems thinking and simulation modelling were combined with the aim of improving understanding of the dynamics of CHANS, and human-water systems in particular, and developing better methods and tools to support more effective policy and management strategies in the future. The work included a systematic review, qualitative and quantitative system dynamics modelling case studies, method development, and agent-based modelling and simulation. The results showed that changes in CHANS are driven by observable and unobservable exchanges of energy, matter and information across space and time that give rise to constantly changing, nonlinear dynamics. Many contemporary tools and methods used in management and policy design are not suited to this dynamic complexity and, instead of embracing complexity, seek to reduce it by excluding structural drivers of endogenous behaviour. This can contribute to unsustainable water use and amplify impacts of climate change in coupled human and water systems. This thesis showed that system dynamics-based approaches can effectively complement conventional static management tools, to better account for dynamic complexity. By tapping into the collective intelligence of actors engaged in the system, the approaches can support more realistic models and more effective and sustainable management, leading to establishment of middle-range theories for management of CHANS

Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning

Reinforcement learning has been explored for many problems, from video games with deterministic environments to portfolio and operations management in which scenarios are stochastic; however, there have been few attempts to test these methods in banking problems. In this study, we sought to find and automatize an optimal credit card limit adjustment policy by employing reinforcement learning techniques. In particular, because of the historical data available, we considered two possible actions per customer, namely increasing or maintaining an individual's current credit limit. To find this policy, we first formulated this decision-making question as an optimization problem in which the expected profit was maximized; therefore, we balanced two adversarial goals: maximizing the portfolio's revenue and minimizing the portfolio's provisions. Second, given the particularities of our problem, we used an offline learning strategy to simulate the impact of the action based on historical data from a super-app (i.e., a mobile application that offers various services from goods deliveries to financial products) in Latin America to train our reinforcement learning agent. Our results show that a Double Q-learning agent with optimized hyperparameters can outperform other strategies and generate a non-trivial optimal policy reflecting the complex nature of this decision. Our research not only establishes a conceptual structure for applying reinforcement learning framework to credit limit adjustment, presenting an objective technique to make these decisions primarily based on data-driven methods rather than relying only on expert-driven systems but also provides insights into the effect of alternative data usage for determining these modifications.Comment: 29 pages, 16 figure

A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.Comment: accepted by 37th IEEE International Conference on Distributed Computing (ICDCS 2017

Active management of multi-service networks.

Future multiservice networks will be extremely large and complex. Novel management solutions will be required to keep the management costs reasonable. Active networking enables management to be delegated to network users as a large set of independent small scale management systems. A novel architecture for an active network based management solution for multiservice networking is presented

A MAS model for optimizing the spatial aspects of livestock production and manure abatement

As a consequence of the EU Nitrates Directive many countries have developed policies to regulate manure production and manure emission on land. Farmers have three allocation options: spreading manure on own land, transporting manure to other farmers’ land and processing manure. To better understand the manure problem as an allocation problem a spatial mathematical programming multi-agent model has been developed. The model is applied for Flanders (Belgium), a highly concentrated livestock area. Using this model, policy alternatives and their cost efficiency can be evaluated. These simulations result in advice on location and type of manure processing and an indicator which creates transparency in the manure and processing market