Organizing timely treatment in multi-disciplinary care

Healthcare providers experience an increased pressure to organize their processes more efficiently and to provide coordinated care over multiple disciplines. Organizing multi-disciplinary care is typically highly constrained, since multiple appointments per patient have to be scheduled with possible restrictions between them. Furthermore, schedules of professionals from various facilities or with different skills must be aligned. Since it is important that patients are treated on time, access time targets are set on the time between referral to the facility and the actual start of the treatment. These targets may vary per patient type: e.g., urgent patients have shorter access time targets than regular patients. In this thesis, we use operations research methods to support multi-disciplinary care settings in providing timely treatments with an excellent quality of care, against affordable costs, while taking patient and employee satisfaction into account. We consider settings in rehabilitation care and radiotherapy, but the underlying planning problems are applicable to many other multi-disciplinary care settings, such as cancer care or specialty clinics. The developed models are applied to case studies in the Sint Maartenskliniek Nijmegen, the AMC Amsterdam and a BCCA cancer clinic in Vancouver, Canada. The results of the thesis demonstrate that adequate admission policies and capacity allocation to different activities and stages in complex treatment processes can improve compliance with access time targets for multi-disciplinary care systems considerably, while using the available resource capacities and taking patient and employee satisfaction into account

Dynamic Resource Allocation For Coordination Of Inpatient Operations In Hospitals

Healthcare systems face difficult challenges such as increasing complexity of processes, inefficient utilization of resources, high pressure to enhance the quality of care and services, and the need to balance and coordinate the staff workload. Therefore, the need for effective and efficient processes of delivering healthcare services increases. Data-driven approaches, including operations research and predictive modeling, can help overcome these challenges and improve the performance of health systems in terms of quality, cost, patient health outcomes and satisfaction. Hospitals are a key component of healthcare systems with many scarce resources such as caregivers (nurses, physicians) and expensive facilities/equipment. Most hospital systems in the developed world have employed some form of an Electronic Health Record (EHR) system in recent years to improve information flow, health outcomes, and reduce costs. While EHR systems form a critical data backbone, there is a need for platforms that can allow coordinated orchestration of the relatively complex healthcare operations. Information available in EHR systems can play a significant role in providing better operational coordination between different departments/services in the hospital through optimized task/resource allocation. In this research, we propose a dynamic real-time coordination framework for resource and task assignment to improve patient flow and resource utilization across the emergency department (ED) and inpatient unit (IU) network within hospitals. The scope of patient flow coordination includes ED, IUs, environmental services responsible for room/bed cleaning/turnaround, and patient transport services. EDs across the U.S. routinely suffer from extended patient waiting times during admission from the ED to the hospital\u27s inpatient units, also known as ED patient `boarding\u27. This ED patient boarding not only compromises patient health outcomes but also blocks access to ED care for new patients from increased bed occupancy. There are also significant cost implications as well as increased stress and hazards to staff. We carry out this research with the goal of enabling two different modes of coordination implementation across the ED-to-IU network to reduce ED patient boarding: Reactive and Proactive. The proposed `reactive\u27 coordination approach is relatively easy to implement in the presence of modern EHR and hospital IT management systems for it relies only on real-time information readily available in most hospitals. This approach focuses on managing the flow of patients at the end of their ED care and being admitted to specific inpatient units. We developed a deterministic dynamic real-time coordination model for resource and task assignment across the ED-to-IU network using mixed-integer programming. The proposed \u27proactive\u27 coordination approach relies on the power of predictive analytics that anticipate ED patient admissions into the hospital as they are still undergoing ED care. The proactive approach potentially allows additional lead-time for coordinating downstream resources, however, it requires the ability to accurately predict ED patient admissions, target IU for admission, as well as the remaining length-of-stay (care) within the ED. Numerous other studies have demonstrated that modern EHR systems combined with advances in data mining and machine learning methods can indeed facilitate such predictions, with reasonable accuracy. The proposed proactive coordination optimization model extends the reactive deterministic MIP model to account for uncertainties associated with ED patient admission predictions, leading to an effective and efficient proactive stochastic MIP model. Both the reactive and proactive coordination methods have been developed to account for numerous real-world operational requirements (e.g., rolling planning horizon, event-based optimization and task assignments, schedule stability management, patient overflow management, gender matching requirements for IU rooms with double occupancy, patient isolation requirements, equity in staff utilization and equity in reducing ED patient waiting times) and computational efficiency (e.g., through model decomposition and efficient construction of scenarios for proactive coordination). We demonstrate the effectiveness of the proposed models using data from a leading healthcare facility in SE-Michigan, U.S. Results suggest that even the highly practical optimization enabled reactive coordination can lead to dramatic reduction in ED patient boarding times. Results also suggest that signification additional reductions in patient boarding are possible through the proposed proactive approach in the presence of reliable analytics models for prediction ED patient admissions and remaining ED length-of-stay. Future research can focus on further extending the scope of coordination to include admissions management (including any necessary approvals from insurance), coordination needs for admissions that stem from outside the ED (e.g., elective surgeries), as well as ambulance diversions to manage patient flows across the region and hospital networks

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

The State-of-the-Art Survey on Optimization Methods for Cyber-physical Networks

Cyber-Physical Systems (CPS) are increasingly complex and frequently integrated into modern societies via critical infrastructure systems, products, and services. Consequently, there is a need for reliable functionality of these complex systems under various scenarios, from physical failures due to aging, through to cyber attacks. Indeed, the development of effective strategies to restore disrupted infrastructure systems continues to be a major challenge. Hitherto, there have been an increasing number of papers evaluating cyber-physical infrastructures, yet a comprehensive review focusing on mathematical modeling and different optimization methods is still lacking. Thus, this review paper appraises the literature on optimization techniques for CPS facing disruption, to synthesize key findings on the current methods in this domain. A total of 108 relevant research papers are reviewed following an extensive assessment of all major scientific databases. The main mathematical modeling practices and optimization methods are identified for both deterministic and stochastic formulations, categorizing them based on the solution approach (exact, heuristic, meta-heuristic), objective function, and network size. We also perform keyword clustering and bibliographic coupling analyses to summarize the current research trends. Future research needs in terms of the scalability of optimization algorithms are discussed. Overall, there is a need to shift towards more scalable optimization solution algorithms, empowered by data-driven methods and machine learning, to provide reliable decision-support systems for decision-makers and practitioners

Introducing reinforcement learning in the Wi-Fi MAC layer to support sustainable communications in e-Health scenarios

The crisis of energy supplies has led to the need for sustainability in technology, especially in the Internet of Things (IoT) paradigm. One solution is the integration of Energy Harvesting (EH) technologies into IoT systems, which reduces the amount of battery replacement. However, integrating EH technologies within IoT systems is challenging, and it requires adaptations at different layers of the IoT protocol stack, especially at Medium Access Control (MAC) layer due to its energy-hungry features. Since Wi-Fi is a widely used wireless technology in IoT systems, in this paper, we perform an extensive set of simulations in a dense solar-based energy-harvesting Wi-Fi network in an e-Health environment. We introduce optimization algorithms, which benefit from the Reinforcement Learning (RL) methods to efficiently adjust to the complexity and dynamic behaviour of the network. We assume the concept of Access Point (AP) coordination to demonstrate the feasibility of the upcoming Wi-Fi amendment IEEE 802.11bn (Wi-Fi 8). This paper shows that the proposed algorithms reduce the network&amp;#x2019;s energy consumption by up to 25% compared to legacy Wi-Fi while maintaining the required Quality of Service (QoS) for e-Health applications. Moreover, by considering the specific adjustment of MAC layer parameters, up to 37% of the energy of the network can be conserved, which illustrates the viability of reducing the dimensions of solar cells, while concurrently augmenting the flexibility of this EH technique for deployment within the IoT devices. We anticipate this research will shed light on new possibilities for IoT energy harvesting integration, particularly in contexts with restricted QoS environments such as e-Healthcare.</p

Resource Allocation in SDN/NFV-Enabled Core Networks

For next generation core networks, it is anticipated to integrate communication, storage and computing resources into one unified, programmable and flexible infrastructure. Software-defined networking (SDN) and network function virtualization (NFV) become two enablers. SDN decouples the network control and forwarding functions, which facilitates network management and enables network programmability. NFV allows the network functions to be virtualized and placed on high capacity servers located anywhere in the network, not only on dedicated devices in current networks. Driven by SDN and NFV platforms, the future network architecture is expected to feature centralized network management, virtualized function chaining, reduced capital and operational costs, and enhanced service quality. The combination of SDN and NFV provides a potential technical route to promote the future communication networks. It is imperative to efficiently manage, allocate and optimize the heterogeneous resources, including computing, storage, and communication resources, to the customized services to achieve better quality-of-service (QoS) provisioning. This thesis makes some in-depth researches on efficient resource allocation for SDN/NFV-enabled core networks in multiple aspects and dimensionality. Typically, the resource allocation task is implemented in three aspects. Given the traffic metrics, QoS requirements, and resource constraints of the substrate network, we first need to compose a virtual network function (VNF) chain to form a virtual network (VN) topology. Then, virtual resources allocated to each VNF or virtual link need to be optimized in order to minimize the provisioning cost while satisfying the QoS requirements. Next, we need to embed the virtual network (i.e., VNF chain) onto the substrate network, in which we need to assign the physical resources in an economical way to meet the resource demands of VNFs and links. This involves determining the locations of NFV nodes to host the VNFs and the routing from source to destination. Finally, we need to schedule the VNFs for multiple services to minimize the service completion time and maximize the network performance. In this thesis, we study resource allocation in SDN/NFV-enabled core networks from the aforementioned three aspects. First, we jointly study how to design the topology of a VN and embed the resultant VN onto a substrate network with the objective of minimizing the embedding cost while satisfying the QoS requirements. In VN topology design, optimizing the resource requirement for each virtual node and link is necessary. Without topology optimization, the resources assigned to the virtual network may be insufficient or redundant, leading to degraded service quality or increased embedding cost. The joint problem is formulated as a Mixed Integer Nonlinear Programming (MINLP), where queueing theory is utilized as the methodology to analyze the network delay and help to define the optimal set of physical resource requirements at network elements. Two algorithms are proposed to obtain the optimal/near-optimal solutions of the MINLP model. Second, we address the multi-SFC embedding problem by a game theoretical approach, considering the heterogeneity of NFV nodes, the effect of processing-resource sharing among various VNFs, and the capacity constraints of NFV nodes. In the proposed resource constrained multi-SFC embedding game (RC-MSEG), each SFC is treated as a player whose objective is to minimize the overall latency experienced by the supported service flow, while satisfying the capacity constraints of all its NFV nodes. Due to processing-resource sharing, additional delay is incurred and integrated into the overall latency for each SFC. The capacity constraints of NFV nodes are considered by adding a penalty term into the cost function of each player, and are guaranteed by a prioritized admission control mechanism. We first prove that the proposed game RC-MSEG is an exact potential game admitting at least one pure Nash Equilibrium (NE) and has the finite improvement property (FIP). Then, we design two iterative algorithms, namely, the best response (BR) algorithm with fast convergence and the spatial adaptive play (SAP) algorithm with great potential to obtain the best NE of the proposed game. Third, the VNF scheduling problem is investigated to minimize the makespan (i.e., overall completion time) of all services, while satisfying their different end-to-end (E2E) delay requirements. The problem is formulated as a mixed integer linear program (MILP) which is NP-hard with exponentially increasing computational complexity as the network size expands. To solve the MILP with high efficiency and accuracy, the original problem is reformulated as a Markov decision process (MDP) problem with variable action set. Then, a reinforcement learning (RL) algorithm is developed to learn the best scheduling policy by continuously interacting with the network environment. The proposed learning algorithm determines the variable action set at each decision-making state and accommodates different execution time of the actions. The reward function in the proposed algorithm is carefully designed to realize delay-aware VNF scheduling. To sum up, it is of great importance to integrate SDN and NFV in the same network to accelerate the evolution toward software-enabled network services. We have studied VN topology design, multi-VNF chain embedding, and delay-aware VNF scheduling to achieve efficient resource allocation in different dimensions. The proposed approaches pave the way for exploiting network slicing to improve resource utilization and facilitate QoS-guaranteed service provisioning in SDN/NFV-enabled networks