An Energy Sharing Game with Generalized Demand Bidding: Model and Properties

This paper proposes a novel energy sharing mechanism for prosumers who can produce and consume. Different from most existing works, the role of individual prosumer as a seller or buyer in our model is endogenously determined. Several desirable properties of the proposed mechanism are proved based on a generalized game-theoretic model. We show that the Nash equilibrium exists and is the unique solution of an equivalent convex optimization problem. The sharing price at the Nash equilibrium equals to the average marginal disutility of all prosumers. We also prove that every prosumer has the incentive to participate in the sharing market, and prosumers' total cost decreases with increasing absolute value of price sensitivity. Furthermore, the Nash equilibrium approaches the social optimal as the number of prosumers grows, and competition can improve social welfare.Comment: 16 pages, 7 figure

On Reward Structures of Markov Decision Processes

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various kinds of "costs" associated with reinforcement learning inspired by the demands in robotic applications, rewards are central to understanding the structure of a Markov decision process and reward-centric notions can elucidate important concepts in reinforcement learning. Specifically, we study the sample complexity of policy evaluation and develop a novel estimator with an instance-specific error bound of $\tilde{O}(\sqrt{\frac{\tau_s}{n}})$ for estimating a single state value. Under the online regret minimization setting, we refine the transition-based MDP constant, diameter, into a reward-based constant, maximum expected hitting cost, and with it, provide a theoretical explanation for how a well-known technique, potential-based reward shaping, could accelerate learning with expert knowledge. In an attempt to study safe reinforcement learning, we model hazardous environments with irrecoverability and proposed a quantitative notion of safe learning via reset efficiency. In this setting, we modify a classic algorithm to account for resets achieving promising preliminary numerical results. Lastly, for MDPs with multiple reward functions, we develop a planning algorithm that computationally efficiently finds Pareto-optimal stochastic policies.Comment: This PhD thesis draws heavily from arXiv:1907.02114 and arXiv:2002.06299; minor edit

Solvent dependence of the rheological properties in hydrogel magnetorheological plastomer

Chemically crosslinked hydrogel magnetorheological (MR) plastomer (MRP) embedded with carbonyl iron particles (CIPs) exhibits excellent magnetic performance (MR effect) in the presence of external stimuli especially magnetic field. However, oxidation and desiccation in hydrogel MRP due to a large amount of water content as a dispersing phase would limit its usage for long‐term applications, especially in industrial engineering. In this study, different solvents such as dimethyl sulfoxide (DMSO) are also used to prepare polyvinyl alcohol (PVA) hydrogel MRP. Thus, to understand the dynamic viscoelastic properties of hydrogel MRP, three different samples with different solvents: water, DMSO, and their binary mixtures (DMSO/water) were prepared and systematically carried out using the oscillatory shear. The outcomes demonstrate that the PVA hydrogel MRP prepared from precursor gel with water shows the highest MR effect of 15,544% among the PVA hydrogel MRPs. However, the samples exhibit less stability and tend to oxidise after a month. Meanwhile, the samples with binary mixtures (DMSO/water) show an acceptable MR effect of 11,024% with good stability and no CIPs oxidation. Otherwise, the sample with DMSO has the lowest MR effect of 7049% and less stable compared to the binary solvent samples. This confirms that the utilisation of DMSO as a new solvent affects the rheological properties and stability of the samples

Capacity Planning for Heterogeneous Patient Populations in Primary Care and Specialty Networks

Access to primary care has a direct impact on morbidity and mortality, and is strongly influenced by indirect waiting time: the delay between the requested and allotted appointment day. Our models describe the heterogeneous appointment seeking patterns of a primary care patient panel using stochastic processes parameterized to reflect the diversity of primary care visit rates in the US. For capacity planning, we estimate the distribution of daily appointments, and show that the distribution variability can be reduced by heuristics that use patient flexibility regarding the day of the appointment. For delays, we demonstrate that in a first-come, first-served system, patients who need the most frequent appointments suffer the greatest delays, motivating the need to reserve slots for high-visit patient classes. To further understand the inequity in delay, we model the primary care appointment system as a Discrete-Time Markov Chain. We derive an analytical expression for delay in terms of the patient’s probability of daily visit. We show that conditions for monotone mapping of the probability of visit to delay are intractable and give numerical results that support monotonicity. In our last chapter, we expand our scope to include specialty care networks. Using patient-level longitudinal data from the Medical Expenditure Panel Survey, we model the sequence of appointments with multiple specialty types and the time intervals between such appointments as a Markov Renewal Process (MRP). We use comorbidity count to model patient heterogeneity and extract the MRP parameters for each patient subgroup. Next, we adapt the steady state results to provide an analytical expression of the expected appointment fill-rate by specialty and patient subgroups. Our analytical results demonstrate that patients with higher comorbidity count typically have a lower fill-rate because of shorter lead time between appointments thereby necessitating either overtime or reserved slots to ensure timely access. We further simulate appointment seeking patterns of a nationally representative panel of patients in the specialty network and estimate the distribution of daily appointment requests for each specialty. Similar to the primary care case, we show that heuristics that leverage patient flexibility regarding the day of the appointment can reduce variability in appointment requests for each specialty

Supply Chain

Traditionally supply chain management has meant factories, assembly lines, warehouses, transportation vehicles, and time sheets. Modern supply chain management is a highly complex, multidimensional problem set with virtually endless number of variables for optimization. An Internet enabled supply chain may have just-in-time delivery, precise inventory visibility, and up-to-the-minute distribution-tracking capabilities. Technology advances have enabled supply chains to become strategic weapons that can help avoid disasters, lower costs, and make money. From internal enterprise processes to external business transactions with suppliers, transporters, channels and end-users marks the wide range of challenges researchers have to handle. The aim of this book is at revealing and illustrating this diversity in terms of scientific and theoretical fundamentals, prevailing concepts as well as current practical applications

The impact of MRP on a traditional manufacturing company

Thesis (M.S.)--Massachusetts Institute of Technology, Sloan School of Management, 1990.Includes bibliographical references (leaves 66-67).by Maria Carolina Briza Junqueira.M.S

A Digital Twin Framework for Production Planning Optimization: Applications for Make-To-Order Manufacturers

In this dissertation, we develop a Digital Twin framework for manufacturing systems and apply it to various production planning and scheduling problems faced by Make-To-Order (MTO) firms. While this framework can be used to digitally represent a particular manufacturing environment with high fidelity, our focus is in using it to generate realistic settings to test production planning and scheduling algorithms in practice. These algorithms have traditionally been tested by either translating a practical situation into the necessary modeling constructs, without discussion of the assumptions and inaccuracies underlying this translation, or by generating random instances of the modeling constructs, without assessing the limitations in accurately representing production environments. The consequence has been a serious gap between theory advancement and industry practice. The major goal of this dissertation is to develop a framework that allows for practical testing, evaluation, and implementation of new approaches for seamless industry adoption. We develop this framework as a modular software package and emphasize the practicality and configurability of the framework, such that minimal modelling effort is required to apply the framework to a multitude of optimization problems and manufacturing systems. Throughout this dissertation, we emphasize the importance of the underlying scheduling problems which provide the basis for additional operational decision making. We focus on the computational evaluation and comparisons of various modeling choices within the developed frameworks, with the objective of identifying models which are both effective and computationally efficient. In Part 1 of this dissertation, we consider a class of Production Planning and Execution problems faced by job shop manufacturing systems. In Part 2 of this dissertation, we consider a class of scheduling problems faced by manufacturers whose production system is dominated by a single operation

Business Process Automation and Managerial Accounting: An SAP Plug and Play Module (FINAL REPORT)

The primary aim of our project is to develop an Enterprise Resource Planning (ERP) platform that enables students at Pace to understand how different interdisciplinary areas in cross-unit and/or cross-enterprise decision making are related. ERP can help us do this since it allows a firm to automate and integrate its business processes, share common data and practices across the entire enterprise, and provide and access information in a real-time environment

When is Agnostic Reinforcement Learning Statistically Tractable?

We study the problem of agnostic PAC reinforcement learning (RL): given a policy class $\Pi$, how many rounds of interaction with an unknown MDP (with a potentially large state and action space) are required to learn an $\epsilon$-suboptimal policy with respect to $\Pi$? Towards that end, we introduce a new complexity measure, called the \emph{spanning capacity}, that depends solely on the set $\Pi$ and is independent of the MDP dynamics. With a generative model, we show that for any policy class $\Pi$, bounded spanning capacity characterizes PAC learnability. However, for online RL, the situation is more subtle. We show there exists a policy class $\Pi$ with a bounded spanning capacity that requires a superpolynomial number of samples to learn. This reveals a surprising separation for agnostic learnability between generative access and online access models (as well as between deterministic/stochastic MDPs under online access). On the positive side, we identify an additional \emph{sunflower} structure, which in conjunction with bounded spanning capacity enables statistically efficient online RL via a new algorithm called POPLER, which takes inspiration from classical importance sampling methods as well as techniques for reachable-state identification and policy evaluation in reward-free exploration.Comment: Accepted to NeurIPS 202