Online Resource Allocation in Episodic Markov Decision Processes

This paper studies a long-term resource allocation problem over multiple periods where each period requires a multi-stage decision-making process. We formulate the problem as an online allocation problem in an episodic finite-horizon constrained Markov decision process with an unknown non-stationary transition function and stochastic non-stationary reward and resource consumption functions. We propose the observe-then-decide regime and improve the existing decide-then-observe regime, while the two settings differ in how the observations and feedback about the reward and resource consumption functions are given to the decision-maker. We develop an online dual mirror descent algorithm that achieves near-optimal regret bounds for both settings. For the observe-then-decide regime, we prove that the expected regret against the dynamic clairvoyant optimal policy is bounded by $\tilde O(\rho^{-1}{H^{3/2}}S\sqrt{AT})$ where $\rho\in(0,1)$ is the budget parameter, $H$ is the length of the horizon, $S$ and $A$ are the numbers of states and actions, and $T$ is the number of episodes. For the decide-then-observe regime, we show that the regret against the static optimal policy that has access to the mean reward and mean resource consumption functions is bounded by $\tilde O(\rho^{-1}{H^{3/2}}S\sqrt{AT})$ with high probability. We test the numerical efficiency of our method for a variant of the resource-constrained inventory management problem

Racial Differences in Diffusion of Intensity-Modulated Radiation Therapy for Localized Prostate Cancer

Intensity-modulated radiation therapy (IMRT), an innovative treatment option for prostate cancer, has rapidly diffused over the past decade. To inform our understanding of racial disparities in prostate cancer treatment and outcomes, this study compared diffusion of IMRT in African American (AA) and Caucasian American (CA) prostate cancer patients during the early years of IMRT diffusion using the Surveillance, Epidemiology and End Results (SEER)–Medicare linked database. A retrospective cohort of 947 AA and 10,028 CA patients diagnosed with localized prostate cancer from 2002 through 2006, who were treated with either IMRT or non-IMRT as primary treatment within 1 year of diagnoses was constructed. Logistic regression was used to examine potential differences in diffusion of IMRT in AA and CA patients, while adjusting for socioeconomic and clinical covariates. A significantly smaller proportion of AA compared with CA patients received IMRT for localized prostate cancer (45% vs. 53%, p < .0001). Racial differences were apparent in multivariable analysis though did not achieve statistical significance, as time and factors associated with race (socioeconomic, geographic, and tumor related factors) explained the preponderance of variance in use of IMRT. Further research examining improved access to innovative cancer treatment and technologies is essential to reducing racial disparities in cancer care

Ampullary cancers harbor ELF3 tumor suppressor gene mutations and exhibit frequent WNT dysregulation

The ampulla of Vater is a complex cellular environment from which adenocarcinomas arise to form a group of histopathologically heterogenous tumors. To evaluate the molecular features of these tumors, 98 ampullary adenocarcinomas were evaluated and compared to 44 distal bile duct and 18 duodenal adenocarcinomas. Genomic analyses revealed mutations in the WNT signaling pathway among half of the patients and in all three adenocarcinomas irrespective of their origin and histological morphology. These tumors were characterized by a high frequency of inactivating mutations of ELF3, a high rate of microsatellite instability, and common focal deletions and amplifications, suggesting common attributes in the molecular pathogenesis are at play in these tumors. The high frequency of WNT pathway activating mutation, coupled with small-molecule inhibitors of β-catenin in clinical trials, suggests future treatment decisions for these patients may be guided by genomic analysis

Canvass: a crowd-sourced, natural-product screening library for exploring biological space

NCATS thanks Dingyin Tao for assistance with compound characterization. This research was supported by the Intramural Research Program of the National Center for Advancing Translational Sciences, National Institutes of Health (NIH). R.B.A. acknowledges support from NSF (CHE-1665145) and NIH (GM126221). M.K.B. acknowledges support from NIH (5R01GM110131). N.Z.B. thanks support from NIGMS, NIH (R01GM114061). J.K.C. acknowledges support from NSF (CHE-1665331). J.C. acknowledges support from the Fogarty International Center, NIH (TW009872). P.A.C. acknowledges support from the National Cancer Institute (NCI), NIH (R01 CA158275), and the NIH/National Institute of Aging (P01 AG012411). N.K.G. acknowledges support from NSF (CHE-1464898). B.C.G. thanks the support of NSF (RUI: 213569), the Camille and Henry Dreyfus Foundation, and the Arnold and Mabel Beckman Foundation. C.C.H. thanks the start-up funds from the Scripps Institution of Oceanography for support. J.N.J. acknowledges support from NIH (GM 063557, GM 084333). A.D.K. thanks the support from NCI, NIH (P01CA125066). D.G.I.K. acknowledges support from the National Center for Complementary and Integrative Health (1 R01 AT008088) and the Fogarty International Center, NIH (U01 TW00313), and gratefully acknowledges courtesies extended by the Government of Madagascar (Ministere des Eaux et Forets). O.K. thanks NIH (R01GM071779) for financial support. T.J.M. acknowledges support from NIH (GM116952). S.M. acknowledges support from NIH (DA045884-01, DA046487-01, AA026949-01), the Office of the Assistant Secretary of Defense for Health Affairs through the Peer Reviewed Medical Research Program (W81XWH-17-1-0256), and NCI, NIH, through a Cancer Center Support Grant (P30 CA008748). K.N.M. thanks the California Department of Food and Agriculture Pierce's Disease and Glassy Winged Sharpshooter Board for support. B.T.M. thanks Michael Mullowney for his contribution in the isolation, elucidation, and submission of the compounds in this work. P.N. acknowledges support from NIH (R01 GM111476). L.E.O. acknowledges support from NIH (R01-HL25854, R01-GM30859, R0-1-NS-12389). L.E.B., J.K.S., and J.A.P. thank the NIH (R35 GM-118173, R24 GM-111625) for research support. F.R. thanks the American Lebanese Syrian Associated Charities (ALSAC) for financial support. I.S. thanks the University of Oklahoma Startup funds for support. J.T.S. acknowledges support from ACS PRF (53767-ND1) and NSF (CHE-1414298), and thanks Drs. Kellan N. Lamb and Michael J. Di Maso for their synthetic contribution. B.S. acknowledges support from NIH (CA78747, CA106150, GM114353, GM115575). W.S. acknowledges support from NIGMS, NIH (R15GM116032, P30 GM103450), and thanks the University of Arkansas for startup funds and the Arkansas Biosciences Institute (ABI) for seed money. C.R.J.S. acknowledges support from NIH (R01GM121656). D.S.T. thanks the support of NIH (T32 CA062948-Gudas) and PhRMA Foundation to A.L.V., NIH (P41 GM076267) to D.S.T., and CCSG NIH (P30 CA008748) to C.B. Thompson. R.E.T. acknowledges support from NIGMS, NIH (GM129465). R.J.T. thanks the American Cancer Society (RSG-12-253-01-CDD) and NSF (CHE1361173) for support. D.A.V. thanks the Camille and Henry Dreyfus Foundation, the National Science Foundation (CHE-0353662, CHE-1005253, and CHE-1725142), the Beckman Foundation, the Sherman Fairchild Foundation, the John Stauffer Charitable Trust, and the Christian Scholars Foundation for support. J.W. acknowledges support from the American Cancer Society through the Research Scholar Grant (RSG-13-011-01-CDD). W.M.W.acknowledges support from NIGMS, NIH (GM119426), and NSF (CHE1755698). A.Z. acknowledges support from NSF (CHE-1463819). (Intramural Research Program of the National Center for Advancing Translational Sciences, National Institutes of Health (NIH); CHE-1665145 - NSF; CHE-1665331 - NSF; CHE-1464898 - NSF; RUI: 213569 - NSF; CHE-1414298 - NSF; CHE1361173 - NSF; CHE1755698 - NSF; CHE-1463819 - NSF; GM126221 - NIH; 5R01GM110131 - NIH; GM 063557 - NIH; GM 084333 - NIH; R01GM071779 - NIH; GM116952 - NIH; DA045884-01 - NIH; DA046487-01 - NIH; AA026949-01 - NIH; R01 GM111476 - NIH; R01-HL25854 - NIH; R01-GM30859 - NIH; R0-1-NS-12389 - NIH; R35 GM-118173 - NIH; R24 GM-111625 - NIH; CA78747 - NIH; CA106150 - NIH; GM114353 - NIH; GM115575 - NIH; R01GM121656 - NIH; T32 CA062948-Gudas - NIH; P41 GM076267 - NIH; R01GM114061 - NIGMS, NIH; R15GM116032 - NIGMS, NIH; P30 GM103450 - NIGMS, NIH; GM129465 - NIGMS, NIH; GM119426 - NIGMS, NIH; TW009872 - Fogarty International Center, NIH; U01 TW00313 - Fogarty International Center, NIH; R01 CA158275 - National Cancer Institute (NCI), NIH; P01 AG012411 - NIH/National Institute of Aging; Camille and Henry Dreyfus Foundation; Arnold and Mabel Beckman Foundation; Scripps Institution of Oceanography; P01CA125066 - NCI, NIH; 1 R01 AT008088 - National Center for Complementary and Integrative Health; W81XWH-17-1-0256 - Office of the Assistant Secretary of Defense for Health Affairs through the Peer Reviewed Medical Research Program; P30 CA008748 - NCI, NIH, through a Cancer Center Support Grant; California Department of Food and Agriculture Pierce's Disease and Glassy Winged Sharpshooter Board; American Lebanese Syrian Associated Charities (ALSAC); University of Oklahoma Startup funds; 53767-ND1 - ACS PRF; PhRMA Foundation; P30 CA008748 - CCSG NIH; RSG-12-253-01-CDD - American Cancer Society; RSG-13-011-01-CDD - American Cancer Society; CHE-0353662 - National Science Foundation; CHE-1005253 - National Science Foundation; CHE-1725142 - National Science Foundation; Beckman Foundation; Sherman Fairchild Foundation; John Stauffer Charitable Trust; Christian Scholars Foundation)Published versionSupporting documentatio

Phase I Trial of Single-Photon Emission Computed Tomography-Guided Liver-Directed Radiotherapy for Patients With Low Functional Liver Volume

BACKGROUND: Traditional constraints specify that 700 cc of liver should be spared a hepatotoxic dose when delivering liver-directed radiotherapy to reduce the risk of inducing liver failure. We investigated the role of single-photon emission computed tomography (SPECT) to identify and preferentially avoid functional liver during liver-directed radiation treatment planning in patients with preserved liver function but limited functional liver volume after receiving prior hepatotoxic chemotherapy or surgical resection. METHODS: This phase I trial with a 3 + 3 design evaluated the safety of liver-directed radiotherapy using escalating functional liver radiation dose constraints in patients with liver metastases. Dose-limiting toxicities were assessed 6-8 weeks and 6 months after completing radiotherapy. RESULTS: All 12 patients had colorectal liver metastases and received prior hepatotoxic chemotherapy; 8 patients underwent prior liver resection. Median computed tomography anatomical nontumor liver volume was 1584 cc (range = 764-2699 cc). Median SPECT functional liver volume was 1117 cc (range = 570-1928 cc). Median nontarget computed tomography and SPECT liver volumes below the volumetric dose constraint were 997 cc (range = 544-1576 cc) and 684 cc (range = 429-1244 cc), respectively. The prescription dose was 67.5-75 Gy in 15 fractions or 75-100 Gy in 25 fractions. No dose-limiting toxicities were observed during follow-up. One-year in-field control was 57%. One-year overall survival was 73%. CONCLUSION: Liver-directed radiotherapy can be safely delivered to high doses when incorporating functional SPECT into the radiation treatment planning process, which may enable sparing of lower volumes of liver than traditionally accepted in patients with preserved liver function. TRIAL REGISTRATION: NCT02626312

Efficacy of acupuncture for chronic low back pain: protocol for a randomized controlled trial

<p>Abstract</p> <p>Background</p> <p>Chronic back pain is a major public health problem and the primary reason patients seek acupuncture treatment. Therefore, an objective assessment of acupuncture efficacy is critical for making informed decisions about its appropriate role for patients with this common condition. This study addresses methodological shortcomings that have plagued previous studies evaluating acupuncture for chronic low back pain.</p> <p>Methods and Design</p> <p>A total of 640 participants (160 in each of four arms) between the ages of 18 and 70 years of age who have low back pain lasting at least 3 months will be recruited from integrated health care delivery systems in Seattle and Oakland. They will be randomized to one of two forms of Traditional Chinese Medical (TCM) acupuncture needling (individualized or standardized), a "control" group (simulated acupuncture), or to continued usual medical care. Ten treatments will be provided over 7 weeks. Study participants and the "Diagnostician" acupuncturists who evaluate participants and propose individualized treatments will be masked to the acupuncture treatment actually assigned each participant. The "Therapist" acupuncturists providing the treatments will not be masked but will have limited verbal interaction with participants. The primary outcomes, standard measures of dysfunction and bothersomeness of low back pain, will be assessed at baseline, and after 8, 26, and 52 weeks by telephone interviewers masked to treatment assignment. General health status, satisfaction with back care, days of back-related disability, and use and costs of healthcare services for back pain will also be measured. The primary analysis comparing outcomes by randomized treatment assignment will be analysis of covariance adjusted for baseline value. For both primary outcome measures, this trial will have 99% power to detect the presence of a minimal clinically significant difference among all four treatment groups and over 80% power for most pairwise comparisons. Secondary analyses will compare the proportions of participants in each group that improve by a clinically meaningful amount.</p> <p>Conclusion</p> <p>Results of this trial will help clarify the value of acupuncture needling as a treatment for chronic low back pain.</p> <p>Trial registration</p> <p>Clinical Trials.gov NCT00065585.</p

From Russia with Love: The Impact of Relocated Firms on Incumbent Survival

We identify the impact of local firm concentration on incumbent performance with a quasi natural experiment. When Germany was divided after World War II, many firms in the machine tool industry fled the Soviet occupied zone to prevent expropriation. We show that the regional location decisions of these firms upon moving to western Germany were driven by non-economic factors and heuristics rather than existing industrial conditions. Relocating firms increased the likelihood of incumbent failure in destination regions, a pattern that differs sharply from new entrants. We further provide evidence that these effects are due to increased competition for local resources