Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

A compelling use case of offline reinforcement learning (RL) is to obtain a policy initialization from existing datasets followed by fast online fine-tuning with limited interaction. However, existing offline RL methods tend to behave poorly during fine-tuning. In this paper, we study the fine-tuning problem in the context of conservative offline RL methods and we devise an approach for learning an effective initialization from offline data that also enables fast online fine-tuning capabilities. Our approach, calibrated Q-learning (Cal-QL), accomplishes this by learning a conservative value function initialization that underestimates the value of the learned policy from offline data, while also ensuring that the learned Q-values are at a reasonable scale. We refer to this property as calibration, and define it formally as providing a lower bound on the true value function of the learned policy and an upper bound on the value of some other (suboptimal) reference policy, which may simply be the behavior policy. We show that a conservative offline RL algorithm that also learns a calibrated value function leads to effective online fine-tuning, enabling us to take the benefits of offline initializations in online fine-tuning. In practice, Cal-QL can be implemented on top of the conservative Q learning (CQL) for offline RL within a one-line code change. Empirically, Cal-QL outperforms state-of-the-art methods on 9/11 fine-tuning benchmark tasks that we study in this paper. Code and video are available at https://nakamotoo.github.io/projects/Cal-QLComment: project page: https://nakamotoo.github.io/projects/Cal-Q

Epiglottis reshaping using CO2 laser: A minimally invasive technique and its potent applications

Laryngomalacia (LRM), is the most common laryngeal abnormality of the newborn, caused by a long curled epiglottis, which prolapses posteriorly. Epiglottis prolapse during inspiration (acquired laryngomalacia) is an unusual cause of airway obstruction and a rare cause of obstructive sleep apnea syndrome (OSAS)

On the Lp-theory of C0-semigroups associated with second-order elliptic operators with complex singular coefficients

A work in Perturbation Theory, with a purpose to consider well-posedness of elliptic and parabolic PDE with singular complex coefficient

Survival of Chondrocytes in Rabbit Septal Cartilage After Electromechanical Reshaping

Electromechanical reshaping (EMR) has been recently described as an alternative method for reshaping facial cartilage without the need for incisions or sutures. This study focuses on determining the short- and long-term viability of chondrocytes following EMR in cartilage grafts maintained in tissue culture. Flat rabbit nasal septal cartilage specimens were bent into semi-cylindrical shapes by an aluminum jig while a constant electric voltage was applied across the concave and convex surfaces. After EMR, specimens were maintained in culture media for 64 days. Over this time period, specimens were serially biopsied and then stained with a fluorescent live–dead assay system and imaged using laser scanning confocal microscopy. In addition, the fraction of viable chondrocytes was measured, correlated with voltage, voltage application time, electric field configuration, and examined serially. The fraction of viable chondrocytes decreased with voltage and application time. High local electric field intensity and proximity to the positive electrode also focally reduced chondrocyte viability. The density of viable chondrocytes decreased over time and reached a steady state after 2–4 weeks. Viable cells were concentrated within the central region of the specimen. Approximately 20% of original chondrocytes remained viable after reshaping with optimal voltage and application time parameters and compared favorably with conventional surgical shape change techniques such as morselization

Chromogranin A, a significant prognostic factor in small cell lung cancer

Chromogranin A (CgA) is a protein present in neuroendocrine vesicles. Small cell lung cancer (SCLC) is considered a neuroendocrine tumour. It is possible to demonstrate CgA expression in SCLC by immunohistochemical methods. Since CgA is released to the circulation it might also work as a clinical tumour marker. We used a newly developed two-site enzyme-linked immunosorbent assay for CgA in plasma from 150 newly diagnosed patients with SCLC. Follow-up was for a minimum of 5 years. Thirty-seven per cent of the patients had elevated pretreatment values and the values were significantly related to stage of disease. Multivariable analysis by Cox's proportional hazard model including nine known prognostic factors disclosed performance status as the most influential prognostic factor followed by stage of disease, CgA and LDH. A simple prognostic index (PI) could be established based on these four pretreatment features. In this way the patients could be separated into three groups with significant different prognosis. The median survival and 95% confidence intervals for the three groups were as follows: 424 days (311–537), 360 days (261–459) and 174 days (105–243). © 1999 Cancer Research Campaig

The feasibility of gene therapy in the treatment of head and neck cancer

Standard approach to the treatment of head and neck cancer include surgery, chemotherapy, and radiation. More recently, dramatic increases in our knowledge of the molecular and genetic basis of cancer combined with advances in technology have resulted in novel molecular therapies for this disease. In particular, gene therapy, which involves the transfer of genetic material to cells to produce a therapeutic effect, has become a promising approach. Clinical trials concerning gene therapy strategies in head and neck cancer as well as combination of these strategies with chemotherapy and radiation therapy will be discussed

Performance of CMS muon reconstruction in pp collision events at sqrt(s) = 7 TeV

The performance of muon reconstruction, identification, and triggering in CMS has been studied using 40 inverse picobarns of data collected in pp collisions at sqrt(s) = 7 TeV at the LHC in 2010. A few benchmark sets of selection criteria covering a wide range of physics analysis needs have been examined. For all considered selections, the efficiency to reconstruct and identify a muon with a transverse momentum pT larger than a few GeV is above 95% over the whole region of pseudorapidity covered by the CMS muon system, abs(eta) < 2.4, while the probability to misidentify a hadron as a muon is well below 1%. The efficiency to trigger on single muons with pT above a few GeV is higher than 90% over the full eta range, and typically substantially better. The overall momentum scale is measured to a precision of 0.2% with muons from Z decays. The transverse momentum resolution varies from 1% to 6% depending on pseudorapidity for muons with pT below 100 GeV and, using cosmic rays, it is shown to be better than 10% in the central region up to pT = 1 TeV. Observed distributions of all quantities are well reproduced by the Monte Carlo simulation.Comment: Replaced with published version. Added journal reference and DO

Performance of CMS muon reconstruction in pp collision events at sqrt(s) = 7 TeV

The performance of muon reconstruction, identification, and triggering in CMS has been studied using 40 inverse picobarns of data collected in pp collisions at sqrt(s) = 7 TeV at the LHC in 2010. A few benchmark sets of selection criteria covering a wide range of physics analysis needs have been examined. For all considered selections, the efficiency to reconstruct and identify a muon with a transverse momentum pT larger than a few GeV is above 95% over the whole region of pseudorapidity covered by the CMS muon system, abs(eta) < 2.4, while the probability to misidentify a hadron as a muon is well below 1%. The efficiency to trigger on single muons with pT above a few GeV is higher than 90% over the full eta range, and typically substantially better. The overall momentum scale is measured to a precision of 0.2% with muons from Z decays. The transverse momentum resolution varies from 1% to 6% depending on pseudorapidity for muons with pT below 100 GeV and, using cosmic rays, it is shown to be better than 10% in the central region up to pT = 1 TeV. Observed distributions of all quantities are well reproduced by the Monte Carlo simulation.Comment: Replaced with published version. Added journal reference and DO

Azimuthal anisotropy of charged particles at high transverse momenta in PbPb collisions at sqrt(s[NN]) = 2.76 TeV

The azimuthal anisotropy of charged particles in PbPb collisions at nucleon-nucleon center-of-mass energy of 2.76 TeV is measured with the CMS detector at the LHC over an extended transverse momentum (pt) range up to approximately 60 GeV. The data cover both the low-pt region associated with hydrodynamic flow phenomena and the high-pt region where the anisotropies may reflect the path-length dependence of parton energy loss in the created medium. The anisotropy parameter (v2) of the particles is extracted by correlating charged tracks with respect to the event-plane reconstructed by using the energy deposited in forward-angle calorimeters. For the six bins of collision centrality studied, spanning the range of 0-60% most-central events, the observed v2 values are found to first increase with pt, reaching a maximum around pt = 3 GeV, and then to gradually decrease to almost zero, with the decline persisting up to at least pt = 40 GeV over the full centrality range measured.Comment: Replaced with published version. Added journal reference and DO