3,798 research outputs found
Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
A compelling use case of offline reinforcement learning (RL) is to obtain a
policy initialization from existing datasets followed by fast online
fine-tuning with limited interaction. However, existing offline RL methods tend
to behave poorly during fine-tuning. In this paper, we study the fine-tuning
problem in the context of conservative offline RL methods and we devise an
approach for learning an effective initialization from offline data that also
enables fast online fine-tuning capabilities. Our approach, calibrated
Q-learning (Cal-QL), accomplishes this by learning a conservative value
function initialization that underestimates the value of the learned policy
from offline data, while also ensuring that the learned Q-values are at a
reasonable scale. We refer to this property as calibration, and define it
formally as providing a lower bound on the true value function of the learned
policy and an upper bound on the value of some other (suboptimal) reference
policy, which may simply be the behavior policy. We show that a conservative
offline RL algorithm that also learns a calibrated value function leads to
effective online fine-tuning, enabling us to take the benefits of offline
initializations in online fine-tuning. In practice, Cal-QL can be implemented
on top of the conservative Q learning (CQL) for offline RL within a one-line
code change. Empirically, Cal-QL outperforms state-of-the-art methods on 9/11
fine-tuning benchmark tasks that we study in this paper. Code and video are
available at https://nakamotoo.github.io/projects/Cal-QLComment: project page: https://nakamotoo.github.io/projects/Cal-Q
Epiglottis reshaping using CO2 laser: A minimally invasive technique and its potent applications
Laryngomalacia (LRM), is the most common laryngeal abnormality of the newborn, caused by a long curled epiglottis, which prolapses posteriorly. Epiglottis prolapse during inspiration (acquired laryngomalacia) is an unusual cause of airway obstruction and a rare cause of obstructive sleep apnea syndrome (OSAS)
On the Lp-theory of C0-semigroups associated with second-order elliptic operators with complex singular coefficients
A work in Perturbation Theory, with a purpose to consider well-posedness of elliptic and parabolic PDE with singular complex coefficient
Survival of Chondrocytes in Rabbit Septal Cartilage After Electromechanical Reshaping
Electromechanical reshaping (EMR) has been recently described as an alternative method for reshaping facial cartilage without the need for incisions or sutures. This study focuses on determining the short- and long-term viability of chondrocytes following EMR in cartilage grafts maintained in tissue culture. Flat rabbit nasal septal cartilage specimens were bent into semi-cylindrical shapes by an aluminum jig while a constant electric voltage was applied across the concave and convex surfaces. After EMR, specimens were maintained in culture media for 64Â days. Over this time period, specimens were serially biopsied and then stained with a fluorescent liveâdead assay system and imaged using laser scanning confocal microscopy. In addition, the fraction of viable chondrocytes was measured, correlated with voltage, voltage application time, electric field configuration, and examined serially. The fraction of viable chondrocytes decreased with voltage and application time. High local electric field intensity and proximity to the positive electrode also focally reduced chondrocyte viability. The density of viable chondrocytes decreased over time and reached a steady state after 2â4Â weeks. Viable cells were concentrated within the central region of the specimen. Approximately 20% of original chondrocytes remained viable after reshaping with optimal voltage and application time parameters and compared favorably with conventional surgical shape change techniques such as morselization
Chromogranin A, a significant prognostic factor in small cell lung cancer
Chromogranin A (CgA) is a protein present in neuroendocrine vesicles. Small cell lung cancer (SCLC) is considered a neuroendocrine tumour. It is possible to demonstrate CgA expression in SCLC by immunohistochemical methods. Since CgA is released to the circulation it might also work as a clinical tumour marker. We used a newly developed two-site enzyme-linked immunosorbent assay for CgA in plasma from 150 newly diagnosed patients with SCLC. Follow-up was for a minimum of 5 years. Thirty-seven per cent of the patients had elevated pretreatment values and the values were significantly related to stage of disease. Multivariable analysis by Cox's proportional hazard model including nine known prognostic factors disclosed performance status as the most influential prognostic factor followed by stage of disease, CgA and LDH. A simple prognostic index (PI) could be established based on these four pretreatment features. In this way the patients could be separated into three groups with significant different prognosis. The median survival and 95% confidence intervals for the three groups were as follows: 424 days (311â537), 360 days (261â459) and 174 days (105â243). Š 1999 Cancer Research Campaig
The feasibility of gene therapy in the treatment of head and neck cancer
Standard approach to the treatment of head and neck cancer include surgery, chemotherapy, and radiation. More recently, dramatic increases in our knowledge of the molecular and genetic basis of cancer combined with advances in technology have resulted in novel molecular therapies for this disease. In particular, gene therapy, which involves the transfer of genetic material to cells to produce a therapeutic effect, has become a promising approach. Clinical trials concerning gene therapy strategies in head and neck cancer as well as combination of these strategies with chemotherapy and radiation therapy will be discussed
Performance of CMS muon reconstruction in pp collision events at sqrt(s) = 7 TeV
The performance of muon reconstruction, identification, and triggering in CMS
has been studied using 40 inverse picobarns of data collected in pp collisions
at sqrt(s) = 7 TeV at the LHC in 2010. A few benchmark sets of selection
criteria covering a wide range of physics analysis needs have been examined.
For all considered selections, the efficiency to reconstruct and identify a
muon with a transverse momentum pT larger than a few GeV is above 95% over the
whole region of pseudorapidity covered by the CMS muon system, abs(eta) < 2.4,
while the probability to misidentify a hadron as a muon is well below 1%. The
efficiency to trigger on single muons with pT above a few GeV is higher than
90% over the full eta range, and typically substantially better. The overall
momentum scale is measured to a precision of 0.2% with muons from Z decays. The
transverse momentum resolution varies from 1% to 6% depending on pseudorapidity
for muons with pT below 100 GeV and, using cosmic rays, it is shown to be
better than 10% in the central region up to pT = 1 TeV. Observed distributions
of all quantities are well reproduced by the Monte Carlo simulation.Comment: Replaced with published version. Added journal reference and DO
Performance of CMS muon reconstruction in pp collision events at sqrt(s) = 7 TeV
The performance of muon reconstruction, identification, and triggering in CMS
has been studied using 40 inverse picobarns of data collected in pp collisions
at sqrt(s) = 7 TeV at the LHC in 2010. A few benchmark sets of selection
criteria covering a wide range of physics analysis needs have been examined.
For all considered selections, the efficiency to reconstruct and identify a
muon with a transverse momentum pT larger than a few GeV is above 95% over the
whole region of pseudorapidity covered by the CMS muon system, abs(eta) < 2.4,
while the probability to misidentify a hadron as a muon is well below 1%. The
efficiency to trigger on single muons with pT above a few GeV is higher than
90% over the full eta range, and typically substantially better. The overall
momentum scale is measured to a precision of 0.2% with muons from Z decays. The
transverse momentum resolution varies from 1% to 6% depending on pseudorapidity
for muons with pT below 100 GeV and, using cosmic rays, it is shown to be
better than 10% in the central region up to pT = 1 TeV. Observed distributions
of all quantities are well reproduced by the Monte Carlo simulation.Comment: Replaced with published version. Added journal reference and DO
Azimuthal anisotropy of charged particles at high transverse momenta in PbPb collisions at sqrt(s[NN]) = 2.76 TeV
The azimuthal anisotropy of charged particles in PbPb collisions at
nucleon-nucleon center-of-mass energy of 2.76 TeV is measured with the CMS
detector at the LHC over an extended transverse momentum (pt) range up to
approximately 60 GeV. The data cover both the low-pt region associated with
hydrodynamic flow phenomena and the high-pt region where the anisotropies may
reflect the path-length dependence of parton energy loss in the created medium.
The anisotropy parameter (v2) of the particles is extracted by correlating
charged tracks with respect to the event-plane reconstructed by using the
energy deposited in forward-angle calorimeters. For the six bins of collision
centrality studied, spanning the range of 0-60% most-central events, the
observed v2 values are found to first increase with pt, reaching a maximum
around pt = 3 GeV, and then to gradually decrease to almost zero, with the
decline persisting up to at least pt = 40 GeV over the full centrality range
measured.Comment: Replaced with published version. Added journal reference and DO
- âŚ