32 research outputs found
ImageNetVC: Zero-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories
Recently, Pretrained Language Models (PLMs) have been serving as
general-purpose interfaces, posing a significant demand for comprehensive
visual knowledge. However, it remains unclear how well current PLMs and their
visually augmented counterparts (VaLMs) can master visual commonsense
knowledge. To investigate this, we propose ImageNetVC, a fine-grained,
human-annotated dataset specifically designed for zero-shot visual commonsense
evaluation across 1,000 ImageNet categories. Utilizing ImageNetVC, we delve
into the fundamental visual commonsense knowledge of both unimodal PLMs and
VaLMs, uncovering the scaling law and the influence of the backbone model on
VaLMs. Furthermore, we investigate the factors affecting the visual commonsense
knowledge of large-scale models, providing insights into the development of
language models enriched with visual commonsense knowledge. Our code and
dataset are available at https://github.com/hemingkx/ImageNetVC
Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation
We propose Speculative Decoding (SpecDec), for the first time ever, to
formally study exploiting the idea of speculative execution to accelerate
autoregressive (AR) decoding. Speculative Decoding has two innovations:
Spec-Drafter -- an independent model specially optimized for efficient and
accurate drafting -- and Spec-Verification -- a reliable method for verifying
the drafted tokens efficiently in the decoding paradigm. Experimental results
on various seq2seq tasks including machine translation and abstractive
summarization show our approach can achieve around speedup for the
popular Transformer architectures with comparable generation quality to beam
search decoding, refreshing the impression that the draft-then-verify paradigm
introduces only speedup. In addition to the
remarkable speedup, we also demonstrate 3 additional advantages of SpecDec,
revealing its practical value for accelerating generative models in real-world
applications. Our models and codes are available at
https://github.com/hemingkx/SpecDec.Comment: (Early 2022): Initially announced with the name
"Generalized Aggressive Decoding"; (September 2022): Renamed to
"Speculative Decoding" as the ICLR'23 submission
(https://openreview.net/pdf?id=H-VlwsYvVi), marking
"Speculative Decoding" has been publicly proposed. : EMNLP'23
Findings camera read
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization
Pretrained language models have achieved remarkable success in natural
language understanding. However, fine-tuning pretrained models on limited
training data tends to overfit and thus diminish performance. This paper
presents Bi-Drop, a fine-tuning strategy that selectively updates model
parameters using gradients from various sub-nets dynamically generated by
dropout. The sub-net estimation of Bi-Drop is performed in an in-batch manner,
so it overcomes the problem of hysteresis in sub-net updating, which is
possessed by previous methods that perform asynchronous sub-net estimation.
Also, Bi-Drop needs only one mini-batch to estimate the sub-net so it achieves
higher utility of training data. Experiments on the GLUE benchmark demonstrate
that Bi-Drop consistently outperforms previous fine-tuning methods.
Furthermore, empirical results also show that Bi-Drop exhibits excellent
generalization ability and robustness for domain transfer, data imbalance, and
low-resource scenarios.Comment: EMNLP 2023 Findings. Camera-ready version. Co-first authors with
equal contribution
The Phylogenetic Origin of oskar Coincided with the Origin of Maternally Provisioned Germ Plasm and Pole Cells at the Base of the Holometabola
The establishment of the germline is a critical, yet surprisingly evolutionarily
labile, event in the development of sexually reproducing animals. In the fly
Drosophila, germ cells acquire their fate early during
development through the inheritance of the germ plasm, a specialized maternal
cytoplasm localized at the posterior pole of the oocyte. The gene
oskar (osk) is both necessary and
sufficient for assembling this substance. Both maternal germ plasm and
oskar are evolutionary novelties within the insects, as the
germline is specified by zygotic induction in basally branching insects, and
osk has until now only been detected in dipterans. In order
to understand the origin of these evolutionary novelties, we used comparative
genomics, parental RNAi, and gene expression analyses in multiple insect
species. We have found that the origin of osk and its role in
specifying the germline coincided with the innovation of maternal germ plasm and
pole cells at the base of the holometabolous insects and that losses of
osk are correlated with changes in germline determination
strategies within the Holometabola. Our results indicate that the invention of
the novel gene osk was a key innovation that allowed the
transition from the ancestral late zygotic mode of germline induction to a
maternally controlled establishment of the germline found in many holometabolous
insect species. We propose that the ancestral role of osk was
to connect an upstream network ancestrally involved in mRNA localization and
translational control to a downstream regulatory network ancestrally involved in
executing the germ cell program
Recommended from our members
Effect of Hydrocortisone on Mortality and Organ Support in Patients With Severe COVID-19: The REMAP-CAP COVID-19 Corticosteroid Domain Randomized Clinical Trial.
Importance: Evidence regarding corticosteroid use for severe coronavirus disease 2019 (COVID-19) is limited. Objective: To determine whether hydrocortisone improves outcome for patients with severe COVID-19. Design, Setting, and Participants: An ongoing adaptive platform trial testing multiple interventions within multiple therapeutic domains, for example, antiviral agents, corticosteroids, or immunoglobulin. Between March 9 and June 17, 2020, 614 adult patients with suspected or confirmed COVID-19 were enrolled and randomized within at least 1 domain following admission to an intensive care unit (ICU) for respiratory or cardiovascular organ support at 121 sites in 8 countries. Of these, 403 were randomized to open-label interventions within the corticosteroid domain. The domain was halted after results from another trial were released. Follow-up ended August 12, 2020. Interventions: The corticosteroid domain randomized participants to a fixed 7-day course of intravenous hydrocortisone (50 mg or 100 mg every 6 hours) (nâ=â143), a shock-dependent course (50 mg every 6 hours when shock was clinically evident) (nâ=â152), or no hydrocortisone (nâ=â108). Main Outcomes and Measures: The primary end point was organ support-free days (days alive and free of ICU-based respiratory or cardiovascular support) within 21 days, where patients who died were assigned -1 day. The primary analysis was a bayesian cumulative logistic model that included all patients enrolled with severe COVID-19, adjusting for age, sex, site, region, time, assignment to interventions within other domains, and domain and intervention eligibility. Superiority was defined as the posterior probability of an odds ratio greater than 1 (threshold for trial conclusion of superiority >99%). Results: After excluding 19 participants who withdrew consent, there were 384 patients (mean age, 60 years; 29% female) randomized to the fixed-dose (nâ=â137), shock-dependent (nâ=â146), and no (nâ=â101) hydrocortisone groups; 379 (99%) completed the study and were included in the analysis. The mean age for the 3 groups ranged between 59.5 and 60.4 years; most patients were male (range, 70.6%-71.5%); mean body mass index ranged between 29.7 and 30.9; and patients receiving mechanical ventilation ranged between 50.0% and 63.5%. For the fixed-dose, shock-dependent, and no hydrocortisone groups, respectively, the median organ support-free days were 0 (IQR, -1 to 15), 0 (IQR, -1 to 13), and 0 (-1 to 11) days (composed of 30%, 26%, and 33% mortality rates and 11.5, 9.5, and 6 median organ support-free days among survivors). The median adjusted odds ratio and bayesian probability of superiority were 1.43 (95% credible interval, 0.91-2.27) and 93% for fixed-dose hydrocortisone, respectively, and were 1.22 (95% credible interval, 0.76-1.94) and 80% for shock-dependent hydrocortisone compared with no hydrocortisone. Serious adverse events were reported in 4 (3%), 5 (3%), and 1 (1%) patients in the fixed-dose, shock-dependent, and no hydrocortisone groups, respectively. Conclusions and Relevance: Among patients with severe COVID-19, treatment with a 7-day fixed-dose course of hydrocortisone or shock-dependent dosing of hydrocortisone, compared with no hydrocortisone, resulted in 93% and 80% probabilities of superiority with regard to the odds of improvement in organ support-free days within 21 days. However, the trial was stopped early and no treatment strategy met prespecified criteria for statistical superiority, precluding definitive conclusions. Trial Registration: ClinicalTrials.gov Identifier: NCT02735707
Effect of angiotensin-converting enzyme inhibitor and angiotensin receptor blocker initiation on organ support-free days in patients hospitalized with COVID-19
IMPORTANCE Overactivation of the renin-angiotensin system (RAS) may contribute to poor clinical outcomes in patients with COVID-19.
Objective To determine whether angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) initiation improves outcomes in patients hospitalized for COVID-19.
DESIGN, SETTING, AND PARTICIPANTS In an ongoing, adaptive platform randomized clinical trial, 721 critically ill and 58 nonâcritically ill hospitalized adults were randomized to receive an RAS inhibitor or control between March 16, 2021, and February 25, 2022, at 69 sites in 7 countries (final follow-up on June 1, 2022).
INTERVENTIONS Patients were randomized to receive open-label initiation of an ACE inhibitor (nâ=â257), ARB (nâ=â248), ARB in combination with DMX-200 (a chemokine receptor-2 inhibitor; nâ=â10), or no RAS inhibitor (control; nâ=â264) for up to 10 days.
MAIN OUTCOMES AND MEASURES The primary outcome was organ supportâfree days, a composite of hospital survival and days alive without cardiovascular or respiratory organ support through 21 days. The primary analysis was a bayesian cumulative logistic model. Odds ratios (ORs) greater than 1 represent improved outcomes.
RESULTS On February 25, 2022, enrollment was discontinued due to safety concerns. Among 679 critically ill patients with available primary outcome data, the median age was 56 years and 239 participants (35.2%) were women. Median (IQR) organ supportâfree days among critically ill patients was 10 (â1 to 16) in the ACE inhibitor group (nâ=â231), 8 (â1 to 17) in the ARB group (nâ=â217), and 12 (0 to 17) in the control group (nâ=â231) (median adjusted odds ratios of 0.77 [95% bayesian credible interval, 0.58-1.06] for improvement for ACE inhibitor and 0.76 [95% credible interval, 0.56-1.05] for ARB compared with control). The posterior probabilities that ACE inhibitors and ARBs worsened organ supportâfree days compared with control were 94.9% and 95.4%, respectively. Hospital survival occurred in 166 of 231 critically ill participants (71.9%) in the ACE inhibitor group, 152 of 217 (70.0%) in the ARB group, and 182 of 231 (78.8%) in the control group (posterior probabilities that ACE inhibitor and ARB worsened hospital survival compared with control were 95.3% and 98.1%, respectively).
CONCLUSIONS AND RELEVANCE In this trial, among critically ill adults with COVID-19, initiation of an ACE inhibitor or ARB did not improve, and likely worsened, clinical outcomes.
TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT0273570
Real-Time and Quantitative Measurement of Crack-Tip Stress Intensity Factors Using Digital Holographic Interferometry
Detection of the crack in an object is a critical problem for the health monitoring of a transparent object. The real-time and quantitative measurement of the crack-tip stress intensity factor (SIF) remains an open issue. In this paper, an approach for real-time and quantitative measurement for the SIFs of a Mode I crack is presented based on digital holographic interferometry (DHI). A transmission digital holographic system is established to measure the phase difference of an object wave during loading. The expression to achieve the SIF from the phase difference is formulated. To enhance the accuracy of measurement, calibrated phase unwrapping based on least-squares and iteration and median filtering is applied to retrieve the actual phase from the noisy wrapped one. The SIFs of the Mode I crack in a transparent polymethyl methacrylate (PMMA) specimen are measured by this approach. The results are compared with the theoretical ones to demonstrate the feasibility of the proposed approach
Preparation and Performance of a Fixed Bed Catalyst for the Oxidation of Sodium Mercaptides
The activated-carbon supported cobalt pthalocyanine as a fixed bed catalyst (CoPc/C) was prepared by impregnation method and its performance on the oxidation of sodium mercaptides in light oil sweetening was investigated. The FTIR, XRD, and SEM analysis indicated that the active component dispersed well on the carrier and the results of the TG analysis showed that CoPc/C has good thermostability. It was tested that the prepared catalyst has a high catalytic activity towards sodium mercaptides. The removal rate of n-C4H9SNa was up to 100 % and for t-C4H9SNa, was 87.5% at a reaction time of 30 min. With the reaction temperature raised from 20 °C to 60 °C at intervals of ten degrees, the oxidation rate increased obviously, especially in the first ten minutes. A kinetic model mainly related to the transfer process was supposed. The catalyst CoPc/C had a good anti-loss performance of the active component both in water and alkali liquor when used. © 2014 BCREC UNDIP. All rights reservedSubmitted: 20th June 2013; Revised: 1st March 2014; Accepted: 22nd March 2014[How to Cite: Heming, W., Xianshang, L., Lijun, Z., Yulu, Z., Daohong, X., (2014). Preparation and Performance of a Fixed Bed Catalyst for the Oxidation of Sodium Mercaptides. Bulletin of Chemical Reaction Engineering & Catalysis, 9 (2): 87-92.(doi:10.9767/bcrec.9.2.5113.87-92)][Permalink/DOI: http://dx.doi.org/10.9767/bcrec.9.2.5113.87-92] </p
Preparation and Performance of a Fixed Bed Catalyst for the Oxidation of Sodium Mercaptides
The activated-carbon supported cobalt pthalocyanine as a fixed bed catalyst (CoPc/C) was prepared by impregnation method and its performance on the oxidation of sodium mercaptides in light oil sweetening was investigated. The FTIR, XRD, and SEM analysis indicated that the active component dispersed well on the carrier and the results of the TG analysis showed that CoPc/C has good thermostability. It was tested that the prepared catalyst has a high catalytic activity towards sodium mercaptides. The removal rate of n-C4H9SNa was up to 100 % and for t-C4H9SNa, was 87.5% at a reaction time of 30 min. With the reaction temperature raised from 20 °C to 60 °C at intervals of ten degrees, the oxidation rate increased obviously, especially in the first ten minutes. A kinetic model mainly related to the transfer process was supposed. The catalyst CoPc/C had a good anti-loss performance of the active component both in water and alkali liquor when used. © 2014 BCREC UNDIP. All rights reservedSubmitted: 20th June 2013; Revised: 1st March 2014; Accepted: 22nd March 2014[How to Cite: Heming, W., Xianshang, L., Lijun, Z., Yulu, Z., Daohong, X., (2014). Preparation and Performance of a Fixed Bed Catalyst for the Oxidation of Sodium Mercaptides. Bulletin of Chemical Reaction Engineering & Catalysis, 9 (2): 87-92.(doi:10.9767/bcrec.9.2.5113.87-92)][Permalink/DOI: http://dx.doi.org/10.9767/bcrec.9.2.5113.87-92] </p