Improving Visual Prompt Tuning for Self-supervised Vision Transformers

Visual Prompt Tuning (VPT) is an effective tuning method for adapting pretrained Vision Transformers (ViTs) to downstream tasks. It leverages extra learnable tokens, known as prompts, which steer the frozen pretrained ViTs. Although VPT has demonstrated its applicability with supervised vision transformers, it often underperforms with self-supervised ones. Through empirical observations, we deduce that the effectiveness of VPT hinges largely on the ViT blocks with which the prompt tokens interact. Specifically, VPT shows improved performance on image classification tasks for MAE and MoCo v3 when the prompt tokens are inserted into later blocks rather than the first block. These observations suggest that there exists an optimal location of blocks for the insertion of prompt tokens. Unfortunately, identifying the optimal blocks for prompts within each self-supervised ViT for diverse future scenarios is a costly process. To mitigate this problem, we propose a simple yet effective method that learns a gate for each ViT block to adjust its intervention into the prompt tokens. With our method, prompt tokens are selectively influenced by blocks that require steering for task adaptation. Our method outperforms VPT variants in FGVC and VTAB image classification and ADE20K semantic segmentation. The code is available at https://github.com/ryongithub/GatedPromptTuning.Comment: International Conference on Machine Learning (ICML) 202

Training IBM Watson Using Automatically Generated Question-Answer Pairs

IBM Watson is a cognitive computing system capable of question answering in natural languages. It is believed that IBM Watson can understand large corpora and answer relevant questions more effectively than any other question-answering system currently available. To unleash the full power of Watson, however, we need to train its instance with a large number of well-prepared question-answer pairs. Obviously, manually generating such pairs in a large quantity is prohibitively time consuming and significantly limits the efficiency of Watson’s training. Recently, a large-scale dataset of over 30 million question-answer pairs was reported. Under the assumption that using such an automatically generated dataset could relieve the burden of manual question-answer generation, we tried to use this dataset to train an instance of Watson and checked the training efficiency and accuracy. According to our experiments, using this auto-generated dataset was effective for training Watson, complementing manually crafted question-answer pairs. To the best of the authors’ knowledge, this work is the first attempt to use a large-scale dataset of automatically generated question-answer pairs for training IBM Watson. We anticipate that the insights and lessons obtained from our experiments will be useful for researchers who want to expedite Watson training leveraged by automatically generated question-answer pairs

Stratifying the early radiologic trajectory in dyspneic patients with COVID-19 pneumonia

OBJECTIVE: This study aimed to stratify the early pneumonia trajectory on chest radiographs and compare patient characteristics in dyspneic patients with coronavirus disease 2019 (COVID-19). MATERIALS AND METHODS: We retrospectively included 139 COVID-19 patients with dyspnea (87 men, 62.7+/-16.3 years) and serial chest radiographs from January to September 2020. Radiographic pneumonia extent was quantified as a percentage using a previously-developed deep learning algorithm. A group-based trajectory model was used to categorize the pneumonia trajectory after symptom onset during hospitalization. Clinical findings, and outcomes were compared, and Cox regression was performed for survival analysis. RESULTS: Radiographic pneumonia trajectories were categorized into four groups. Group 1 (n = 83, 59.7%) had negligible pneumonia, and group 2 (n = 29, 20.9%) had mild pneumonia. Group 3 (n = 13, 9.4%) and group 4 (n = 14, 10.1%) showed similar considerable pneumonia extents at baseline, but group 3 had decreasing pneumonia extent at 1-2 weeks, while group 4 had increasing pneumonia extent. Intensive care unit admission and mortality were significantly more frequent in groups 3 and 4 than in groups 1 and 2 (P \u3c .05). Groups 3 and 4 shared similar clinical and laboratory findings, but thrombocytopenia ( \u3c 150x103/muL) was exclusively observed in group 4 (P = .016). When compared to groups 1 and 2, group 4 (hazard ratio, 63.3; 95% confidence interval, 7.9-504.9) had a two-fold higher risk for mortality than group 3 (hazard ratio, 31.2; 95% confidence interval, 3.5-280.2), and this elevated risk was maintained after adjusting confounders. CONCLUSION: Monitoring the early radiologic trajectory beyond baseline further prognosticated at-risk COVID-19 patients, who potentially had thrombo-inflammatory responses

The Mediating and moderating effects of teacher-child relationships on social behavior and peer preference

Abstract The purpose of this study was to investigate the mediating and moderating effects of teacher-child relationships on children's social behavior and peer preference. The participants were 508 children and 28 head teachers of their classes. Teachers measured the children's social behavior and the teacher-child relationships. Peer preference was measured by peer nomination. The association between prosocial behavior and peer preference was partially mediated by teacher-child conflict. The association between withdrawal, aggression and peer preference was fully mediated by teacher-child conflict. The moderating effects of teacher-child conflict were found between prosocial behavior and peer preference. In addition, teacher-child conflict moderated the association between physical aggression and peer preference. (peer preference), -(teacher-child relationships), (prosocial behavior), (withdrawal)

A novel de novo mutation in the serine-threonine kinase STK11 gene in a Korean patient with Peutz-Jeghers syndrome

BACKGROUND: Peutz-Jeghers syndrome (PJS) is an unusual autosomal dominant disorder characterized by mucocutaneous pigmentation and multiple gastrointestinal hamartomatous polyps. Patients with PJS are at an increased risk of developing multi-organ cancer, most frequently those involving the gastrointestinal tract. Germline mutation of the STK11 gene, which encodes a serine-threonine kinase, is responsible for PJS. METHODS: Using DNA samples obtained from the patient and his family members, we sequenced nine exons and flanking intron regions of the STK11 gene using polymerase chain reaction (PCR) and direct sequencing. RESULTS: Sequencing of the STK11 gene in the proband of the family revealed a novel 1-base pair deletion of guanine (G) in exon 6 (c.826delG; Gly276AlafsX11). This mutation resulted in a premature termination at codon 286, predicting a partial loss of the kinase domain and complete loss of the C-terminal domain. We did not observe this mutation in both parents of the PJS patient. Therefore, it is considered a novel de novo mutation. CONCLUSION: The results presented herein enlarge the spectrum of mutations of the STK11 gene by identifying a novel de novo mutation in a PJS patient and further support the hypothesis that STK11 mutations are disease-causing mutations for PJS with or without a positive family history

Titanium dioxide induces apoptotic cell death through reactive oxygen species-mediated Fas upregulation and Bax activation

Background: Titanium dioxide (TiO2) has been widely used in many areas, including biomedicine, cosmetics, and environmental engineering. Recently, it has become evident that some TiO2 particles have a considerable cytotoxic effect in normal human cells. However, the molecular basis for the cytotoxicity of TiO2 has yet to be defined.Methods and results: In this study, we demonstrated that combined treatment with TiO2 nanoparticles sized less than 100 nm and ultraviolet A irradiation induces apoptotic cell death through reactive oxygen species-dependent upregulation of Fas and conformational activation of Bax in normal human cells. Treatment with P25 TiO2 nanoparticles with a hydrodynamic size distribution centered around 70 nm (TiO2P25-70) together with ultraviolet A irradiation-induced caspase-dependent apoptotic cell death, accompanied by transcriptional upregulation of the death receptor, Fas, and conformational activation of Bax. In line with these results, knockdown of either Fas or Bax with specific siRNA significantly inhibited TiO2-induced apoptotic cell death. Moreover, inhibition of reactive oxygen species with an antioxidant, N-acetyl-L-cysteine, clearly suppressed upregulation of Fas, conformational activation of Bax, and subsequent apoptotic cell death in response to combination treatment using TiO2P25-70 and ultraviolet A irradiation.Conclusion: These results indicate that sub-100 nm sized TiO2 treatment under ultraviolet A irradiation induces apoptotic cell death through reactive oxygen species-mediated upregulation of the death receptor, Fas, and activation of the preapoptotic protein, Bax. Elucidating the molecular mechanisms by which nanosized particles induce activation of cell death signaling pathways would be critical for the development of prevention strategies to minimize the cytotoxicity of nanomaterials.This work was supported by the Korea Ministry of Environment and The Eco-Technopia 21 Project (091-091-081)

Toward Green Synthesis of Graphene Oxide Using Recycled Sulfuric Acid via Couette-Taylor Flow

Developing eco-friendly and cost-effective processes for the synthesis of graphene oxide (GO) is essential for its widespread industrial applications. In this work, we propose a green synthesis technique for GO production using recycled sulfuric acid and filter-processed oxidized natural graphite obtained from a Couette-Taylor flow reactor. The viscosity of reactant mixtures processed from Couette-Taylor flow was considerably lower (???200 cP at 25 ??C) than that of those from Hummers' method, which enabled the simple filtration process. The filtered sulfuric acid can be recycled and reused for the repetitive GO synthesis with negligible differences in the as-synthesized GO qualities. This removal of sulfuric acid has great potential in lowering the overall GO production cost as the amount of water required during the fabrication process, which takes a great portion of the total production cost, can be dramatically reduced after such acid filtration. The proposed eco-friendly GO fabrication process is expected to promote the commercial application of graphene materials into industry shortly

Kaempferol inhibits IL‑1β‑induced proliferation of rheumatoid arthritis synovial fibroblasts and the production of COX‑2, PGE2 and MMPs

Inflammatory cytokines, matrix metalloproteinases (MMPs) and cyclooxygenase (COX)‑2 released from rheumatoid arthritis synovial fibroblasts (RASFs) are involved in the destruction of both articular bone and cartilage. Kaempferol has been reported to act as an antioxidant and anti‑inflammatory agent by inhibiting nitric oxide synthase and COX enzymes. The aim of the present study was to determine the effects of kaempferol on the interleukin‑1β (IL‑1β)‑induced proliferation of RASFs and the production of MMPs, COX and prostaglandin E2 (PGE2) by RASFs. The proliferation of the RASFs stimulated with IL‑1β and treated with/without kaempferol was evaluated by CCK‑8 assay. The expression of MMPs, TIMP metallopeptidase inhibitor‑1 (TIMP‑1), COXs, PGE2 and that of intracellular MAPK signaling molecules, including p‑ERK, p‑p38, p‑JNK and nuclear factor‑κB (NF‑κB) was examined by immunoblotting or semi‑quantitative reverse transcription‑polymerase chain reaction (RT‑PCR) and ELISA under the conditions described above. Kaempferol inhibited the proliferation of both unstimulated and IL‑1β‑stimulated RASFs, as well as the mRNA and protein expression of MMP‑1, MMP-3, COX‑2 and PGE2 induced by IL‑1β. Kaempferol also inhibited the phosphorylation of ERK‑1/2, p38 and JNK, as well as the activation of NF‑κB induced by IL‑1β. These results indicate that kaempferol inhibits synovial fibroblast proliferation, as well as the production of and MMPs, COX‑2 and PGE2, which is involved in articular inflammation and destruction in rheumatoid arthritis (RA). Our data suggest that kaempferol may be a novel therapeutic agent for the treatment of RA

Protein Inhibitor of Activated STAT3 (PIAS3) Is Down-Regulated in Eutopic Endometrium of Women with Endometriosis

Endometriosis is a major cause of chronic pelvic pain and infertility. Activation of STAT3 appears central to the inflammatory phenotype of eutopic endometrium in women with endometriosis. However, the molecular mechanism by which this occurs remains unknown. Our objective is to determine how STAT3 activity is regulated in endometriosis. Protein inhibitor of activated STAT3 (PIAS3) is a negative regulator of STAT3 activity. We examined the levels of PIAS3 in endometrium from women with and without endometriosis using Western blot analysis and immunohistochemistry. Levels of PIAS3 are significantly lower, in contrast with phosphorylation of STAT3, in women with endometriosis compared to women without endometriosis. Furthermore, induction of endometriosis in the baboon showed a significant reduction of PIAS3 expression during the progression of the disease. Interferon-γ (INFγ) reduces PIAS3 protein levels and increases phospho-STAT3 levels through CXCL10 in endometrial cells, Ishikawa, and 12Z cells. These results suggest that attenuation of PIAS3 causes aberrant activation of STAT3 in endometriosis, leading to inflammatory changes that may impair fertility or cause pain