On the Difference of BERT-style and CLIP-style Text Encoders

Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e.g., BERT, one of the representative models. Recently, contrastive language-image pretraining (CLIP) has also attracted attention, especially its vision models that achieve excellent performance on a broad range of vision tasks. However, few studies are dedicated to studying the text encoders learned by CLIP. In this paper, we analyze the difference between BERT-style and CLIP-style text encoders from three experiments: (i) general text understanding, (ii) vision-centric text understanding, and (iii) text-to-image generation. Experimental analyses show that although CLIP-style text encoders underperform BERT-style ones for general text understanding tasks, they are equipped with a unique ability, i.e., synesthesia, for the cross-modal association, which is more similar to the senses of humans.Comment: Natural Language Processing. 10 pages, 1 figure. Findings of ACL-202

Integrated transcriptome analysis reveals miRNA-mRNA crosstalk in laryngeal squamous cell carcinoma.

Next generation sequencing (NGS) has proven to be a powerful tool in delineating myriads of molecular subtypes of cancer, as well as in revealing accumulation of genomic mutations throughout cancer progression. Whole genome microRNA (miRNA) and mRNA expression profiles were obtained from patients with laryngeal squamous cell carcinoma (LSCC) using deep sequencing technology, and were analyzed by utilizing integrative computational approaches. A large number of protein-coding and non-coding genes were detected to be differentially expressed, indicating a functional switch in LSCC cells. A total of 127 mutated genes were detected to be significantly associated with ectoderm and epidermis development. Eleven miRNAs were found to be differentially expressed, including a potential cancer suppressor miRNA, mir-34c, which was dramatically down-regulated. Integrated analysis of mRNA and miRNA transcriptomes further revealed correlated dynamics among 11 miRNAs and 138 targeted genes, forming a highly dynamical co-regulation network response to LSCC development

Design of Affordable 3D Printers

poster abstractThe recent expiration of Fused Deposition Modeling (FDM) patents sparked a growth in the 3D printing industry. Fused Deposition Modeling is the most common way of 3D printing parts. It takes a material, usually a plastic, melts it, and then builds a part layer by layer from the molten material. As patents for 3D printing technologies continue to expire, 3D printing will continue to see a large growth in popularity for several different applications; however, there are currently limitations on 3D printers preventing them from entering certain markets. The goal of our project was to address two of the biggest current limitations: the cost of the 3D printer and the ability to print with different materials. We addressed these issues by researching and building two different types of 3D printers along with researching different ways to print different materials. The goal for the first project was to design and assemble an affordable ceramic 3D printer. We researched and purchased an affordable delta 3D printer kit and an affordable ceramic extrusion system. The goal for the second project was to design and assemble an affordable dual extruder desktop 3D printer that could print two different plastics. We successfully built the delta 3D printer and it is working correctly. The dual extruder desktop 3D printer has been assembled. For both projects, we were able to assemble low-cost 3D printers. In conclusion, this research has resulted in two affordable 3D printers with the potential to 3D print different materials

HuatuoGPT, towards Taming Language Model to Be a Doctor

In this paper, we present HuatuoGPT, a large language model (LLM) for medical consultation. The core recipe of HuatuoGPT is to leverage both \textit{distilled data from ChatGPT} and \textit{real-world data from doctors} in the supervised fine-tuned stage. The responses of ChatGPT are usually detailed, well-presented and informative while it cannot perform like a doctor in many aspects, e.g. for integrative diagnosis. We argue that real-world data from doctors would be complementary to distilled data in the sense the former could tame a distilled language model to perform like doctors. To better leverage the strengths of both data, we train a reward model to align the language model with the merits that both data bring, following an RLAIF (reinforced learning from AI feedback) fashion. To evaluate and benchmark the models, we propose a comprehensive evaluation scheme (including automatic and manual metrics). Experimental results demonstrate that HuatuoGPT achieves state-of-the-art results in performing medical consultation among open-source LLMs in GPT-4 evaluation, human evaluation, and medical benchmark datasets. It is worth noting that by using additional real-world data and RLAIF, the distilled language model (i.e., HuatuoGPT) outperforms its teacher model ChatGPT in most cases. Our code, data, and models are publicly available at \url{https://github.com/FreedomIntelligence/HuatuoGPT}. The online demo is available at \url{https://www.HuatuoGPT.cn/}

CMB: A Comprehensive Medical Benchmark in Chinese

Large Language Models (LLMs) provide a possibility to make a great breakthrough in medicine. The establishment of a standardized medical benchmark becomes a fundamental cornerstone to measure progression. However, medical environments in different regions have their local characteristics, e.g., the ubiquity and significance of traditional Chinese medicine within China. Therefore, merely translating English-based medical evaluation may result in \textit{contextual incongruities} to a local region. To solve the issue, we propose a localized medical benchmark called CMB, a Comprehensive Medical Benchmark in Chinese, designed and rooted entirely within the native Chinese linguistic and cultural framework. While traditional Chinese medicine is integral to this evaluation, it does not constitute its entirety. Using this benchmark, we have evaluated several prominent large-scale LLMs, including ChatGPT, GPT-4, dedicated Chinese LLMs, and LLMs specialized in the medical domain. It is worth noting that our benchmark is not devised as a leaderboard competition but as an instrument for self-assessment of model advancements. We hope this benchmark could facilitate the widespread adoption and enhancement of medical LLMs within China. Check details in \url{https://cmedbenchmark.llmzoo.com/}

Partial preservation of the normal thyroid gland based on tumor diameter may be possible in small medullary thyroid carcinoma: a two-center 15-year retrospective study

BackgroundAt present, there are some controversies in the formulation of surgical protocol for small medullary thyroid carcinoma(s-MTC). We wanted to explore the feasibility of normal thyroid gland retention in small medullary thyroid carcinoma based on different tumor diameters and its prognostic impact on the tumor.MethodsThe data of patients with stage T1 MTC treated at Tianjin Cancer Hospital and Sichuan Cancer Hospital from 2006 to 2021 were analyzed. The tumor diameters of 0.5 cm and 1.0 cm were used as dividing points. The outcomes were tumor recurrence, metastasis, or patient death. Survival was estimated by the Kapan–Meier curve.ResultsA total of 121 T1 s-MTC patients were included, including 55 with total thyroidectomy (TT) and 66 with subthyroidectomy (Sub-TT). There were eleven cases of tumor recurrence and metastasis, and four patients died. When the tumor diameter was 1.0 cm as the cut-off point, tumor diameter (p = 0.010), TT (p = 0.028), unilateral and bilateral type (p = 0.009), and TNM staging (p = 0.007) had significant effects on progression-free survival (PFS). The tumor diameter, unilateral and bilateral type, and TT were risk factors for the prognosis of T1 MTC (p < 0.05).ConclusionThe tumor diameter of 1.0 cm can be used as a cut-off point for stage T1 MTC. Alt-hough there was no significant difference in overall survival (OS) between T1a and T1b in patients, tumor diameter significantly influenced PFS. TT is not necessary for patients with sporadic MTC with T1a

Preparation and Properties Magnetite/Polyimide Composites

To make the polyimide (PI) composite material having both heat resistance and magnetic permeability, Fe3O4/PI magnetic polyimide composite powder was synthesized by one-step solvothermal method. 4,4'-diaminodiphenyl ether (ODA) as monomers, ether anhydride polyamic acid (PAA) was designed and prepared by Fe3+ and polyimide precursor-polyamic acid triethylamine salt (PAAS) was reacted in a reactor, and the PI/Fe3O4 composite powders were synthesized under the condition of high temperature and high pressure. The obtained PI/Fe3O4 composite powders were analyzed by infrared spectroscopy, microscopic morphology and thermal properties. The morphology and structure of the samples were both characterized by scanning electron microscopy (SEM) and infrared spectroscopy (IR). The thermal properties of the composite microspheres were studied via thermogravimetric analysis (TGA) and the magnetic properties were determined by a vibrating sample magnetometer (VSM). The saturation magnetization decreases gradually, increasing the polyimide content, with a saturation magnetization of PI/Fe3O4 determined to be 20.29 emu/g

Designing a Low-cost, Light-weight Electric Snowmobile

poster abstractThe Indiana University-Purdue University, Indianapolis Jaguar team is participating in the 2015 SAE Clean Snowmobile Challenge (CSC) with the aim of achieving a low-cost snowmobile that weigh less than 200 kg (441 lb). It was re-engineered to be an environmental friendly snowmobile without forgetting to improve the towing capability and less noise. We are using the Phantom Snowmobile PD250LT that is manufactured by FMC Motor Company and sold through out North America. The modified NetGain WarP 7 DC-series motor is connected directly to the sprocket shaft using a Goodyear Synchronous Belt. The belt itself is very quiet and can produce less than 59 dB

Additives synergy for stable interface formation on rechargeable lithium metal anodes

Abstract(#br)The attention towards lithium (Li) metal anodes has been rekindled in recent years as it would boost the energy-density of Li batteries. However, notorious safety issues and cycling instability severely hinder their commercialization, especially when cycled in traditional carbonic ester electrolytes that exhibit a wide voltage window and are compatible with most of the cathode materials. Herein, lithium difluorophosphate (LiDFP) and vinylene carbonate (VC) are combined, and demonstrated to be synergistic in constructing in situ a mechanically stable and highly Li-ion conducting surface film on the Li metal anode. This results in uniform and compact Li deposition largely suppressing the formation of Li dendrites, dead lithium and irreversible Li-species as revealed by operando neutron depth profiling (NDP). This enables long-term cycling stability and enhancement of the Coulombic efficiency for rechargeable Li metal anodes. By combining solid state nuclear magnetic resonance (SSNMR) and spectroscopic studies, it is demonstrated that VC slows down the LiDFP reduction, yet promoting the breaking of the P–F bonds, which leads to a protective film. This film is rich in LiF–Li 3 PO 4 inorganic compounds, distributed homogeneously, that is embedded in a matrix of P–O–C species and macromolecular organic compounds like lithium ethylene dicarbonate. This composition is responsible for the improved ionic conductivity and mechanical stability of the protective film during extended cycles. The detailed insight in the additives interaction provides new opportunities for the design of rational surface films necessary for realizing high-performance lithium metal batteries

Evolutionary trajectory of the replication mode of bacterial replicons

As typical bacterial replicons, circular chromosomes replicate bidirectionally and circular plasmids replicate either bidirectionally or unidirectionally. Whereas the finding of chromids (plasmid-derived chromosomes) in multiple bacterial lineages provides circumstantial evidence that chromosomes likely evolved from plasmids, all experimentally assayed chromids were shown to use bidirectional replication. Here, we employed a model system, the marine bacterial genus Pseudoalteromonas, members of which consistently carry a chromosome and a chromid. We provide experimental and bioinformatic evidence that while chromids in a few strains replicate bidirectionally, most replicate unidirectionally. This is the first experimental demonstration of the unidirectional replication mode in bacterial chromids. Phylogenomic and comparative genomic analyses showed that the bidirectional replication evolved only once from a unidirectional ancestor and that this transition was associated with insertions of exogenous DNA and relocation of the replication terminus region (ter2) from near the origin site (ori2) to a position roughly opposite it. This process enables a plasmid-derived chromosome to increase its size and expand the bacterium’s metabolic versatility while keeping its replication synchronized with that of the main chromosome. A major implication of our study is that the uni- and bidirectionally replicating chromids may represent two stages on the evolutionary trajectory from unidirectionally replicating plasmids to bidirectionally replicating chromosomes in bacteria. Further bioinformatic analyses predicted unidirectionally replicating chromids in several unrelated bacterial phyla, suggesting that evolution from unidirectionally to bidirectionally replicating replicons occurred multiple times in bacteria