GLM-130B: An Open Bilingual Pre-trained Model

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good as GPT-3 (davinci) and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we face numerous unexpected technical and engineering challenges, particularly on loss spikes and divergence. In this paper, we introduce the training process of GLM-130B including its design choices, training strategies for both efficiency and stability, and engineering efforts. The resultant GLM-130B model offers significant outperformance over GPT-3 175B (davinci) on a wide range of popular English benchmarks while the performance advantage is not observed in OPT-175B and BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN 3.0 260B -- the largest Chinese language model -- across related benchmarks. Finally, we leverage a unique scaling property of GLM-130B to reach INT4 quantization without post training, with almost no performance loss, making it the first among 100B-scale models and more importantly, allowing its effective inference on 4$\times$RTX 3090 (24G) or 8$\times$RTX 2080 Ti (11G) GPUs, the most affordable GPUs required for using 100B-scale models. The GLM-130B model weights are publicly accessible and its code, training logs, related toolkit, and lessons learned are open-sourced at \url{https://github.com/THUDM/GLM-130B/}.Comment: Accepted to ICLR 202

Orbital floor repair using patient specific osteoinductive implant made by stereolithography

The orbital floor (OF) is an anatomical location in the craniomaxillofacial (CMF) region known to be highly variable in shape and size. When fractured, implants commonly consisting of titanium meshes are customized by plying and crude hand-shaping. Nevertheless, more precise customized synthetic grafts are needed to meticulously reconstruct the patients’ OF anatomy with better fidelity. As alternative to titanium mesh implants dedicated to OF repair, we propose a flexible patient-specific implant (PSI) made by stereolithography (SLA), offering a high degree of control over its geometry and architecture. The PSI is made of biodegradable poly(trimethylene carbonate) (PTMC) loaded with 40 wt % of hydroxyapatite (called Osteo-PTMC). In this work, we developed a complete work-flow for the additive manufacturing of PSIs to be used to repair the fractured OF, which is clinically relevant for individualized medicine. This work-flow consists of (i) the surgical planning, (ii) the design of virtual PSIs and (iii) their fabrication by SLA, (iv) the monitoring and (v) the biological evaluation in a preclinical large-animal model. We have found that once implanted, titanium meshes resulted in fibrous tissue encapsulation, whereas Osteo-PMTC resulted in rapid neovascularization and bone morphogenesis, both ectopically and in the OF region, and without the need of additional biotherapeutics such as bone morphogenic proteins. Our study supports the hypothesis that the composite osteoinductive Osteo-PTMC brings advantages compared to standard titanium mesh, by stimulating bone neoformation in the OF defects. PSIs made of Osteo-PTMC represent a significant advancement for patients whereby the anatomical characteristics of the OF defect restrict the utilization of traditional hand-shaped titanium mesh

AgentBench: Evaluating LLMs as Agents

Large Language Models (LLMs) are becoming increasingly smart and autonomous, targeting real-world pragmatic missions beyond traditional NLP tasks. As a result, there has been an urgent need to evaluate LLMs as agents on challenging tasks in interactive environments. We present AgentBench, a multi-dimensional evolving benchmark that currently consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities in a multi-turn open-ended generation setting. Our extensive test over 27 API-based and open-sourced (OSS) LLMs shows that, while top commercial LLMs present a strong ability of acting as agents in complex environments, there is a significant disparity in performance between them and OSS competitors. We identify the typical reasons of failures in environments and LLMs, showing that poor long-term reasoning, decision-making, and instruction following abilities are the main obstacles for developing usable LLM agents. Training on code and high quality multi-turn alignment data could improve agent performance. Datasets, environments, and an integrated evaluation package for AgentBench are released at \url{https://github.com/THUDM/AgentBench}.Comment: 55 page

Review of various treatment options and potential therapies for osteonecrosis of the femoral head

Size and location of the lesion, subchondral collapse occurrence, and articular cartilage involvement are general disease progression criteria for direct osteonecrosis of the femoral head (ONFH) classifications. Treatment options for ONFH are usually based on individual factors and lesion characteristics. Although spontaneous repair of ONFH occurs in some cases, untreated ONFH is unlikely to escape the fate of subchondral collapse and usually ends up with total hip arthroplasty. Operations to preserve the femoral head, e.g., core decompression and bone grafting, are usually recommended in younger patients. They are helpful to relieve pain and improve function in the affected femoral head without subchondral collapse, however, poor prognosis after surgical procedures remains the major problem for ONFH. Pharmacological and physical therapies only work in the early stage of ONFH and have also been recommended as a supplement or prevention treatment for osteonecrosis. Following advances in basic science, many new insights focus on bone tissue engineering to optimize therapies and facilitate prognosis of ONFH. In this review, disease classifications, current treatment options, potential therapies, and the relevant translational barriers are reviewed in the context of clinical application and preclinical exploration, which would provide guidance for preferable treatment options and translation into novel therapies

Advancements in SiC-Reinforced Metal Matrix Composites for High-Performance Electronic Packaging: A Review of Thermo-Mechanical Properties and Future Trends

With the advancement of semiconductor technology, chip cooling has become a major obstacle to enhancing the capabilities of power electronic systems. Traditional electronic packaging materials are no longer able to meet the heat dissipation requirements of high-performance chips. High thermal conductivity (TC), low coefficient of thermal expansion (CTE), good mechanical properties, and a rich foundation in microfabrication techniques are the fundamental requirements for the next generation of electronic packaging materials. Currently, metal matrix composites (MMCs) composed of high TC matrix metals and reinforcing phase materials have become the mainstream direction for the development and application of high-performance packaging materials. Silicon carbide (SiC) is the optimal choice for the reinforcing phase due to its high TC, low CTE, and high hardness. This paper reviews the research status of SiC-reinforced aluminum (Al) and copper (Cu) electronic packaging materials, along with the factors influencing their thermo-mechanical properties and improvement measures. Finally, the current research status and limitations of conventional manufacturing methods for SiC-reinforced MMCs are summarized, and an outlook on the future development trends of electronic packaging materials is provided

Recent Advances of Microneedles and Their Application in Disease Treatment

For decades, scientists have been doing a lot of research and exploration to find effective long-term analgesic and/or disease-modifying treatments. Microneedles (MNs) are a simple, effective, and painless transdermal drug delivery technology that has emerged in recent years, and exhibits great promise for realizing intelligent drug delivery. With the development of materials science and fabrication technology, the MN transdermal drug delivery technology has been applied and popularized in more and more fields, including chronic illnesses such as arthritis or diabetes, cancer, dermatocosmetology, family planning, and epidemic disease prevention, and has made fruitful achievements. This paper mainly reviews the latest research status of MNs and their fabrication methodology, and summarizes the application of MNs in the treatment of various diseases, as well as the potential to use nanotechnology to develop more intelligent MNs-based drug delivery systems

LL-37-Coupled Porous Composite Scaffold for the Treatment of Infected Segmental Bone Defect

Increased multiantibiotic-resistant bacteria means that infected bone defects remain a significant challenge to clinics. Great interest has emerged in the use of non-antibiotic antimicrobials to reduce the rate of multiantibiotic-resistant bacterial infection and facilitate bone regeneration. The cationic antimicrobial peptide LL-37 is the sole human cathelicidin and has shown nonspecific activity against a broad spectrum of microorganisms. In this study, we fabricated the poly(lactic-co-glycolic acid)/β-calcium phosphate/peptide LL-37 (PLGA/TCP/LL-37, PTL) scaffold with low-temperature 3D-printing technology for the treatment of infected segmental bone defects. The prepared scaffolds were divided into three groups: a high LL-37 concentration group (PTHL), low LL-37 concentration group (PTLL) and blank control group (PT). The cytocompatibility and antimicrobial activity of the engineered scaffolds were tested in vitro, and their osteogenesis properties were assessed in vivo in a rat infected bone defect model. We found the fabricated PTL scaffold had a well-designed porous structure that could support a steady and prolonged LL-37 release. Furthermore, the PTHL group showed strong antibacterial activity against Staphylococcus aureus (S. aureus) and Escherichia coli (E. coli) without any inhibition of the proliferation or alkaline phosphatase activity of rat bone marrow mesenchymal stem cells (BMSCs) in vitro. In addition, the infected femoral defects implanted with PTHL group displayed new bone formation in four weeks without any evidence of residual bacteria, which showed similar antibacterial outcomes to the vancomycin and cancellous bone mixture group. In conclusion, the PTHL composite scaffold is a promising non-antibiotic antimicrobial graft with good biodegradability, biocompatibility, and osteogenic capability for infected bone defects

Quantitative determination of residual 1,4-dioxane in three-dimensional printed bone scaffold

Summary: Background/Objective: A novel porous scaffold poly (lactide-co-glycolide) and tricalcium phosphate (PLGA/TCP) was developed by three-dimensional printing technology for bone defect repair. As a Class 2 solvent with less severe toxicity, content of residual 1,4-dioxane in this newly developed scaffold should be rigorously controlled when it is translated to clinical use. In this study, a headspace gas chromatography-mass spectrometric (HS-GC-MS) method and related testing protocol were developed for quantitative determination of 1,4-dioxane in the PLGA/TCP composite scaffolds. Methods: Matrix effect analysis was used to optimise the pretreatment method of the scaffolds. Then, the procedure for testing 1,4-dioxane using HS-GC-MS was set up. The accuracy, precision, and robustness of this newly developed quantitative method were also validated before quantification of 1,4-dioxane in the scaffolds with different drying procedures. Results: Dimethyl formamide (DMF) was the optimal solvent for dissolving scaffolds for GC-MS with proper sensitivity and without matrix effect. Then, the optimised procedure was determined as: the scaffolds were dissolved in DMF and kept at 90°C for 40 minutes, separated on a HP-5MS column, and detected by mass spectroscopy. Recovery experiments gave 97.9–100.7% recovery for 1,4-dioxane. The linear range for 1,4-dioxane was determined as 1–40 ppm with linear correlation coefficient ≥ 0.9999. Intraday and interday precision was determined as being within relative standard deviation of below 0.68%. The passable drying procedure was related to lyophilising (−50°C, 50 Pa) the scaffolds for 2 days and drying in vacuum (50 Pa) for 7 days. Conclusion: This is the first quantitative method established to test 1,4-dixoane in a novel scaffold. This method was validated with good accuracy and reproducibility, and met the methodological requirements of the Guideline 9101 documented in the Chinese Pharmacopoeia 2015 Edition. The translational potential of this article: This quantitative method for determination of residual 1,4-dioxane in the novel scaffolds is a key technical method during its translation into clinical use because this method is an important and indispensable file in the enterprise standard when the porous scaffold is registered as a Class III implanted medical device for bone defect repair, which is used to guarantee the safety of the scaffolds. It is also applied to optimise the drying process of scaffolds and to monitor the quality of scaffolds in the industrialisation process. Further, this method provides references for other solvents quantitative determination in porous scaffolds or materials. Keywords: headspace gas chromatography mass spectrometry, three-dimensional printing porous scaffold, 1,4-dioxan