FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization

Large language models (LLMs) have demonstrated state-of-the-art performance across various tasks. However, the latency of inference and the large GPU memory consumption of LLMs restrict their deployment performance. Recently, there have been some efficient attempts to quantize LLMs, yet inference with large batch size or long sequence still has the issue of being compute-bound. Fine-grained quantization methods have showcased their proficiency in achieving low-bit quantization for LLMs, while requiring FP16 data type for linear layer computations, which is time-consuming when dealing with large batch size or long sequence. In this paper, we introduce a method called FlattenQuant, which significantly reduces the maximum value of the tensor by flattening the large channels in the tensor, to achieve low bit per-tensor quantization with minimal accuracy loss. Our experiments show that FlattenQuant can directly use 4 bits to achieve 48.29% of the linear layer calculation in LLMs, with the remaining layers using 8 bits. The 4-bit matrix multiplication introduced in the FlattenQuant method can effectively address the compute-bound caused by large matrix calculation. Our work achieves up to 2$\times$ speedup and 2.3$\times$ memory reduction for LLMs with negligible loss in accuracy

Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent

Large Language Models (LLMs) have revolutionized open-domain dialogue agents but encounter challenges in multi-character role-playing (MCRP) scenarios. To address the issue, we present Neeko, an innovative framework designed for efficient multiple characters imitation. Unlike existing methods, Neeko employs a dynamic low-rank adapter (LoRA) strategy, enabling it to adapt seamlessly to diverse characters. Our framework breaks down the role-playing process into agent pre-training, multiple characters playing, and character incremental learning, effectively handling both seen and unseen roles. This dynamic approach, coupled with distinct LoRA blocks for each character, enhances Neeko's adaptability to unique attributes, personalities, and speaking patterns. As a result, Neeko demonstrates superior performance in MCRP over most existing methods, offering more engaging and versatile user interaction experiences. Code and data are available at https://github.com/weiyifan1023/Neeko

Feasibility of flattening filter free beams for hippocampal avoidance whole-brain radiotherapy: a dosimetric and radiobiological analysis

ObjectivesThe purpose of this study is to evaluate the potential of the flattening filter free (FFF) mode of a linear accelerator for patients with hippocampal avoidance whole-brain radiotherapy (HA-WBRT) by comparison with flattened beams (FF) technique in the application of volumetric modulated arc therapy (VMAT) and intensity modulated radiation therapy (IMRT) using dosimetric and radiobiological indexes based on the volume of hippocampus and target.Methods2 VMAT- and 2 IMRT- plans were optimized in Eclipse planning system with 2 different delivery modes (6 MV standard vs. 6 MV FFF) for each of 25 patients. Dose distributions of the target and organs at risk (OARs), normal tissue complication probability (NTCP) of the hippocampus, monitor units, treatment time and quality assurance results were evaluated to compare the normal and FFF beam characteristics by Wilcoxon matched-pair signed-rank test with a significance level of 0.05.ResultsVMAT-FFF provided the significantly best homogeneity and conformity of the target, delivered the lowest dose to hippocampus and the other OARs, and led to the lowest NTCP of the hippocampus among all modalities, which has the potential to alleviate neurocognitive decline after WBRT. IMRT-FFF reduced the dose to the lens with similar dose distributions of the target compared with IMRT-FF, whereas the lower dose to the hippocampus was achieved using the conventional beams. The monitor units were obviously increased by 19.2% for VMAT and 33.8% for IMRT, when FFF beams w ere used. The removal of flattening filter for IMRT resulted in a 26% reduction in treatment time, but VMAT had the similar treatment time for the two modes owing to the limitation of gantry rotation speed. Gamma analysis showed an excellent agreement for all plans at 3%/2 mm, and no statistical differences were found between FF and FFF.ConclusionIn conclusion, this study suggests that FFF mode is feasible and advantageous in HA-WBRT and VMAT-FFF is the optimal solution in terms of dose distribution of the target, OARs sparing, NTCP of the hippocampus and delivery efficiency compared to the other three techniques. Additionally, the advantages of the FFF technique for VMAT are more prominent in cases with small hippocampal volumes

Adaptive Tuning of Robotic Polishing Skills based on Force Feedback Model

Acquiring human skills offers an efficient approach to tackle complex task planning challenges. When performing a learned skill model for a continuous contact task, such as robot polishing in an uncertain environment, the robot needs to be able to adaptively modify the skill model to suit the environment and perform the desired task. The environmental perturbation of the polishing task is mainly reflected in the variation of contact force. Therefore, adjusting the task skill model by providing feedback on the contact force deviation is an effective way to meet the task requirements. In this study, a phase-modulated diagonal recurrent neural network (PMDRNN) is proposed for force feedback model learning in the robotic polishing task. The contact between the tool and the workpiece in the polishing task can be considered a dynamic system. In comparison to the existing feedforward neural network phase-modulated neural network (PMNN), PMDRNN combines the diagonal recurrent network structure with the phase-modulated neural network layer to improve the learning performance of the feedback model for dynamic systems. Specifically, data from real-world robot polishing experiments are used to learn the feedback model. PMDRNN demonstrates a significant reduction in the training error of the feedback model when compared to PMNN. Building upon this, the combination of PMDRNN and dynamic movement primitives (DMPs) can be used for real-time adjustment of skills for polishing tasks and effectively improve the robustness of the task skill model. Finally, real-world robotic polishing experiments are conducted to demonstrate the effectiveness of the approach.Comment: This paper has been accepted by The 2023 IEEE International Conference on Robotics and Biomimetics (IEEE ROBIO 2023

Exploring how workspace awareness cues affect distributed meeting outcome

Nowadays, using the online whiteboard to share knowledge in distributed meetings has become a common practice. Existing studies and practices have attempted to visualize attendees’ interactive activities in whiteboard tools to support the virtual team’s workspace awareness (WA). However, the impact of such visual cues on meeting success remains unclear. For this purpose, we primarily explore whether and to what extent WA cues are conducive to meeting outcome. This study applies activity theory to guide our prototype design and research analysis. A customized web-based whiteboard interface is implemented under two conditions. We conduct a study with 42 subjects in a distributed meeting scenario via a controlled experiment. Also, we analyze the system affordance via user experience. The results demonstrate that the benefits of WA cues to meeting outcome are especially embodied in goal attainment and quality of contributions, but not effectively supported in productivity and user satisfaction. Moreover, subjects report that they do not feel distracted by the system’s visual cues because they do not notice those cues most of the time and use them only when needed. Drawing upon findings from our trial work, we provide several implications for designing a collaborative knowledge-sharing environment to assist the visual support of WA in distributed meetings

Research progress on the role of lncRNA–miRNA networks in regulating adipogenic and osteogenic differentiation of bone marrow mesenchymal stem cells in osteoporosis

Osteoporosis (OP) is characterized by a decrease in osteoblasts and an increase in adipocytes in the bone marrow compartment, alongside abnormal bone/fat differentiation, which ultimately results in imbalanced bone homeostasis. Bone marrow mesenchymal stem cells (BMSCs) can differentiate into osteoblasts and adipocytes to maintain bone homeostasis. Several studies have shown that lncRNAs are competitive endogenous RNAs that form a lncRNA–miRNA network by targeting miRNA for the regulation of bone/fat differentiation in BMSCs; this mechanism is closely related to the corresponding treatment of OP and is important in the development of novel OP-targeted therapies. However, by reviewing the current literature, it became clear that there are limited summaries discussing the effects of the lncRNA–miRNA network on osteogenic/adipogenic differentiation in BMSCs. Therefore, this article provides a review of the current literature to explore the impact of the lncRNA–miRNA network on the osteogenic/adipogenic differentiation of BMSCs, with the aim of providing a new theoretical basis for the treatment of OP

Increased Vesicular Monoamine Transporter 2 (VMAT2) and Dopamine Transporter (DAT) Expression in Adolescent Brain Development: A Longitudinal Micro-PET/CT Study in Rodent

Background: Brain development and maturation in adolescence is a complex process with active changes of metabolic and neurotransmission pathways. Positron emission tomography (PET) is a useful imaging modality for tracking metabolic and functional changes in adolescent brain. In this study, changes of glucose metabolism, expression of vesicular monoamine transporter 2 and dopamine transporter during adolescent brain development in rats were investigated with PET/CT.Methods: A longitudinal PET/CT study of age-dependent changes of VMAT2, DAT and glucose metabolism in adolescent brain was conducted in a group of Wistar rats (n = 6) post sequential intravenous injection of 18F-PF-(+)-DTBZ, 11C-CFT, and 18F-FDG, respectively. PET acquisition was performed at 2, 4, 9, and 12 months of age. Radiotracer uptake in different brain regions, including the striatum, cerebellum, and hippocampus, were quantified and recorded as Standardized uptake value (SUV) and striatal specific uptake ratio (SUVR: SUV in brain regions/SUV in cerebellum).Results: Variable uptake of 18F-PF-(+)-DTBZ and 11C-CFT were detected, with highest level uptake in the striatum and accumbens. There was significant age-dependent increase of 18F-PF-(+)-DTBZ and 11C-CFT uptake in the striatum from 2 months of age (SUV: 1.36 ± 0.22, 1.37 ± 0.39, respectively), to 4 months (SUV: 2.22 ± 0.29, 2.04 ± 0.33), 9 months (1.98 ± 0.34, 2.09 ± 0.18), 12 months (SUV: 1.93 ± 0.19, 2.00 ± 0.17) of age, SUV of 18F-FDG also increased from 2 months of age to older ages (SUV in the striatum: 3.71 ± 0.78 at 2 month, 5.28 ± 0.81, 5.14 ± 0.73, 4.94 ± 0.50 at 4, 9, 12 month, respectively).Conclusion: Age-dependent increases of striatal of 18F-FDG, 18F-PF-(+)-DTBZ, and 11C-CFT uptake were detected in rats from 2 to 4 month of age, demonstrating striatal development presents over the first 4 months of age. Four months of age can be considered a safe threshold to launch brain disease studies for exclusion of confusion of continuing tissue development. These findings support further investigation of age-dependent changes in expression of DAT, VMAT2, and glucose metabolism for their potential use as a new imaging biomarker for study of brain development and functional maturation

A PLSPM-Based Test Statistic for Detecting Gene-Gene Co-Association in Genome-Wide Association Study with Case-Control Design

For genome-wide association data analysis, two genes in any pathway, two SNPs in the two linked gene regions respectively or in the two linked exons respectively within one gene are often correlated with each other. We therefore proposed the concept of gene-gene co-association, which refers to the effects not only due to the traditional interaction under nearly independent condition but the correlation between two genes. Furthermore, we constructed a novel statistic for detecting gene-gene co-association based on Partial Least Squares Path Modeling (PLSPM). Through simulation, the relationship between traditional interaction and co-association was highlighted under three different types of co-association. Both simulation and real data analysis demonstrated that the proposed PLSPM-based statistic has better performance than single SNP-based logistic model, PCA-based logistic model, and other gene-based methods

Integrative transcriptomic profiling of mRNA, miRNA, circRNA, and lncRNA in alveolar macrophages isolated from PRRSV-infected porcine

IntroductionThe porcine reproductive and respiratory syndrome virus (PRRSV) continues to pose a significant threat to the global swine industry, attributed largely to its immunosuppressive properties and the chronic nature of its infection. The absence of effective vaccines and therapeutics amplifies the urgency to deepen our comprehension of PRRSV’s intricate pathogenic mechanisms. Previous transcriptomic studies, although informative, are partially constrained by their predominant reliance on in vitro models or lack of long-term infections. Moreover, the role of circular RNAs (circRNAs) during PRRSV invasion is yet to be elucidated.MethodsIn this study, we employed an in vivo approach, exposing piglets to a PRRSV challenge over varied durations of 3, 7, or 21 days. Subsequently, porcine alveolar macrophages were isolated for a comprehensive transcriptomic investigation, examining the expression patterns of mRNAs, miRNAs, circRNAs, and long non-coding RNAs (lncRNAs).ResultsDifferentially expressed RNAs from all four categories were identified, underscoring the dynamic interplay among these RNA species during PRRSV infection. Functional enrichment analyses indicate that these differentially expressed RNAs, as well as their target genes, play a pivotal role in immune related pathways. For the first time, we integrated circRNAs into the lncRNA-miRNA-mRNA relationship, constructing a competitive endogenous RNA (ceRNA) network. Our findings highlight the immune-related genes, CTLA4 and SAMHD1, as well as their associated miRNAs, lncRNAs, and circRNAs, suggesting potential therapeutic targets for PRRS. Importantly, we corroborated the expression patterns of selected RNAs through RT-qPCR, ensuring consistency with our transcriptomic sequencing data.DiscussionThis study sheds lights on the intricate RNA interplay during PRRSV infection and provides a solid foundation for future therapeutic strategizing