Bitformer: An efficient Transformer with bitwise operation-based attention for Big Data Analytics at low-cost low-precision devices

In the current landscape of large models, the Transformer stands as a cornerstone, playing a pivotal role in shaping the trajectory of modern models. However, its application encounters challenges attributed to the substantial computational intricacies intrinsic to its attention mechanism. Moreover, its reliance on high-precision floating-point operations presents specific hurdles, particularly evident in computation-intensive scenarios such as edge computing environments. These environments, characterized by resource-constrained devices and a preference for lower precision, necessitate innovative solutions. To tackle the exacting data processing demands posed by edge devices, we introduce the Bitformer model, an inventive extension of the Transformer paradigm. Central to this innovation is a novel attention mechanism that adeptly replaces conventional floating-point matrix multiplication with bitwise operations. This strategic substitution yields dual advantages. Not only does it maintain the attention mechanism's prowess in capturing intricate long-range information dependencies, but it also orchestrates a profound reduction in the computational complexity inherent in the attention operation. The transition from an $O(n^2d)$ complexity, typical of floating-point operations, to an $O(n^2T)$ complexity characterizing bitwise operations, substantiates this advantage. Notably, in this context, the parameter $T$ remains markedly smaller than the conventional dimensionality parameter $d$. The Bitformer model in essence endeavors to reconcile the indomitable requirements of modern computing landscapes with the constraints posed by edge computing scenarios. By forging this innovative path, we bridge the gap between high-performing models and resource-scarce environments, thus unveiling a promising trajectory for further advancements in the field

Storyfier: Exploring Vocabulary Learning Support with Text Generation Models

Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for any target words of learners' interests, and they seldom help practice word usage. In this paper, we work with teachers and students to iteratively develop Storyfier, which leverages text generation models to enable learners to read a generated story that covers any target words, conduct a story cloze test, and use these words to write a new story with adaptive AI assistance. Our within-subjects study (N=28) shows that learners generally favor the generated stories for connecting target words and writing assistance for easing their learning workload. However, in the read-cloze-write learning sessions, participants using Storyfier perform worse in recalling and using target words than learning with a baseline tool without our AI features. We discuss insights into supporting learning tasks with generative models.Comment: To appear at the 2023 ACM Symposium on User Interface Software and Technology (UIST); 16 pages (7 figures, 23 tables

Controllable thioester-based hydrogen sulfide slow-releasing donors as cardioprotective agents

Hydrogen sulfide (H2S) is an important signaling molecule with promising protective effects in many physiological and pathological processes. However, the study of H2S has been impeded by the lack of appropriate H2S donors that could mimic its slow-releasing process in vivo. Herein, we report the rational design, synthesis, and biological evaluation of a series of thioester-based H2S donors. These cysteine-activated H2S donors release H2S in a slow and controllable manner. Most of the donors comprising an allyl moiety showed significant cytoprotective effects in H9c2 cellular models of oxidative damage. The most potent donor 5e decreased the mitochondrial membrane potential (MMP) loss and lactate dehydrogenase (LDH) release in H2O2-stimulated H9c2 cells. More importantly, donor 5e exhibited a potent cardioprotective effect in an in vivo myocardial infarction (MI) mouse model by reducing myocardial infarct size and cardiomyocyte apoptosis. Taken together, our studies demonstrated that these new allyl thioesters are potential cardioprotective agents by releasing H2S

The structural modification of natural products for novel drug discovery

Introduction: Throughout history, natural products (NPs) have provided a rich source of compounds that have wide applications in the fields of medicine, health sciences, pharmacy and biology. Although naturally active substances are good lead compounds for the discovery of new drugs, most of them suffer from various deficiencies or shortcomings, such as complex structures, poor stability and solubility. Therefore, structural modification of NPs is needed to develop novel compounds with specific properties. Areas covered: This article presents an overview on the structural modifications of NPs in drug development. The application of multiple classes of NPs to the treatment of conditions such as cancers, infection, Alzheimer’s and diabetes are discussed. This article also reveals that modification of NPs is a versatile approach to explore their mode of actions, which may lead to the discovery of novel drugs. Expert opinion: NPs are usually described by structural diversity and complexity. The use of isolated NPs as scaffolds for modification is a good approach to drug discovery and development. Despite many limitations associated with NPs, the total synthesis, semisynthetic modification, SAR-based modification, sometimes even a single atom alteration, may lead to the discovery of a novel drug

A missense mutation in Pitx2 leads to early-onset glaucoma via NRF2-YAP1 axis.

Glaucoma is a leading cause of blindness, affecting 70 million people worldwide. Owing to the similarity in anatomy and physiology between human and mouse eyes and the ability to genetically manipulate mice, mouse models are an invaluable resource for studying mechanisms underlying disease phenotypes and for developing therapeutic strategies. Here, we report the discovery of a new mouse model of early-onset glaucoma that bears a transversion substitution c. G344T, which results in a missense mutation, p. R115L in PITX2. The mutation causes an elevation in intraocular pressure (IOP) and progressive death of retinal ganglion cells (RGC). These ocular phenotypes recapitulate features of pathologies observed in human glaucoma. Increased oxidative stress was evident in the inner retina. We demonstrate that the mutant PITX2 protein was not capable of binding to Nuclear factor-like 2 (NRF2), which regulates Pitx2 expression and nuclear localization, and to YAP1, which is necessary for co-initiation of transcription of downstream targets. PITX2-mediated transcription of several antioxidant genes were also impaired. Treatment with N-Acetyl-L-cysteine exerted a profound neuroprotective effect on glaucoma-associated neuropathies, presumably through inhibition of oxidative stress. Our study demonstrates that a disruption of PITX2 leads to glaucoma optic pathogenesis and provides a novel early-onset glaucoma model that will enable elucidation of mechanisms underlying the disease as well as to serve as a resource to test new therapeutic strategies

Recent Advances of Cell Membrane Coated Nanoparticles in Treating Cardiovascular Disorders

Cardiovascular diseases (CVDs) are the leading cause of death worldwide, causing approximately 17.9 million deaths annually, an estimated 31% of all deaths, according to the WHO. CVDs are essentially rooted in atherosclerosis and are clinically classified into coronary heart disease, stroke and peripheral vascular disorders. Current clinical interventions include early diagnosis, the insertion of stents, and long-term preventive therapy. However, clinical diagnostic and therapeutic tools are subject to a number of limitations including, but not limited to, potential toxicity induced by contrast agents and unexpected bleeding caused by anti-platelet drugs. Nanomedicine has achieved great advancements in biomedical area. Among them, cell membrane coated nanoparticles, denoted as CMCNPs, have acquired enormous expectations due to their biomimetic properties. Such membrane coating technology not only helps avoid immune clearance, but also endows nanoparticles with diverse cellular and functional mimicry. In this review, we will describe the superiorities of CMCNPs in treating cardiovascular diseases and their potentials in optimizing current clinical managements

An Experimental Investigation of Calibration Techniques for Imbalanced Data

Calibration is a technique used to obtain accurate probability estimation for classification problems in real applications. Class imbalance can create considerable challenges in obtaining accurate probabilities for calibration methods. However, previous research has paid little attention to this issue. In this paper, we present an experimental investigation of some prevailing calibration methods in different imbalance scenarios. Several performance metrics are considered to evaluate different aspects of calibration performance. The experimental results show that the performance of different calibration techniques depends on the metrics and the degree of the imbalance ratio. Isotonic Regression has better overall performance on imbalanced datasets than parametric and other complex non-parametric methods. However, it performs unstably in highly imbalanced scenarios. This study provides some insights into calibration methods on imbalanced datasets, and it can be a reference for the future development of calibration methods in class imbalance scenarios.status: Published onlin