Qualifying Chinese Medical Licensing Examination with Knowledge Enhanced Generative Pre-training Model

Generative Pre-Training (GPT) models like ChatGPT have demonstrated exceptional performance in various Natural Language Processing (NLP) tasks. Although ChatGPT has been integrated into the overall workflow to boost efficiency in many domains, the lack of flexibility in the finetuning process hinders its applications in areas that demand extensive domain expertise and semantic knowledge, such as healthcare. In this paper, we evaluate ChatGPT on the China National Medical Licensing Examination (CNMLE) and propose a novel approach to improve ChatGPT from two perspectives: integrating medical domain knowledge and enabling few-shot learning. By using a simple but effective retrieval method, medical background knowledge is extracted as semantic instructions to guide the inference of ChatGPT. Similarly, relevant medical questions are identified and fed as demonstrations to ChatGPT. Experimental results show that directly applying ChatGPT fails to qualify the CNMLE at a score of 51 (i.e., only 51\% of questions are answered correctly). While our knowledge-enhanced model achieves a high score of 70 on CNMLE-2022 which not only passes the qualification but also surpasses the average score of humans (61). This research demonstrates the potential of knowledge-enhanced ChatGPT to serve as versatile medical assistants, capable of analyzing real-world medical problems in a more accessible, user-friendly, and adaptable manner

Diversity Order Analysis for Quantized Constant Envelope Transmission

Quantized constant envelope (QCE) transmission is a popular and effective technique to reduce the hardware cost and improve the power efficiency of 5G and beyond systems equipped with large antenna arrays. It has been widely observed that the number of quantization levels has a substantial impact on the system performance. This paper aims to quantify the impact of the number of quantization levels on the system performance. Specifically, we consider a downlink single-user multiple-input-single-output (MISO) system with M-phase shift keying (PSK) constellation under the Rayleigh fading channel. We first derive a novel bound on the system symbol error probability (SEP). Based on the derived SEP bound, we characterize the achievable diversity order of the quantized matched filter (MF) precoding strategy. Our results show that full diversity order can be achieved when the number of quantization levels L is greater than the PSK constellation order M, i.e., L>M, only half diversity order is achievable when L=M, and the achievable diversity order is 0 when L<M. Simulation results verify our theoretical analysis.Comment: 9 pages, 3 figures, submitted for possible publicatio

Globally Optimal Beamforming Design for Integrated Sensing and Communication Systems

In this paper, we propose a multi-input multi-output (MIMO) beamforming transmit optimization model for joint radar sensing and multi-user communications, where the design of the beamformers is formulated as an optimization problem whose objective is a weighted combination of the sum rate and the Cram\'{e}r-Rao bound (CRB), subject to the transmit power budget constraint. The formulated problem is challenging to obtain a global solution, because the sum rate maximization (SRM) problem itself (even without considering the sensing metric) is known to be NP-hard. In this paper, we propose an efficient global branch-and-bound algorithm for solving the formulated problem based on the McCormick envelope relaxation and the semidefinite relaxation (SDR) technique. The proposed algorithm is guaranteed to find the global solution for the considered problem, and thus serves as an important benchmark for performance evaluation of the existing local or suboptimal algorithms for solving the same problem.Comment: 5 pages, 2 figures, submitted for possible publicatio

YATO: Yet Another deep learning based Text analysis Open toolkit

We introduce YATO, an open-source, easy-to-use toolkit for text analysis with deep learning. Different from existing heavily engineered toolkits and platforms, YATO is lightweight and user-friendly for researchers from cross-disciplinary areas. Designed in a hierarchical structure, YATO supports free combinations of three types of widely used features including 1) traditional neural networks (CNN, RNN, etc.); 2) pre-trained language models (BERT, RoBERTa, ELECTRA, etc.); and 3) user-customized neural features via a simple configurable file. Benefiting from the advantages of flexibility and ease of use, YATO can facilitate fast reproduction and refinement of state-of-the-art NLP models, and promote the cross-disciplinary applications of NLP techniques. The code, examples, and documentation are publicly available at https://github.com/jiesutd/YATO. A demo video is also available at https://youtu.be/tSjjf5BzfQg

Shadow Datasets, New challenging datasets for Causal Representation Learning

Discovering causal relations among semantic factors is an emergent topic in representation learning. Most causal representation learning (CRL) methods are fully supervised, which is impractical due to costly labeling. To resolve this restriction, weakly supervised CRL methods were introduced. To evaluate CRL performance, four existing datasets, Pendulum, Flow, CelebA(BEARD) and CelebA(SMILE), are utilized. However, existing CRL datasets are limited to simple graphs with few generative factors. Thus we propose two new datasets with a larger number of diverse generative factors and more sophisticated causal graphs. In addition, current real datasets, CelebA(BEARD) and CelebA(SMILE), the originally proposed causal graphs are not aligned with the dataset distributions. Thus, we propose modifications to them

Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling

Large language models (LLMs) and vision language models (VLMs) demonstrate excellent performance on a wide range of tasks by scaling up parameter counts from O(10^9) to O(10^{12}) levels and further beyond. These large scales make it impossible to adapt and deploy fully specialized models given a task of interest. Parameter-efficient fine-tuning (PEFT) emerges as a promising direction to tackle the adaptation and serving challenges for such large models. We categorize PEFT techniques into two types: intrusive and non-intrusive. Intrusive PEFT techniques directly change a model's internal architecture. Though more flexible, they introduce significant complexities for training and serving. Non-intrusive PEFT techniques leave the internal architecture unchanged and only adapt model-external parameters, such as embeddings for input. In this work, we describe AdaLink as a non-intrusive PEFT technique that achieves competitive performance compared to SoTA intrusive PEFT (LoRA) and full model fine-tuning (FT) on various tasks. We evaluate using both text-only and multimodal tasks, with experiments that account for both parameter-count scaling and training regime (with and without instruction tuning)

virtual machine replay update: improved implementation for modern hardware architecture

This paper describes a successive and updated work of Revirt project which presents a virtual machine replay framework on Xen hypervisor. As both the commodity hardware and Xen hypervisor have been changed significantly since the first publication of Revirt, the initial implementation does not meet the needs of modern architecture any more. This paper presents an improved implementation of virtual machine execution replay system called CASMotion. CASMotion has three contributions. First, CASMotion uses the performance monitor of Intel Core2 processor to construct time point of recorded events, which makes the event record more complete and precise. Second, CASMotion can fully support multi-core hardware platform which is prevalent today. Third, CASMotion is developed with more general architecture design, which makes it deployable on upstream Xen hypervisor and Dom0. Our experiments under a varity of workloads shows CASMotion has low performance impact on monitored DomU. The growth of record log is also in acceptable range. Index Terms-Revirt, execution replay, determinism, virtual machines, Xen. &copy; 2012 IEEE.IEEE Reliability SocietyThis paper describes a successive and updated work of Revirt project which presents a virtual machine replay framework on Xen hypervisor. As both the commodity hardware and Xen hypervisor have been changed significantly since the first publication of Revirt, the initial implementation does not meet the needs of modern architecture any more. This paper presents an improved implementation of virtual machine execution replay system called CASMotion. CASMotion has three contributions. First, CASMotion uses the performance monitor of Intel Core2 processor to construct time point of recorded events, which makes the event record more complete and precise. Second, CASMotion can fully support multi-core hardware platform which is prevalent today. Third, CASMotion is developed with more general architecture design, which makes it deployable on upstream Xen hypervisor and Dom0. Our experiments under a varity of workloads shows CASMotion has low performance impact on monitored DomU. The growth of record log is also in acceptable range. Index Terms-Revirt, execution replay, determinism, virtual machines, Xen. &copy; 2012 IEEE

Systemic threats to hypervisor non-control data

Hypervisors are becoming a widespread virtualisation layer in current computer systems. Recent successful attacks against hypervisors indicate that they face the similar integrity threats as traditional operating systems. Current approaches that secure hypervisors mainly focus on code or control-data integrity, without paying attention to non-control data integrity. In this study the authors construct attacks that target hypervisor non-control data to demonstrate which types of data within the Xen hypervisor are critical to system security. It shows privilege, resource utilisation and security policy related data are vulnerable to return-oriented programming or DMA attacks. By modifying their values from one to another, the whole system's performance will be affected. By discussing current approaches that secure hypervisors, which are not suitable for non-control data, the work is to motivate new innovation in this area to protect them.Hypervisors are becoming a widespread virtualisation layer in current computer systems. Recent successful attacks against hypervisors indicate that they face the similar integrity threats as traditional operating systems. Current approaches that secure hypervisors mainly focus on code or control-data integrity, without paying attention to non-control data integrity. In this study the authors construct attacks that target hypervisor non-control data to demonstrate which types of data within the Xen hypervisor are critical to system security. It shows privilege, resource utilisation and security policy related data are vulnerable to return-oriented programming or DMA attacks. By modifying their values from one to another, the whole system's performance will be affected. By discussing current approaches that secure hypervisors, which are not suitable for non-control data, the work is to motivate new innovation in this area to protect them

deterministic replay of multithread applications using virtual machine

The deterministic replay technology usually is used to reproduce non-deterministic failures, especially the concurrency failures which are hard to debug with cyclic-debugging techniques. Previous techniques either incur large overhead or require custom hardware extensions. They have also suffered from the interference with irrelevant processes. This paper introduces WDRVirt, a new software based replay system that provides deterministic replay for concurrent applications. This paper makes three contributions. First, WDRVirt enforces a deterministic replay for the interleaving of lock acquisitions with low performance overhead. Second, different from the previous works, we customize the lightweight virtual machine execution environment as the container for the target program. WDRVirt replays the container to avoid the interference with other irrelevant processes. At last, WDRVirt is deployed into both virtual machine(VM) and virtual machine manager(VMM) to deal with different types of non-deterministic event. We have implemented this system based on the Xen virtualization platform. Our experiments with real-world benchmarks demonstrate the effectiveness of WDRVirt. &copy; 2012 IEEE.IEEE Comput. Soc. Tech. Comm. Distrib. Process.The deterministic replay technology usually is used to reproduce non-deterministic failures, especially the concurrency failures which are hard to debug with cyclic-debugging techniques. Previous techniques either incur large overhead or require custom hardware extensions. They have also suffered from the interference with irrelevant processes. This paper introduces WDRVirt, a new software based replay system that provides deterministic replay for concurrent applications. This paper makes three contributions. First, WDRVirt enforces a deterministic replay for the interleaving of lock acquisitions with low performance overhead. Second, different from the previous works, we customize the lightweight virtual machine execution environment as the container for the target program. WDRVirt replays the container to avoid the interference with other irrelevant processes. At last, WDRVirt is deployed into both virtual machine(VM) and virtual machine manager(VMM) to deal with different types of non-deterministic event. We have implemented this system based on the Xen virtualization platform. Our experiments with real-world benchmarks demonstrate the effectiveness of WDRVirt. &copy; 2012 IEEE