Quantifying Discourse Support for Omitted Pronouns

Pro-drop is commonly seen in many languages, but its discourse motivations have not been well characterized. Inspired by the topic chain theory in Chinese, this study shows how character-verb usage continuity distinguishes dropped pronouns from overt references to story characters. We model the choice to drop vs. not drop as a function of character-verb continuity. The results show that omitted subjects have higher character history-current verb continuity salience than non-omitted subjects. This is consistent with the idea that discourse coherence with a particular topic, such as a story character, indeed facilitates the omission of pronouns in languages and contexts where they are optional.Comment: to be published in Proceedings of the Fifth Workshop on Computational Models of Reference, Anaphora and Coreference, 202

Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attention

Recent large language models (LLMs) have revealed strong abilities to understand natural language. Since most of them share the same basic structure, i.e. the transformer block, possible contributors to their success in the training process are scaling and instruction tuning. However, how these factors affect the models' language perception is unclear. This work compares the self-attention of several existing LLMs (LLaMA, Alpaca and Vicuna) in different sizes (7B, 13B, 30B, 65B), together with eye saccade, an aspect of human reading attention, to assess the effect of scaling and instruction tuning on language perception. Results show that scaling enhances the human resemblance and improves the effective attention by reducing the trivial pattern reliance, while instruction tuning does not. However, instruction tuning significantly enhances the models' sensitivity to instructions. We also find that current LLMs are consistently closer to non-native than native speakers in attention, suggesting a sub-optimal language perception of all models. Our code and data used in the analysis is available on GitHub

Telepath: Understanding Users from a Human Vision Perspective in Large-Scale Recommender Systems

Designing an e-commerce recommender system that serves hundreds of millions of active users is a daunting challenge. From a human vision perspective, there're two key factors that affect users' behaviors: items' attractiveness and their matching degree with users' interests. This paper proposes Telepath, a vision-based bionic recommender system model, which understands users from such perspective. Telepath is a combination of a convolutional neural network (CNN), a recurrent neural network (RNN) and deep neural networks (DNNs). Its CNN subnetwork simulates the human vision system to extract key visual signals of items' attractiveness and generate corresponding activations. Its RNN and DNN subnetworks simulate cerebral cortex to understand users' interest based on the activations generated from browsed items. In practice, the Telepath model has been launched to JD's recommender system and advertising system. For one of the major item recommendation blocks on the JD app, click-through rate (CTR), gross merchandise value (GMV) and orders have increased 1.59%, 8.16% and 8.71% respectively. For several major ads publishers of JD demand-side platform, CTR, GMV and return on investment have increased 6.58%, 61.72% and 65.57% respectively by the first launch, and further increased 2.95%, 41.75% and 41.37% respectively by the second launch.Comment: 8 pages, 11 figures, 1 tabl

ACQ: Improving Generative Data-free Quantization Via Attention Correction

Data-free quantization aims to achieve model quantization without accessing any authentic sample. It is significant in an application-oriented context involving data privacy. Converting noise vectors into synthetic samples through a generator is a popular data-free quantization method, which is called generative data-free quantization. However, there is a difference in attention between synthetic samples and authentic samples. This is always ignored and restricts the quantization performance. First, since synthetic samples of the same class are prone to have homogenous attention, the quantized network can only learn limited modes of attention. Second, synthetic samples in eval mode and training mode exhibit different attention. Hence, the batch-normalization statistics matching tends to be inaccurate. ACQ is proposed in this paper to fix the attention of synthetic samples. An attention center position-condition generator is established regarding the homogenization of intra-class attention. Restricted by the attention center matching loss, the attention center position is treated as the generator's condition input to guide synthetic samples in obtaining diverse attention. Moreover, we design adversarial loss of paired synthetic samples under the same condition to prevent the generator from paying overmuch attention to the condition, which may result in mode collapse. To improve the attention similarity of synthetic samples in different network modes, we introduce a consistency penalty to guarantee accurate BN statistics matching. The experimental results demonstrate that ACQ effectively improves the attention problems of synthetic samples. Under various training settings, ACQ achieves the best quantization performance. For the 4-bit quantization of Resnet18 and Resnet50, ACQ reaches 67.55% and 72.23% accuracy, respectively

Mechanical properties, in vitro corrosion and biocompatibility of newly developed biodegradable Mg-Zr-Sr-Ho alloys for biomedical applications

Our previous studies have demonstrated that Mg-Zr-Sr alloys can be anticipated as excellent biodegradable implant materials for load-bearing applications. In general, rare earth elements (REEs) are widely used in magnesium (Mg) alloys with the aim of enhancing the mechanical properties of Mg-based alloys. In this study, the REE holmium (Ho) was added to an Mg-1Zr-2Sr alloy at different concentrations of Mg1Zr2SrxHo alloys (x = 0, 1, 3, 5 wt. %) and the microstructure, mechanical properties, degradation behaviour and biocompatibility of the alloys were systematically investigated. The results indicate that the addition of Ho to Mg1Zr2Sr led to the formation of the intermetallic phases MgHo3, Mg2Ho and Mg17Sr2 which resulted in enhanced mechanical strength and decreased degradation rates of the Mg-Zr-Sr-Ho alloys. Furthermore, Ho addition (≤5 wt. %) to Mg-Zr-Sr alloys led to enhancement of cell adhesion and proliferation of osteoblast cells on the Mg-Zr-Sr-Ho alloys. The in vitro biodegradation and the biocompatibility of the Mg-Zr-Sr-Ho alloys were both influenced by the Ho concentration in the Mg alloys; Mg1Zr2Sr3Ho exhibited lower degradation rates than Mg1Zr2Sr and displayed the best biocompatibility compared with the other alloys

A Time-slice Based Hybrid Routing for Delay Tolerant Networks

Abstract: The non-existence of an end-to-end path poses a challenge in adapting the traditional routing algorithms to delay tolerant networks (DTNs). This paper innovatively puts forward the concept of "time-slice" to make full use of the respective advantages of single copy strategy and multiple-copy strategy thus getting a right balance between high message delivery ratio and low network overloads. We investigate making the routing decision based only on no more than one-hop information of neighbor nodes so as to enhance the practicability of our routing by reducing the complexity of neighbor discovery. Then a time-slice based hybrid routing protocol is proposed. Simulation results show that our proposed routing achieves the overall best performance than other protocols. When the network resource is constrained, our proposed routing scheme is more scalable than others

Narrative review of magnetic resonance imaging in quantifying liver iron load

ObjectiveTo summarize the research progress of magnetic resonance imaging (MRI) in quantifying liver iron load.MethodsTo summarize the current status and progress of MRI technology in the quantitative study of liver iron load through reviewing the relevant literature at home and abroad.ResultsDifferent MRI sequence examination techniques have formed a series of non-invasive methods for the examination of liver iron load. These techniques have important clinical significance in the imaging diagnosis of liver iron load. So far, the main MRI methods used to assess liver iron load are: signal intensity measurement method (signal intensity, SI) [signal intensity ratio (SIR) and difference in in-phase and out-of-phase signal intensity], T2/R2 measurement (such as FerriScan technique), ultra-short echo time (UTE) imaging technique, and susceptibility weighted imaging (including conventional susceptibility weighted imaging) (SWI), quantitative susceptibility mapping (QSM), T2*/R2* measurement, Dixon and its derivative techniques.ConclusionMRI has become the first choice for the non-invasive examination of liver iron overload, and it is helpful to improve the early detection of liver injury, liver fibrosis, liver cirrhosis and liver cancer caused by liver iron overload