How hybrid working from home works out

Hybrid working from home (WFH), whereby employees work a mix of days at home and at work each week, has become dominant for graduate employees in the US. This paper evaluates a randomized control trial on 1612 engineers, marketing and finance employees of a large technology firm that allowed odd birthday employees to WFH on Wednesday and Friday and kept even birthday employees full time in the office. There are four key results. First, WFH reduced attrition rates by 35% and improved self-reported work satisfaction scores, highlighting how employees place a considerable value on this amenity. Second, WFH reduced hours worked on home days but increased it on other work days and the weekend, highlighting how home-working alters the structure of the working week. Third, WFH employees increased individual messaging and group video call communication, even when in the office, reflecting the impact of remote work on working patterns. Finally, while there was no significant impact of WFH on performance ratings or promotions, lines of code written increased by 8%, and employees' self-assessed productivity was up 1.8%, suggesting a small positive impact. Given these benefits for retention, job satisfaction, and productivity, after the experiment ended the firm extended hybrid WFH to the entire company

Neural Snowball for Few-Shot Relation Learning

Knowledge graphs typically undergo open-ended growth of new relations. This cannot be well handled by relation extraction that focuses on pre-defined relations with sufficient training data. To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations. More specifically, we use Relational Siamese Networks (RSN) to learn the metric of relational similarities between instances based on existing relations and their labeled data. Afterwards, given a new relation and its few-shot instances, we use RSN to accumulate reliable instances from unlabeled corpora; these instances are used to train a relation classifier, which can further identify new facts of the new relation. The process is conducted iteratively like a snowball. Experiments show that our model can gather high-quality instances for better few-shot relation learning and achieves significant improvement compared to baselines. Codes and datasets are released on https://github.com/thunlp/Neural-Snowball.Comment: Accepted by AAAI202

Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models

Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise. However, it is important to note that parameter sharing does not alleviate computational burdens associated with inference, thus impeding its practicality in situations characterized by limited stringent latency requirements or computational resources. Building upon neural ordinary differential equations (ODEs), we introduce a straightforward technique to enhance the inference efficiency of parameter-shared PLMs. Additionally, we propose a simple pre-training technique that leads to fully or partially shared models capable of achieving even greater inference acceleration. The experimental results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs, providing novel insights into more efficient utilization of parameter-shared models in resource-constrained settings.Comment: EMNLP 2023 Finding

Denoising Relation Extraction from Document-level Distant Supervision

Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance. However, the existing success of DS cannot be directly transferred to the more challenging document-level relation extraction (DocRE), since the inherent noise in DS may be even multiplied in document level and significantly harm the performance of RE. To address this challenge, we propose a novel pre-trained model for DocRE, which denoises the document-level DS data via multiple pre-training tasks. Experimental results on the large-scale DocRE benchmark show that our model can capture useful information from noisy DS data and achieve promising results.Comment: EMNLP 2020 short pape

Comparative Study on the Effects and Mechanisms of Salt and Alkali on the Quality Formation of Noodles

In order to systematically investigate the quality differences between salted and alkaline noodles and the underlying mechanism, the effect of NaCl and K2CO3 on the quality formation of noodles was studied by evaluating the farinograph and extensograph properties and viscosity properties of wheat flour, as well as the cooking properties, texture properties, and storage stability of noodles. Additionally, the microstructure, protein aggregation and volatile components were investigated to explore the underlying mechanisms of the quality differences. The results showed that salt improved the extensibility of dough and noodles, endowed noodles with higher smoothness and elasticity, and showed little effect on starch viscosity, while alkali enhanced the tensile resistance of dough and the hardness and breaking force of cooked noodles, and increased the gelatinization viscosity of starch. The cooking loss of noodles increased with the addition of salt and alkali. Both salt and alkali significantly inhibited the increase of total plate count (TPC) in fresh noodles and consequently enhanced the storage stability, but 0.5% K2CO3 accelerated the browning rate. The scanning electron microscopy (SEM) results showed that NaCl induced a smooth surface microstructure, while K2CO3 resulted in a rough surface of noodles; K2CO3 significantly increased the rate and extent of thermal polymerization of proteins in noodles. NaCl increased the type and concentration of volatile components in noodles, thereby making the aroma of noodles more intense, but the flavor was similar to that of the control. K2CO3 significantly changed the flavor of noodles and resulted in the generation of unique aldehyde compounds

Emergent Modularity in Pre-trained Transformers

This work examines the presence of modularity in pre-trained Transformers, a feature commonly found in human brains and thought to be vital for general intelligence. In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes. (2) function-based neuron grouping: we explore finding a structure that groups neurons into modules by function, and each module works for its corresponding function. Given the enormous amount of possible structures, we focus on Mixture-of-Experts as a promising candidate, which partitions neurons into experts and usually activates different experts for different inputs. Experimental results show that there are functional experts, where clustered are the neurons specialized in a certain function. Moreover, perturbing the activations of functional experts significantly affects the corresponding function. Finally, we study how modularity emerges during pre-training, and find that the modular structure is stabilized at the early stage, which is faster than neuron stabilization. It suggests that Transformers first construct the modular structure and then learn fine-grained neuron functions. Our code and data are available at https://github.com/THUNLP/modularity-analysis.Comment: Findings of ACL 202

Multi-task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs

Many popular knowledge graphs such as Freebase, YAGO or DBPedia maintain a list of non-discrete attributes for each entity. Intuitively, these attributes such as height, price or population count are able to richly characterize entities in knowledge graphs. This additional source of information may help to alleviate the inherent sparsity and incompleteness problem that are prevalent in knowledge graphs. Unfortunately, many state-of-the-art relational learning models ignore this information due to the challenging nature of dealing with non-discrete data types in the inherently binary-natured knowledge graphs. In this paper, we propose a novel multi-task neural network approach for both encoding and prediction of non-discrete attribute information in a relational setting. Specifically, we train a neural network for triplet prediction along with a separate network for attribute value regression. Via multi-task learning, we are able to learn representations of entities, relations and attributes that encode information about both tasks. Moreover, such attributes are not only central to many predictive tasks as an information source but also as a prediction target. Therefore, models that are able to encode, incorporate and predict such information in a relational learning context are highly attractive as well. We show that our approach outperforms many state-of-the-art methods for the tasks of relational triplet classification and attribute value prediction.Comment: Accepted at CIKM 201