Global temporal typing patterns in foreign language writing: Exploring language proficiency through recurrence quantification analysis (RQA)

Recurrence quantification analysis (RQA) is a time-series analysis method that uses autocorrelation properties of typing data to detect regularities within the writing process. The following paper first gives a detailed introduction to RQA and its application to time series data. We then apply RQA to keystroke logging data of first and foreign language writing to illustrate how outcome measures of RQA can be understood as skill-driven constraints on keyboard typing performance. Forty native German students performed two prompted writing assignments, one in German and one in English, a standardized copy task, and a standardized English placement test. We assumed more fluent and skilled writing to reveal more structured typing time series patterns. Accordingly, we expected writing in a well-mastered first language to coincide with higher values in relevant RQA measures as compared to writing in a foreign language. Results of mixed model ANOVAs confirmed our hypothesis. We further observed that RQA measures tend to be higher, thus indicating more structured data, whenever parameters of pause, burst, and revision analyses indicate more fluent writing. Multiple regression analyses revealed that, in addition to typing skills, language proficiency significantly predicts outcomes of RQA. Thus, the present data emphasize RQA being a valuable resource for studying time series data that yields meaningful information about the effort a writer must exert during text production

ODSum: New Benchmarks for Open Domain Multi-Document Summarization

Open-domain Multi-Document Summarization (ODMDS) is a critical tool for condensing vast arrays of documents into coherent, concise summaries. With a more inter-related document set, there does not necessarily exist a correct answer for the retrieval, making it hard to measure the retrieving performance. We propose a rule-based method to process query-based document summarization datasets into ODMDS datasets. Based on this method, we introduce a novel dataset, ODSum, a sophisticated case with its document index interdependent and often interrelated. We tackle ODMDS with the \textit{retrieve-then-summarize} method, and the performance of a list of retrievers and summarizers is investigated. Through extensive experiments, we identify variances in evaluation metrics and provide insights into their reliability. We also found that LLMs suffer great performance loss from retrieving errors. We further experimented methods to improve the performance as well as investigate their robustness against imperfect retrieval. We will release our data and code at https://github.com/yale-nlp/ODSum

Computational modeling of turn-taking dynamics in spoken conversations

The study of human interaction dynamics has been at the center for multiple research disciplines in- cluding computer and social sciences, conversational analysis and psychology, for over decades. Recent interest has been shown with the aim of designing computational models to improve human-machine interaction system as well as support humans in their decision-making process. Turn-taking is one of the key aspects of conversational dynamics in dyadic conversations and is an integral part of human- human, and human-machine interaction systems. It is used for discourse organization of a conversation by means of explicit phrasing, intonation, and pausing, and it involves intricate timing. In verbal (e.g., telephone) conversation, the turn transitions are facilitated by inter- and intra- speaker silences and over- laps. In early research of turn-taking in the speech community, the studies include durational aspects of turns, cues for turn yielding intention and lastly designing turn transition modeling for spoken dia- log agents. Compared to the studies of turn transitions very few works have been done for classifying overlap discourse, especially the competitive act of overlaps and function of silences. Given the limitations of the current state-of-the-art, this dissertation focuses on two aspects of con- versational dynamics: 1) design automated computational models for analyzing turn-taking behavior in a dyadic conversation, 2) predict the outcome of the conversations, i.e., observed user satisfaction, using turn-taking descriptors, and later these two aspects are used to design a conversational profile for each speaker using turn-taking behavior and the outcome of the conversations. The analysis, experiments, and evaluation has been done on a large dataset of Italian call-center spoken conversations where customers and agents are engaged in real problem-solving tasks. Towards solving our research goal, the challenges include automatically segmenting and aligning speakers’ channel from the speech signal, identifying and labeling the turn-types and its functional aspects. The task becomes more challenging due to the presence of overlapping speech. To model turn- taking behavior, the intension behind these overlapping turns needed to be considered. However, among all, the most critical question is how to model observed user satisfaction in a dyadic conversation and what properties of turn-taking behavior can be used to represent and predict the outcome. Thus, the computational models for analyzing turn-taking dynamics, in this dissertation includes au- tomatic segmenting and labeling turn types, categorization of competitive vs non-competitive overlaps, silences (e.g., lapse, pauses) and functions of turns in terms of dialog acts. The novel contributions of the work presented here are to 1. design of a fully automated turn segmentation and labeling (e.g., agent vs customer’s turn, lapse within the speaker, and overlap) system. 2. the design of annotation guidelines for segmenting and annotating the speech overlaps with the competitive and non-competitive labels. 3. demonstrate how different channels of information such as acoustic, linguistic, and psycholin- guistic feature sets perform in the classification of competitive vs non-competitive overlaps. 4. study the role of speakers and context (i.e., agents’ and customers’ speech) for conveying the information of competitiveness for each individual feature set and their combinations. 5. investigate the function of long silences towards the information flow in a dyadic conversation. The extracted turn-taking cues is then used to automatically predict the outcome of the conversation, which is modeled from continuous manifestations of emotion. The contributions include 1. modeling the state of the observed user satisfaction in terms of the final emotional manifestation of the customer (i.e., user). 2. analysis and modeling turn-taking properties to display how each turn type influence the user satisfaction. 3. study of how turn-taking behavior changes within each emotional state. Based on the studies conducted in this work, it is demonstrated that turn-taking behavior, specially competitiveness of overlaps, is more than just an organizational tool in daily human interactions. It represents the beneficial information and contains the power to predict the outcome of the conversation in terms of satisfaction vs not-satisfaction. Combining the turn-taking behavior and the outcome of the conversation, the final and resultant goal is to design a conversational profile for each speaker. Such profiled information not only facilitate domain experts but also would be useful to the call center agent in real time. These systems are fully automated and no human intervention is required. The findings are po- tentially relevant to the research of overlapping speech and automatic analysis of human-human and human-machine interactions

FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics

Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer representations, limiting the capture of structurally relevant gene contexts. To address these limitations and further our understanding of complex relationships between metagenomic sequences and their functions, we introduce a protein-based gene representation as a context-aware and structure-relevant tokenizer. Our approach includes Masked Gene Modeling (MGM) for gene group-level pre-training, providing insights into inter-gene contextual information, and Triple Enhanced Metagenomic Contrastive Learning (TEM-CL) for gene-level pre-training to model gene sequence-function relationships. MGM and TEM-CL constitute our novel metagenomic language model {\NAME}, pre-trained on 100 million metagenomic sequences. We demonstrate the superiority of our proposed {\NAME} on eight datasets

Pausing as practice in strategy - making and engagement - a case study

The study explores pausing in action is, within the ambit of Strategy-as-practice (s-ap) as an emergent school of thought. Pausing is thus discerned during the implementation phase of the strategy of a credit risk system within a South African bank, as strategy is known to take shape during implementation. Different sites of the bank’s systems – change, strategy practitioners, and their times of pausing, form the unit of analysis. Strategy-making and engagement are explored by understanding the influence of pausing on enabling or disenabling the strategic outcome of the risk system. Pausing is situated in an applied and theoretical gap as an intangible under-theorised strategy practice. Practitioners, as champions or non-champions of strategy, pause in various ways, and attribute meaning to this ‘action’. Their account of pausing is recognised for its value-adding or diminishing dimensions to strategy-making. The study follows a comprehensive literature review which shows limited theoretical positions on embodied, latent practices, such as pausing, as strategic practices. The body of knowledge provides a challenge for scholars to consider perceived ‘silences’ or the ‘receding’ of strategists as un-remarked dimensions of strategy, which could nevertheless be instrumental in the nature of the strategic outcome. The contribution of the current study identifies pausing as a strategic practice, especially when considered within the structure of engagement and social learningBusiness ManagementM. Com. (Business Management

A Model of the Network Architecture of the Brain that Supports Natural Language Processing

For centuries, neuroscience has proposed models of the neurobiology of language processing that are static and localised to few temporal and inferior frontal regions. Although existing models have offered some insight into the processes underlying lower-level language features, they have largely overlooked how language operates in the real world. Here, we aimed at investigating the network organisation of the brain and how it supports language processing in a naturalistic setting. We hypothesised that the brain is organised in a multiple core-periphery and dynamic modular architecture, with canonical language regions forming high-connectivity hubs. Moreover, we predicted that language processing would be distributed to much of the rest of the brain, allowing it to perform more complex tasks and to share information with other cognitive domains. To test these hypotheses, we collected the Naturalistic Neuroimaging Database of people watching full length movies during functional magnetic resonance imaging. We computed network algorithms to capture the voxel-wise architecture of the brain in individual participants and inspected variations in activity distribution over different stimuli and over more complex language features. Our results confirmed the hypothesis that the brain is organised in a flexible multiple core-periphery architecture with large dynamic communities. Here, language processing was distributed to much of the rest of the brain, together forming multiple communities. Canonical language regions constituted hubs, explaining why they consistently appear in various other neurobiology of language models. Moreover, language processing was supported by other regions such as visual cortex and episodic memory regions, when processing more complex context-specific language features. Overall, our flexible and distributed model of language comprehension and the brain points to additional brain regions and pathways that could be exploited for novel and more individualised therapies for patients suffering from speech impairments

A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co-speech gestures is a long-standing problem in computer animation and is considered an enabling technology in film, games, virtual social spaces, and for interaction with social robots. The problem is made challenging by the idiosyncratic and non-periodic nature of human co-speech gesture motion, and by the great diversity of communicative functions that gestures encompass. Gesture generation has seen surging interest recently, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep-learning-based generative models, that benefit from the growing availability of data. This review article summarizes co-speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule-based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text, and non-linguistic input. We also chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method. Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human-like motion; grounding the gesture in the co-occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.Comment: Accepted for EUROGRAPHICS 202

Exploring Associations between Language and Working Memory Abilities in Children with Specific or Combined Impairments in Language and Working Memory

Children with disproportionate deficits in language, known as Specific Language Impairment (SLI), often demonstrate deficits in nonverbal cognitive abilities, such as working memory. Such findings have prompted much debate on the association between language and working memory functioning. The primary aim of this thesis was to examine the connection between working memory and language abilities among children with specific or combined impairments in these domains. Study 1 examined the potential of narrative retell performance to indicate impairment in language or working memory among 17 children with specific or combined impairment in language or working memory as well as 9 controls. Quantitative analysis using logistic regression revealed that language impairment was predicted best by the interaction between mean length of utterance, percent grammatical utterances, and age, whereas working memory impairment was best predicted by the interaction between events recalled and subordinate clauses per utterance. Exploratory qualitative analysis using qualitative descriptors differentiated narratives of children with and without impairment and revealed clusters of descriptors that identified contrasting speaking styles. Study 2 tested domain-specific interventions in language or working memory using a single subject design. Chapter 3 reports the effects of a narrative-based language intervention for 10 children with language impairment with or without working memory impairment. Results showed gains on narrative ability for most participants, and broader linguistic gains for half of the participants. Intervention effects on related domains (i.e., working memory, reading, math) were evident for some participants as well. Chapter 4 reports the effects of a working memory training program for 7 children with working memory impairment with or without language impairment. Results showed training effects on working memory tasks similar to training tasks for all participants. Transfer to language ability was seen for 4 participants, and transfer to reading or math was evident for 3 participants. Responder analyses for Study 2 showed associations between intervention effectiveness and baseline cognitive abilities, age, speaking style, and intervention intensity. Results support the view that working memory and language are separable but closely related cognitive processes. Responder analyses highlight the importance of considering heterogeneity among children with impairments in research and clinical settings

Development of initial competence in listening comprehension: a tentative analysis

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão. Programa de Pós-graduação em Inglês e Literatura Correspondent