Question Classification in the Cancer Domain

We are investigating question classification for restricted domains with the broader goal of supporting mixed-initiative interaction on mobile phones. In this thesis, we present the development of a new domain-specific corpus of cancer-related questions, a new taxonomy of Expected Answer types, and our efforts toward training a classifier. This work is the first of its kind in the cancer domain using a corpus consisting of real user questions gathered from cQA websites, and a taxonomy built from that corpus. Our goal is to create software to engage newly diagnosed prostate cancer patients in question-answering dialogs related to their treatment options. We are focusing our work on the interaction environment afforded by text and multimedia (SMS and MMS) messaging using mobile telephones, because of the prevalence of this technology and the growing popularity of text messaging, especially among underserved populations

Análisis de calidad y recuperación de información en foros de discusión

Actualmente las organizaciones dedican mucho esfuerzo a resolver problemas utilizando estrategias ya probadas, lo que requiere brindar especial importancia a los sistemas de Information Retrieval (IR). Ante un problema que surge, lo primero que se analiza es si ha sido resuelto antes, bajo qué situaciones y en qué contexto se dieron problemas similares. Es aquí donde cobran gran importancia los sitios web donde se propician discusiones e intercambio de ideas sobre problemas comunes. Herramientas colaborativas como los blogs, wikis y foros de discusión hacen posible el reúso de conocimiento disponible en la Web. Entre ellas, nuestro proyecto se enfoca en los foros de discusión como herramienta fundamental, ya que contiene una base de conocimiento lo suficientemente completa para ser reutilizada. La enorme cantidad de información existente en los foros, sumado a la simplicidad de uso de los mismos, hacen necesario definir estrategias para clasificar las soluciones ya probadas, fundamentales en los sistemas IR. De esta manera el principal objetivo de este proyecto es la creación de una herramienta que realice una clasificación automática de la información contenida en foros de discusión, utilizando la información existente en los hilos de discusión, que luego de ser analizada, será procesada, reutilizada y clasificada para solucionar problemas específicos recurrentes que fueran surgiendo.Eje: Ingeniería de Software.Red de Universidades con Carreras en Informátic

CHATREPORT: Democratizing sustainability disclosure analysis through LLM-based tools

In the face of climate change, are companies really taking substantial steps toward more sustainable operations? A comprehensive answer lies in the dense, information-rich landscape of corporate sustainability reports. However, the sheer volume and complexity of these reports make human analysis very costly. Therefore, only a few entities worldwide have the resources to analyze these reports at scale, which leads to a lack of transparency in sustainability reporting. Empowering stakeholders with LLM-based automatic analysis tools can be a promising way to democratize sustainability report analysis. However, developing such tools is challenging due to (1) the hallucination of LLMs and (2) the inefficiency of bringing domain experts into the AI development loop. In this paper, we ChatReport, a novel LLM-based system to automate the analysis of corporate sustainability reports, addressing existing challenges by (1) making the answers traceable to reduce the harm of hallucination and (2) actively involving domain experts in the development loop. We make our methodology, annotated datasets, and generated analyses of 1015 reports publicly available

Leveraging Formulae and Text for Improved Math Retrieval

Large collections containing millions of math formulas are available online. Retrieving math expressions from these collections is challenging. Users can use formula, formula+text, or math questions to express their math information needs. The structural complexity of formulas requires specialized processing. Despite the existence of math search systems and online community question-answering websites for math, little is known about mathematical information needs. This research first explores the characteristics of math searches using a general search engine. The findings show how math searches are different from general searches. Then, test collections for math-aware search are introduced. The ARQMath test collections have two main tasks: 1) finding answers for math questions and 2) contextual formula search. In each test collection (ARQMath-1 to -3) the same collection is used, Math Stack Exchange posts from 2010 to 2018, introducing different topics for each task. Compared to the previous test collections, ARQMath has a much larger number of diverse topics, and improved evaluation protocol. Another key role of this research is to leverage text and math information for improved math information retrieval. Three formula search models that only use the formula, with no context are introduced. The first model is an n-gram embedding model using both symbol layout tree and operator tree representations. The second model uses tree-edit distance to re-rank the results from the first model. Finally, a learning-to-rank model that leverages full-tree, sub-tree, and vector similarity scores is introduced. To use context, Math Abstract Meaning Representation (MathAMR) is introduced, which generalizes AMR trees to include math formula operations and arguments. This MathAMR is then used for contextualized formula search using a fine-tuned Sentence-BERT model. The experiments show tree-edit distance ranking achieves the current state-of-the-art results on contextual formula search task, and the MathAMR model can be beneficial for re-ranking. This research also addresses the answer retrieval task, introducing a two-step retrieval model in which similar questions are first found and then answers previously given to those similar questions are ranked. The proposed model, fine-tunes two Sentence-BERT models, one for finding similar questions and another one for ranking the answers. For Sentence-BERT model, raw text as well as MathAMR are used

Combating Attacks and Abuse in Large Online Communities

Internet users today are connected more widely and ubiquitously than ever before. As a result, various online communities are formed, ranging from online social networks (Facebook, Twitter), to mobile communities (Foursquare, Waze), to content/interests based networks (Wikipedia, Yelp, Quora). While users are benefiting from the ease of access to information and social interactions, there is a growing concern for users' security and privacy against various attacks such as spam, phishing, malware infection and identity theft. Combating attacks and abuse in online communities is challenging. First, today’s online communities are increasingly dependent on users and user-generated content. Securing online systems demands a deep understanding of the complex and often unpredictable human behaviors. Second, online communities can easily have millions or even billions of users, which requires the corresponding security mechanisms to be highly scalable. Finally, cybercriminals are constantly evolving to launch new types of attacks. This further demands high robustness of security defenses. In this thesis, we take concrete steps towards measuring, understanding, and defending against attacks and abuse in online communities. We begin with a series of empirical measurements to understand user behaviors in different online services and the uniquesecurity and privacy challenges that users are facing with. This effort covers a broad set of popular online services including social networks for question and answering (Quora), anonymous social networks (Whisper), and crowdsourced mobile communities (Waze). Despite the differences of specific online communities, our study provides a first look at their user activity patterns based on empirical data, and reveals the need for reliable mechanisms to curate user content, protect privacy, and defend against emerging attacks. Next, we turn our attention to attacks targeting online communities, with focus on spam campaigns. While traditional spam is mostly generated by automated software, attackers today start to introduce "human intelligence" to implement attacks. This is maliciouscrowdsourcing (or crowdturfing) where a large group of real-users are organized to carry out malicious campaigns, such as writing fake reviews or spreading rumors on social media. Using collective human efforts, attackers can easily bypass many existing defenses (e.g.,CAPTCHA). To understand the ecosystem of crowdturfing, we first use measurements to examine their detailed campaign organization, workers and revenue. Based on insights from empirical data, we develop effective machine learning classifiers to detect crowdturfingactivities. In the meantime, considering the adversarial nature of crowdturfing, we also build practical adversarial models to simulate how attackers can evade or disrupt machine learning based defenses. To aid in this effort, we next explore using user behavior models to detect a wider range of attacks. Instead of making assumptions about attacker behavior, our idea is to model normal user behaviors and capture (malicious) behaviors that are deviated from norm. In this way, we can detect previously unknown attacks. Our behavior model is based on detailed clickstream data, which are sequences of click events generated by users when using the service. We build a similarity graph where each user is a node and the edges are weightedby clickstream similarity. By partitioning this graph, we obtain "clusters" of users with similar behaviors. We then use a small set of known good users to "color" these clusters to differentiate the malicious ones. This technique has been adopted by real-world social networks (Renren and LinkedIn), and already detected unexpected attacks. Finally, we extend clickstream model to understanding more-grained behaviors of attackers (and real users), and tracking how user behavior changes over time. In summary, this thesis illustrates a data-driven approach to understanding and defending against attacks and abuse in online communities. Our measurements have revealed new insights about how attackers are evolving to bypass existing security defenses today. Inaddition, our data-driven systems provide new solutions for online services to gain a deep understanding of their users, and defend them from emerging attacks and abuse

Language representations for computational argumentation

Argumentation is an essential feature and, arguably, one of the most exciting phenomena of natural language use. Accordingly, it has fascinated scholars and researchers in various fields, such as linguistics and philosophy, for long. Its computational analysis, falling under the notion of computational argumentation, is useful in a variety of domains of text for a range of applications. For instance, it can help to understand users’ stances in online discussion forums towards certain controversies, to provide targeted feedback to users for argumentative writing support, and to automatically summarize scientific publications. As in all natural language processing pipelines, the text we would like to analyze has to be introduced to computational argumentation models in the form of numeric features. Choosing such suitable semantic representations is considered a core challenge in natural language processing. In this context, research employing static and contextualized pretrained text embedding models has recently shown to reach state-of-the-art performances for a range of natural language processing tasks. However, previous work has noted the specific difficulty of computational argumentation scenarios with language representations as one of the main bottlenecks and called for targeted research on the intersection of the two fields. Still, the efforts focusing on the interplay between computational argumentation and representation learning have been few and far apart. This is despite (a) the fast-growing body of work in both computational argumentation and representation learning in general and (b) the fact that some of the open challenges are well known in the natural language processing community. In this thesis, we address this research gap and acknowledge the specific importance of research on the intersection of representation learning and computational argumentation. To this end, we (1) identify a series of challenges driven by inherent characteristics of argumentation in natural language and (2) present new analyses, corpora, and methods to address and mitigate each of the identified issues. Concretely, we focus on five main challenges pertaining to the current state-of-the-art in computational argumentation: (C1) External knowledge: static and contextualized language representations encode distributional knowledge only. We propose two approaches to complement this knowledge with knowledge from external resources. First, we inject lexico-semantic knowledge through an additional prediction objective in the pretraining stage. In a second study, we demonstrate how to inject conceptual knowledge post hoc employing the adapter framework. We show the effectiveness of these approaches on general natural language understanding and argumentative reasoning tasks. (C2) Domain knowledge: pretrained language representations are typically trained on big and general-domain corpora. We study the trade-off between employing such large and general-domain corpora versus smaller and domain-specific corpora for training static word embeddings which we evaluate in the analysis of scientific arguments. (C3) Complementarity of knowledge across tasks: many computational argumentation tasks are interrelated but are typically studied in isolation. In two case studies, we show the effectiveness of sharing knowledge across tasks. First, based on a corpus of scientific texts, which we extend with a new annotation layer reflecting fine-grained argumentative structures, we show that coupling the argumentative analysis with other rhetorical analysis tasks leads to performance improvements for the higher-level tasks. In the second case study, we focus on assessing the argumentative quality of texts. To this end, we present a new multi-domain corpus annotated with ratings reflecting different dimensions of argument quality. We then demonstrate the effectiveness of sharing knowledge across the different quality dimensions in multi-task learning setups. (C4) Multilinguality: argumentation arguably exists in all cultures and languages around the globe. To foster inclusive computational argumentation technologies, we dissect the current state-of-the-art in zero-shot cross-lingual transfer. We show big drops in performance when it comes to resource-lean and typologically distant target languages. Based on this finding, we analyze the reasons for these losses and propose to move to inexpensive few-shot target-language transfer, leading to consistent performance improvements in higher-level semantic tasks, e.g., argumentative reasoning. (C5) Ethical considerations: envisioned computational argumentation applications, e.g., systems for self-determined opinion formation, are highly sensitive. We first discuss which ethical aspects should be considered when representing natural language for computational argumentation tasks. Focusing on the issue of unfair stereotypical bias, we then conduct a multi-dimensional analysis of the amount of bias in monolingual and cross-lingual embedding spaces. In the next step, we devise a general framework for implicit and explicit bias evaluation and debiasing. Employing intrinsic bias measures and benchmarks reflecting the semantic quality of the embeddings, we demonstrate the effectiveness of new debiasing methods, which we propose. Finally, we complement this analysis by testing the original as well as the debiased language representations for stereotypically unfair bias in argumentative inferences. We hope that our contributions in language representations for computational argumentation fuel more research on the intersection of the two fields and contribute to fair, efficient, and effective natural language processing technologies

Justification for Class 3 Permit Modification, Corrective Action Complete with Controls, Solid Waste Management Unit 76, Mixed Waste Landfill, Sandia National Laboratories/New Mexico, EPA ID Number NM5890110518 Volumes I through VIII

The Department of Energy/National Nuclear Security Administration (DOE) and Sandia Corporation (Sandia) are submitting a request for a Class 3 Modification to Module IV of Hazardous Waste Permit NM5890110518-1 (the Permit). DOE and Sandia are requesting that the New Mexico Environment Department (NMED) designate solid waste management unit (SWMU) 76 as approved for Corrective Action Complete status. NMED made a preliminary determination in October 2014 that corrective action is complete at this SWMU. SWMU 76, known as the Mixed Waste Landfill (MWL), is a 2.6-acre site at Sandia National Laboratories, located on Kirtland Air Force Base immediately southeast of Albuquerque, New Mexico. Radioactive wastes and mixed wastes (radioactive wastes that are also hazardous wastes) were disposed of in the MWL from March 1959 through December 1988. The meximum depth of burial is approximately 25 feet below the ground surface. Groundwater occurs approximately 500 feet below the ground surface at the MWL. DOE and Sandia have implemented corrective measures at SWMU 76 in accordance with the requirements of the Permit; an April 2004 Compliance Order on Consent between NMED, DOE, and Sandia; and the plans approved by NMED. On January 8, 2014, NMED approved a long-term monitoring and maintenance plan (LTMMP) for SWMU 76. DOE and Sandia have implemented the approved LTMMP, maintaining the controls established through the corrective measures. The permit modification request consists of a letter with two enclosures: 1. A brief history or corrective action at SWMU 76 2. An index of the supporting documents that comprise the justification for the permit modification request. The supporting documents are included in an 8-volume set: Justification for Class 3 Permit Modification for Corrective Action Complete With Controls, Solid Waste Management Unit 76, Mixed Waste Landfill. Volume/pages: I/858. II/420. III/556. IV/1128. V/848. VI/1110. VII/914. VIII/866

Exploring contingent employment policy in IT – impacts upon IT project management capabilities enhancement in large Hong Kong organisations

Large Hong Kong organisations rely heavily on IT projects to sharpen their competitiveness. Contingent IT project employment is a globally increasing trend that results in high staff turnover. This raises risks that organisational project knowledge is rarely retained when contingent workers leave organisations and as individual contingent workers become frustrated by employer’s lack of commitment to their future. Contingent employment policy has generally been assumed to impede IT project management’s organisational capability and competitiveness through knowledge leakage and inhibited organisational learning. Limited research has been undertaken on the impact of contingent IT project management employment on organisational capability though there have been numerous separate studies on: IT project management; general contingent employment; and enhancing IT project management capabilities. This thesis combines these themes to (i) explore the importance of continuous advancement of IT project management capabilities to business successes; (ii) identify and explain the contingent and permanent employment policies of IT professionals (including project managers) in large Hong Kong organisations; (iii) investigate and explain the impacts of contingent employment policies on enhancing IT project management capabilities; (iv) identify and explain the practices of advancing IT project management capabilities as an individual, as a group and as a large organisation; and (v) identify and present possible solutions to satisfy the needs to advance IT project management capabilities using contingent employment. A case study multiple-case, comparative research design was followed that relied on open-ended interviews supported by semi-structured interviews and using archival documentation. Three case study organisations typify large Hong Kong organisations that are major IT workforce employers. The first and second case study organisations are representative of a large IT users organisation and an IT and communications services organisation (the two key categories of IT employers) respectively employing a high percentage (over 50%) of contingent IT workers. The third case study organisation is a contrast case since it employs a low percentage (below 20%) of contingent IT workers and is a typical IT users organisation. Analysis concluded that the degree of projectisation, project resource strategies and investment on IT project management capabilities have to fit the organisation’s specific business dynamics. The business situation of organisations determines IT projects’ scale and complexity. This leads IT groups to be organised along a functional, balanced matrix or projectised structure spectrum. Organisations with greater projectisation are more likely to rely on contingent IT workers. Continuing to enhance IT project management capability while depending on an increasing percentage of mobile external resources (including contingent workers) may require an organisation to invest more in project governance and support structures, project management methodologies and tools. Alternatively, organisations may prefer retaining in-house staff to capture their tacit organisational knowledge and invest in cognitive and operational learning to retain codified organisational knowledge while avoiding weakness in reflective learning and social learning processes

Understanding patient experience from online medium

Improving patient experience at hospitals leads to better health outcomes. To improve this, we must first understand and interpret patients' written feedback. Patient-generated texts such as patient reviews found on RateMD, or online health forums found on WebMD are venues where patients post about their experiences. Due to the massive amounts of patient-generated texts that exist online, an automated approach to identifying the topics from patient experience taxonomy is the only realistic option to analyze these texts. However, not only is there a lack of annotated taxonomy on these media, but also word usage is colloquial, making it challenging to apply standardized NLP technique to identify the topics that are present in the patient-generated texts. Furthermore, patients may describe multiple topics in the patient-generated texts which drastically increases the complexity of the task. In this thesis, we address the challenges in comprehensively and automatically understanding the patient experience from patient-generated texts. We first built a set of rich semantic features to represent the corpus which helps capture meanings that may not typically be captured by the bag-of-words (BOW) model. Unlike the BOW model, semantic feature representation captures the context and in-depth meaning behind each word in the corpus. To the best of our knowledge, no existing work in understanding patient experience from patient-generated texts delves into which semantic features help capture the characteristics of the corpus. Furthermore, patients generally talk about multiple topics when they write in patient-generated texts, and these are frequently interdependent of each other. There are two types of topic interdependencies, those that are semantically similar, and those that are not. We built a constraint-based deep neural network classifier to capture the two types of topic interdependencies and empirically show the classification performance improvement over the baseline approaches. Past research has also indicated that patient experiences differ depending on patient segments [1-4]. The segments can be based on demographics, for instance, by race, gender, or geographical location. Similarly, the segments can be based on health status, for example, whether or not the patient is taking medication, whether or not the patient has a particular disease, or whether or not the patient is readmitted to the hospital. To better understand patient experiences, we built an automated approach to identify patient segments with a focus on whether the person has stopped taking the medication or not. The technique used to identify the patient segment is general enough that we envision the approach to be applicable to other types of patient segments. With a comprehensive understanding of patient experiences, we envision an application system where clinicians can directly read the most relevant patient-generated texts that pertain to their interest. The system can capture topics from patient experience taxonomy that is of interest to each clinician or designated expert, and we believe the system is one of many approaches that can ultimately help improve the patient experience