Design Principles for Signal Detection in Modern Job Application Systems: Identifying Fabricated Qualifications

Hiring a new employee is traditionally thought to be an uncertain investment. This uncertainty is lessened by the presence of signals that indicate job fitness. Ideally, job applicants objectively signal their qualifications, and those signals are correctly assessed by the hiring team. In reality, signal manipulation is pervasive in the hiring process, mitigating the reliability of signals used to make hiring decisions. To combat these inefficiencies, we propose and evaluate SIGHT, a theoretical class of systems affording more robust signal evaluation during the job application process. A prototypical implementation of the SIGHT framework was evaluated using a mock-interview paradigm. Results provide initial evidence that SIGHT systems can elicit and capture qualification signals beyond what can be traditionally obtained from a typical application and that SIGHT systems can assess signals more effectively than unaided decision-making. SIGHT principles may extend to domains such as audit and security interviews

Using Narrative Disclosures to Detect Financial Fraud

This dissertation measures the information content in narrative financial disclosures to identify linguistic differences in manager and analyst language when fraud versus when it is not. The first chapter describes the motivation for this research and an overview of the research domain. Next, I review the literature covering textual analysis of narrative disclosures and present a heuristic and classification scheme for studies in this context. In Chapter 3, I compare the language across two common narrative disclosure types: quarterly earnings calls and the Management’s Discussion & Analysis (MD&A) section of quarterly and annual financial statements and find evidence of restricted incremental information from the CFOs of fraudulent companies. Chapter 4 uses a quasi-experiment to compare analyst the frequency and topics of analysts’ question during earnings calls. I find that relative to nonfraudulent firms, analysts ask the managers of fraudulent firms more questions overall, and are more persistent in asking questions as a call progresses. Chapter 5 is an exploratory study of dominance and linguistic style matching from managers and analysts when interacting in the question-and-answer portion of an earnings call. The dissertation concludes with a discussion of the work, limitations, and avenues for future research

Data Quality Relevance in Linguistic Analysis: The Impact of Transcription Errors on Multiple Methods of Linguistic Analysis

There is an enormous amount of recorded speech generated daily, and quickly transcribing and analyzing the text of this speech could have tremendous value to organizations and researchers. However, the speech transcription process has historically been laborious, expensive, and slow. Automatic speech recognition (ASR) tools have matured a great deal in the last decade and may be a suitable method to generate large scale, high quality transcriptions. These tools are are fast and economical, but generally produce errors at a much greater rate than human transcribers. It is unknown whether these errors matter when conducting psycholinguistic research. In this study, we will investigate the accuracy of earnings conference call transcripts produced by multiple tools and the impact of that transcription accuracy on the results of subsequent text mining analysis. While prior studies have focused on a single form of text mining, we will conduct three types of text analysis: bag-of-words based classification, lexicon-based classification and sentiment analysis. The results will show whether a different level of transcription quality is required for different types of text mining and the feasibility of using automated transcription services across a range of text mining applications

Does Accuracy Matter?: Methodological Considerations When Using Automated Speech-to-Text for Social Science Research

The analysis of spoken language has been integral to a breadth of research in social science and beyond. However, for analyses to occur with efficiency, language must be in the form of computer-readable text. Historically, the speech-to-text process has occurred manually using human transcriptionists. Automated speech recognition (ASR) is advertised as an efficient and inexpensive alternative, but research shows this method of speech-to-text is prone to error. This paper investigates the viability of using error prone ASR transcriptions as part of the methodological process of language analysis. Results show that at the individual feature level, analysis of ASR transcriptions differ dramatically from human transcriptions. However, when the same features are used for classification, a common machine learning task, performance results between ASR and human transcriptions are similar. We present these findings and conclude with a discussion on the methodological considerations for researchers who opt to use automated speech recognition for social science research

Table_4_Linguistic measures of personality in group discussions.docx

This investigation sought to find the relationships among multiple dimensions of personality and multiple features of language style. Unlike previous investigations, after controlling for such other moderators as culture and socio-demographics, the current investigation explored those dimensions of naturalistic spoken language that most closely align with communication. In groups of five to eight players, participants (N = 340) from eight international locales completed hour-long competitive games consisting of a series of ostensible missions. Composite measures of quantity, lexical diversity, sentiment, immediacy and negations were measured with an automated tool called SPLICE and with Linguistic Inquiry and Word Count. We also investigated style dynamics over the course of an interaction. We found predictors of extraversion, agreeableness, and neuroticism, but overall fewer significant associations than prior studies, suggesting greater heterogeneity in language style in contexts entailing interactivity, conversation rather than solitary message production, oral rather than written discourse, and groups rather than dyads. Extraverts were found to maintain greater linguistic style consistency over the course of an interaction. The discussion addresses the potential for Type I error when studying the relationship between language and personality.</p

Table_2_Linguistic measures of personality in group discussions.DOCX

This investigation sought to find the relationships among multiple dimensions of personality and multiple features of language style. Unlike previous investigations, after controlling for such other moderators as culture and socio-demographics, the current investigation explored those dimensions of naturalistic spoken language that most closely align with communication. In groups of five to eight players, participants (N = 340) from eight international locales completed hour-long competitive games consisting of a series of ostensible missions. Composite measures of quantity, lexical diversity, sentiment, immediacy and negations were measured with an automated tool called SPLICE and with Linguistic Inquiry and Word Count. We also investigated style dynamics over the course of an interaction. We found predictors of extraversion, agreeableness, and neuroticism, but overall fewer significant associations than prior studies, suggesting greater heterogeneity in language style in contexts entailing interactivity, conversation rather than solitary message production, oral rather than written discourse, and groups rather than dyads. Extraverts were found to maintain greater linguistic style consistency over the course of an interaction. The discussion addresses the potential for Type I error when studying the relationship between language and personality.</p

Data_Sheet_1_Linguistic measures of personality in group discussions.doc

This investigation sought to find the relationships among multiple dimensions of personality and multiple features of language style. Unlike previous investigations, after controlling for such other moderators as culture and socio-demographics, the current investigation explored those dimensions of naturalistic spoken language that most closely align with communication. In groups of five to eight players, participants (N = 340) from eight international locales completed hour-long competitive games consisting of a series of ostensible missions. Composite measures of quantity, lexical diversity, sentiment, immediacy and negations were measured with an automated tool called SPLICE and with Linguistic Inquiry and Word Count. We also investigated style dynamics over the course of an interaction. We found predictors of extraversion, agreeableness, and neuroticism, but overall fewer significant associations than prior studies, suggesting greater heterogeneity in language style in contexts entailing interactivity, conversation rather than solitary message production, oral rather than written discourse, and groups rather than dyads. Extraverts were found to maintain greater linguistic style consistency over the course of an interaction. The discussion addresses the potential for Type I error when studying the relationship between language and personality.</p

Table_1_Linguistic measures of personality in group discussions.XLSX

This investigation sought to find the relationships among multiple dimensions of personality and multiple features of language style. Unlike previous investigations, after controlling for such other moderators as culture and socio-demographics, the current investigation explored those dimensions of naturalistic spoken language that most closely align with communication. In groups of five to eight players, participants (N = 340) from eight international locales completed hour-long competitive games consisting of a series of ostensible missions. Composite measures of quantity, lexical diversity, sentiment, immediacy and negations were measured with an automated tool called SPLICE and with Linguistic Inquiry and Word Count. We also investigated style dynamics over the course of an interaction. We found predictors of extraversion, agreeableness, and neuroticism, but overall fewer significant associations than prior studies, suggesting greater heterogeneity in language style in contexts entailing interactivity, conversation rather than solitary message production, oral rather than written discourse, and groups rather than dyads. Extraverts were found to maintain greater linguistic style consistency over the course of an interaction. The discussion addresses the potential for Type I error when studying the relationship between language and personality.</p