Discovering a Domain Knowledge Representation for Image Grouping: Multimodal Data Modeling, Fusion, and Interactive Learning

In visually-oriented specialized medical domains such as dermatology and radiology, physicians explore interesting image cases from medical image repositories for comparative case studies to aid clinical diagnoses, educate medical trainees, and support medical research. However, general image classification and retrieval approaches fail in grouping medical images from the physicians\u27 viewpoint. This is because fully-automated learning techniques cannot yet bridge the gap between image features and domain-specific content for the absence of expert knowledge. Understanding how experts get information from medical images is therefore an important research topic. As a prior study, we conducted data elicitation experiments, where physicians were instructed to inspect each medical image towards a diagnosis while describing image content to a student seated nearby. Experts\u27 eye movements and their verbal descriptions of the image content were recorded to capture various aspects of expert image understanding. This dissertation aims at an intuitive approach to extracting expert knowledge, which is to find patterns in expert data elicited from image-based diagnoses. These patterns are useful to understand both the characteristics of the medical images and the experts\u27 cognitive reasoning processes. The transformation from the viewed raw image features to interpretation as domain-specific concepts requires experts\u27 domain knowledge and cognitive reasoning. This dissertation also approximates this transformation using a matrix factorization-based framework, which helps project multiple expert-derived data modalities to high-level abstractions. To combine additional expert interventions with computational processing capabilities, an interactive machine learning paradigm is developed to treat experts as an integral part of the learning process. Specifically, experts refine medical image groups presented by the learned model locally, to incrementally re-learn the model globally. This paradigm avoids the onerous expert annotations for model training, while aligning the learned model with experts\u27 sense-making

Visual-Linguistic Semantic Alignment: Fusing Human Gaze and Spoken Narratives for Image Region Annotation

Advanced image-based application systems such as image retrieval and visual question answering depend heavily on semantic image region annotation. However, improvements in image region annotation are limited because of our inability to understand how humans, the end users, process these images and image regions. In this work, we expand a framework for capturing image region annotations where interpreting an image is influenced by the end user\u27s visual perception skills, conceptual knowledge, and task-oriented goals. Human image understanding is reflected by individuals\u27 visual and linguistic behaviors, but the meaningful computational integration and interpretation of their multimodal representations (e.g. gaze, text) remain a challenge. Our work explores the hypothesis that eye movements can help us understand experts\u27 perceptual processes and that spoken language descriptions can reveal conceptual elements of image inspection tasks. We propose that there exists a meaningful relation between gaze, spoken narratives, and image content. Using unsupervised bitext alignment, we create meaningful mappings between participants\u27 eye movements (which reveal key areas of images) and spoken descriptions of those images. The resulting alignments are then used to annotate image regions with concept labels. Our alignment accuracy exceeds baseline alignments that are obtained using both simultaneous and a fixed-delay temporal correspondence. Additionally, comparison of alignment accuracy between a method that identifies clusters in the images based on eye movements and a method that identifies clusters using image features shows that the two approaches perform well on different types of images and concept labels. This suggests that an image annotation framework could integrate information from more than one technique to handle heterogeneous images. The resulting alignments can be used to create a database of low-level image features and high-level semantic annotations corresponding to perceptually important image regions. We demonstrate the applicability of the proposed framework with two datasets: one consisting of general-domain images and another with images from the domain of medicine. This work is an important contribution toward the highly challenging problem of fusing human-elicited multimodal data sources, a problem that will become increasingly important as low-resource scenarios become more common

Metacognition and Decision-Making Style in Clinical Narratives

Clinical decision-making has high-stakes outcomes for both physicians and patients, yet little research has attempted to model and automatically annotate such decision-making. The dual process model (Evans, 2008) posits two types of decision-making, which may be ordered on a continuum from intuitive to analytical (Hammond, 1981). Training clinicians to recognize decision-making style and select the most appropriate mode of reasoning for a particular context may help reduce diagnostic error (Norman, 2009). This study makes preliminary steps towards detection of decision style, based on an annotated dataset of image-based clinical reasoning in which speech data were collected from physicians as they inspected images of dermatological cases and moved towards diagnosis (Hochberg et al., 2014a). A classifier was developed based on lexical, speech, disfluency, physician demographic, cognitive, and diagnostic difficulty features to categorize diagnostic narratives as intuitive vs. analytical; the model improved on the baseline by over 30%. The introduced computational model provides construct validity for the dual process theory. Eventually, such modeling may be incorporated into instructional systems that teach clinicians to become more effective decision makers. In addition, metacognition, or self-assessment and self-management of cognitive processes, has been shown beneficial to decision-making (Batha & Carroll, 2007; Ewell-Kumar, 1999). This study measured physicians\u27 metacognitive awareness, an online component of metacognition, based on the confidence-accuracy relationship, and also exploited the corpus annotation of decision style to derive decision metrics. These metrics were used to examine the relationships between decision style, metacognitive awareness, expertise, case difficulty, and diagnostic accuracy. Based on statistical analyses, intuitive reasoning was associated with greater diagnostic accuracy, with an advantage for expert physicians. Case difficulty was associated with greater user of analytical decision-making, while metacognitive awareness was linked to decreased diagnostic accuracy. These results offer a springboard for further research on the interactions between decision style, metacognitive awareness, physician and case characteristics, and diagnostic accuracy

Conceptual Metaphor in the Health Care Culture

The conceptual metaphor has meaning only when understood within the cultural framework which gives rise to the conceptualization. The purpose of this study was to investigate the interaction of cognition (conceptual metaphor) and culture as manifest during intercultural communication in teaching-learning sessions between health care providers and patients. An ethnography of communication (Hymes, 1974; Saville-Troike, 1989) was the method employed to investigate the use of metaphor by patients, nurses and other health care professionals. Patients were viewed as a sojourner group in the health care culture; nurses and their health care partners were seen as a host group. Data were collected during a six month period using participant observation and key informant interviews with groups of sojourners/patients and a host/staff group consisting of six different health care specialties. The communication setting was an outpatient diabetic education program for those with a new diagnosis or whose condition had recently become unstable. The duration of the education program was six sessions, with variable participant attendance rates. Ethnographic findings indicated that the communication of each of the two groups presented a variety of distinctive features, as well as shared features. The sojourner group communication events and acts included the creation of pre- and post-sessions, the creation of personal narratives, and the practice of stopping the communication. Hosts also generated distinct communication events and acts which were stand-alone sessions, the use of a lecture format, a minimized response to the sojourner narratives, as well as confrontation of non-adherent sojourners. They shared several constructs and meanings in the use of several metaphoric domains, as well as the use of the machine metaphor of control (Ting-Toomey, 1987). Both groups also exhibited instances of parallel meanings in regard to the metaphor they used. The two groups shared many of the same source and target domains but some were incongruently interpreted by the groups. The findings have implications for future research into use of machine metaphors in health care communication, as well as implications for those health professionals who implement patient teaching to become more cognizant that their metaphors should be examined for effectiveness. Health communicators, who plan and implement programs, need to recognize that health communication may be more effective when they create a communication partnership toward encouraging the “voice” of patients in the process

Framing university small group talk : knowledge construction through lexical concepts

PhD ThesisKnowledge construction in educational discourse continues to interest practitioners and researchers due to the conceptually “natural” connection between knowledge and learning for professional development. Frames have conceptual and practical advantages over other units of inquiry concerning meaning negotiation for knowledge construction. They are relatively stable data-structures representing prototypical situations retrieved from real world experiences, cover larger units of meaning beyond the immediate sequential mechanism at interaction, and have been inherently placed at the semantic-pragmatic interface for empirical observation. Framing in a particular context – university small group talk has been an under-researched field, while the relationship between talk and knowledge through collaborative work has been identified below/at the Higher Educational level. Involving higher level cognitive activities and distinct interactional patterns, university small group talk is worth close examination and systematic investigation. This study applies Corpus Linguistics and Interactional Linguistics approaches to examine a subset of a one-million-word corpus of university small group talk at a UK university. Specifically, it provides a detailed examination of the participants’ framing behaviours for knowledge construction through their talk of disciplinary lexical concepts. Analysis reveals how the participants draw upon schematized knowledge structures evoked by particular lexical choices and how they invoke expanded scenarios via pragmatic mappings in the ongoing interaction. Additionally, it is demonstrated how the framing moves are related to the structural uniqueness of university small group talk, the contextualized speaker roles and the institutional procedures and routines. This study deepens the understanding of the relationship between linguistically constructed knowledge and the way interlocutors conceptualize the world through institutionalized collaboration, building upon the existing research on human reliance upon structures to interpret reality at both the conceptual and the action levels. The study also addresses interaction research in Higher Educational settings, by discussing how the cognitive-communicative duality of framing is sensitive to various contextual resources, distinct discourse structures and task procedures through the group dynamics

The Fascinating World of Widespread Idioms: A Tribute to Elisabeth Piirainen (1943-2017).

I dedicate this review essay to the memory of the world-renowned researcher into international phraseology Elisabeth Piirainen, who left this world unexpectedly on 29 December 2017. Her outstanding scholarship and unmatched dedication to phraseology will serve as an inspiration to many present and future scholars. May her soul rest in peace! Widespread Idioms in Europe and Beyond. Toward a Lexi-con of Common Figurative Units. By Elisabeth Piirainen. New York, Washington, D.C./Baltimore, Bern, Frankfurt, Berlin, Brussels, Vienna, Oxford: Peter Lang, 2012. Pp. 591; Lexicon of Common Figurative Units. Widespread Idioms in Europe and Beyond. Volume II. By Elisabeth Piirainen. In cooperation with József Attila Balázsi. New York, Bern, Frankfurt, Berlin, Brus-sels, Vienna, Oxford, Warsaw: Peter Lang, 2016. Pp. 776

The Fascinating World of Widespread Idioms: A Tribute to Elisabeth Piirainen (1943-2017).

I dedicate this review essay to the memory of the world-renowned researcher into international phraseology Elisabeth Piirainen, who left this world unexpectedly on 29 December 2017. Her outstanding scholarship and unmatched dedication to phraseology will serve as an inspiration to many present and future scholars. May her soul rest in peace! Widespread Idioms in Europe and Beyond. Toward a Lexi-con of Common Figurative Units. By Elisabeth Piirainen. New York, Washington, D.C./Baltimore, Bern, Frankfurt, Berlin, Brussels, Vienna, Oxford: Peter Lang, 2012. Pp. 591; Lexicon of Common Figurative Units. Widespread Idioms in Europe and Beyond. Volume II. By Elisabeth Piirainen. In cooperation with József Attila Balázsi. New York, Bern, Frankfurt, Berlin, Brus-sels, Vienna, Oxford, Warsaw: Peter Lang, 2016. Pp. 776

A Linguistic Analysis of the Entrepreneurial Pitch

An entrepreneur faces many challenges in the quest to bring an idea to fruition, and the first step in gaining financial support most commonly comes in the form of a pitch. The goal is always ultimately to persuade investors in the validity and profitability of a product or idea. A wealth of information currently exists in helping entrepreneurs create pitch content as well as giving advice on presentation skills. While this information is important for consideration, the vast majority of available knowledge hovers in the intuitive realm. Little to no quantitative, academic research exists on the actual use of language in an oral presentation. Therefore, this thesis intended to conduct a micro linguistic analysis of the entrepreneurial pitch in order to determine what linguistic devices were employed at different textual levels within the pitch that may have a yet unrecognized, but significant, impact on the decisions of investors. Borrowing theory developed by scholars like Schiffrin, Widdowson, and Lakoff and Johnson, an analysis of the syntactic structure of fourteen pitches at the word, sentence, and meta level was conducted with the intention of producing a general rubric influenced by linguistic theories and research. This study adds new insights to the growing body of research by showing what linguistic elements of an entrepreneurial pitch may affect the persuasiveness of a pitch. Advice is given concerning specific and general elements that had a quantifiable effect in the study and are useful for an entrepreneur to be aware of when crafting a pitch