Doctor of Philosophy

dissertationLeft-right (LR) determination during embryogenesis is critical for the correct positioning of the various visceral organs, such as the lungs and the heart. In establishing LR asymmetry in the mouse, the initial LR signal that is determined in the ciliated embryonic node is transferred to the lateral plate mesoderm (LPM). The cellular and molecular mechanisms for this signal transfer have not been well characterized. Here, we studied the role of endoderm cells in this process by analyzing mouse Sox17 null mutant embryos as a model that develops endoderm-specific defects. Sox17 mutant embryos showed no expression or significantly reduced expression of LR asymmetric genes in the left LPM. A series of experiments revealed the importance of intercellular communication through gap junctions in endoderm cells for LR signal transfer from the node. We also found that SOX17 function is essential not only to form complete epithelial structures that express CX43 and localize connexin proteins on the cell membrane but also to turn off Nodal gene expression in differentiating endoderm cells through a Nodal endoderm-specific enhancer

Computer vision

The field of computer vision is surveyed and assessed, key research issues are identified, and possibilities for a future vision system are discussed. The problems of descriptions of two and three dimensional worlds are discussed. The representation of such features as texture, edges, curves, and corners are detailed. Recognition methods are described in which cross correlation coefficients are maximized or numerical values for a set of features are measured. Object tracking is discussed in terms of the robust matching algorithms that must be devised. Stereo vision, camera control and calibration, and the hardware and systems architecture are discussed

Modelling the relationship between gesture motion and meaning

There are many ways to say “Hello,” be it a wave, a nod, or a bow. We greet others not only with words, but also with our bodies. Embodied communication permeates our interactions. A fist bump, thumbs-up, or pat on the back can be even more meaningful than hearing “good job!” A friend crossing their arms with a scowl, turning away from you, or stiffening up can feel like a harsh rejection. Social communication is not exclusively linguistic, but is a multi-sensory affair. It’s not that communication without these bodily cues is impossible, but it is impoverished. Embodiment is a fundamental human experience. Expressing ourselves through our bodies provides a powerful channel through which we express a plethora of meta-social information. And integral to communication, expression, and social engagement is our utilization of conversational gesture. We use gestures to express extra-linguistic information, to emphasize our point, and to embody mental and linguistic metaphors that add depth and color to social interaction. The gesture behaviour of virtual humans when compared to human-human conversation is limited, depending on the approach taken to automate performances of these characters. The generation of nonverbal behaviour for virtual humans can be approximately classified as either: 1) data-driven approaches that learn a mapping from aspects of the verbal channel, such as prosody, to gestures; or 2) rule bases approaches that are often tailored by designers for specific applications. This thesis is an interdisciplinary exploration that bridges these two approaches, and brings data-driven analyses to observational gesture research. By marrying a rich history of gesture research in behavioral psychology with data-driven techniques, this body of work brings rigorous computational methods to gesture classification, analysis, and generation. It addresses how researchers can exploit computational methods to make virtual humans gesture with the same richness, complexity, and apparent effortlessness as you and I. Throughout this work the central focus is on metaphoric gestures. These gestures are capable of conveying rich, nuanced, multi-dimensional meaning, and raise several challenges in their generation, including establishing and interpreting a gesture’s communicative meaning, and selecting a performance to convey it. As such, effectively utilizing these gestures remains an open challenge in virtual agent research. This thesis explores how metaphoric gestures are interpreted by an observer, how one can generate such rich gestures using a mapping between utterance meaning and gesture, as well as how one can use data driven techniques to explore the mapping between utterance and metaphoric gestures. The thesis begins in Chapter 1 by outlining the interdisciplinary space of gesture research in psychology and generation in virtual agents. It then presents several studies that address presupposed assumptions raised about the need for rich, metaphoric gestures and the risk of false implicature when gestural meaning is ignored in gesture generation. In Chapter 2, two studies on metaphoric gestures that embody multiple metaphors argue three critical points that inform the rest of the thesis: that people form rich inferences from metaphoric gestures, these inferences are informed by cultural context and, more importantly, that any approach to analyzing the relation between utterance and metaphoric gesture needs to take into account that multiple metaphors may be conveyed by a single gesture. A third study presented in Chapter 3 highlights the risk of false implicature and discusses this in the context of current subjective evaluations of the qualitative influence of gesture on viewers. Chapters 4 and 5 then present a data-driven analysis approach to recovering an interpretable explicit mapping from utterance to metaphor. The approach described in detail in Chapter 4 clusters gestural motion and relates those clusters to the semantic analysis of associated utterance. Then, Chapter 5 demonstrates how this approach can be used both as a framework for data-driven techniques in the study of gesture as well as form the basis of a gesture generation approach for virtual humans. The framework used in the last two chapters ties together the main themes of this thesis: how we can use observational behavioral gesture research to inform data-driven analysis methods, how embodied metaphor relates to fine-grained gestural motion, and how to exploit this relationship to generate rich, communicatively nuanced gestures on virtual agents. While gestures show huge variation, the goal of this thesis is to start to characterize and codify that variation using modern data-driven techniques. The final chapter of this thesis reflects on the many challenges and obstacles the field of gesture generation continues to face. The potential for applications of Virtual Agents to have broad impacts on our daily lives increases with the growing pervasiveness of digital interfaces, technical breakthroughs, and collaborative interdisciplinary research efforts. It concludes with an optimistic vision of applications for virtual agents with deep models of non-verbal social behaviour and their potential to encourage multi-disciplinary collaboration

Belief Representations for Planning with Contact Uncertainty

While reaching for your morning coffee you may accidentally bump into the table, yet you reroute your motion with ease and grab your cup. An effective autonomous robot will need to have a similarly seamless recovery from unexpected contact. As simple as this may seem, even sensing this contact is a challenge for many robots, and when detected contact is often treated as an error that an operator is expected to resolve. Robots operating in our daily environments will need to reason about the information they have gained from contact and replan autonomously. This thesis examines planning under uncertainty with contact sensitive robot arms. Robots do not have skin and cannot precisely sense the location of contact. This leads to the proposed Collision Hypothesis Set model for representing a belief over the possible occupancy of the world sensed through contact. To capture the specifics of planning in an unknown world with this measurement model, this thesis develops a POMDP approach called the Blindfolded Traveler's Problem. A good prior over the possible obstacles the robot might encounter is key to effective planning. This thesis develops a neural network approach for sampling potential obstacles that are consistent with both what a robot sees from its camera and what it feels through contact.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169845/1/bsaund_1.pd

The Role of Knowledge in Visual Shape Representation

This report shows how knowledge about the visual world can be built into a shape representation in the form of a descriptive vocabulary making explicit the important geometrical relationships comprising objects' shapes. Two computational tools are offered: (1) Shapestokens are placed on a Scale-Space Blackboard, (2) Dimensionality-reduction captures deformation classes in configurations of tokens. Knowledge lies in the token types and deformation classes tailored to the constraints and regularities ofparticular shape worlds. A hierarchical shape vocabulary has been implemented supporting several later visual tasks in the two-dimensional shape domain of the dorsal fins of fishes

Implementation and Application of Genomic Association Methods to Clostridium Difficile Toxicity and Clinical Infection Outcomes

Clostridium difficile is a major cause of healthcare-associated infections in the United States. A C. difficile infection can lead to a range of outcomes including diarrhea, intensive care unit admission, abdominal surgery, or death. Pathogenesis is mediated by the release of toxin from C. difficile cells growing in the intestines. Some patients are more vulnerable to infection, including those with previous antibiotic exposure and advanced age. Host factors can affect the likelihood of infection but also the severity of infection. Additionally, infection severity can be influenced by the genome of the infecting strain(s). Host-pathogen interactions are extremely complex and very little is known about the interplay between host factors and C. difficile genomic variation with respect to infection likelihood and outcomes. With the recent deluge of whole genome sequencing data, the contribution of bacterial genomic variation to infections can be more comprehensively evaluated than ever before. The work described in this dissertation used two different approaches to test for associations between C. difficile genomic variation and clinically relevant phenotypes. In the first approach we implemented and applied a novel convergence-based bacterial genome-wide association study (bGWAS) algorithm for quantitative traits. We introduce the algorithm using a set of data generated in silico to realistically model bacterial genome variation and phenotypes under various evolutionary regimes. When the algorithm was applied to C. difficile genomic variants and toxin activity our bGWAS identified known toxin regulatory genes associated with toxin activity, supporting the value of our approach. Besides identifying key cis-regulatory variants in the toxin-producing locus, we observed several associations that connect toxin activity to a complex network of trans-regulatory genes. Many highly associated variants occur in flagellar genes and indicate coregulation of toxicity and motility. We propose new variants associated with toxin activity for future functional validation. This study focused on a complex phenotype, toxin activity, within a highly controlled in vitro system. We next investigated the impact of bacterial genetic variation on human infections. The increased complexity of this human-pathogen interaction justified a different association approach to better understand the independent contribution of bacterial genomic variation to infection. In a set of clinically derived isolates, we tested for the association between variants in trehalose metabolism operons and infection severity while incorporating and controlling for infection severity-modulating patient characteristics. Trehalose utilization variants were recently proposed to modulate C. difficile infections in a mouse model. Interestingly, we observed that this in vivo result did not translate to our clinical cohort as we found no evidence of an association between any of the trehalose utilization variants and patient infection outcomes. Taken together, these results demonstrate the utility of applying multiple approaches for identifying genomic variants associated with clinical outcomes that account for either bacterial population structure or host factors.PHDMicrobiology & ImmunologyUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/166125/1/katiephd_1.pd

CMCF: An Architecture for Realtime Gesture Generation by Clustering Gestures by Motion and Communicative Function

Gestures augment speech by performing a variety of communicative functions in humans and virtual agents, and are often related to speech by complex semantic, rhetorical, prosodic, and affective elements. In this paper we briefly present an architecture for human-like gesturing in virtual agents that is designed to realize complex speech-to-gesture mappings by exploiting existing machine-learning based parsing tools and techniques to extract these functional elements from speech. We then deeply explore the rhetorical branch of this architecture, objectively assessing specifically whether existing rhetorical parsing techniques can classify gestures into classes with distinct movement properties. To do this, we take a corpus of spontaneously generated gestures and correlate their movement to co-speech utterances. We cluster gestures based on their rhetorical properties, and then by their movement. Our objective analysis suggests that some rhetorical structures are identifiable by our movement features while others require further exploration. We explore possibilities behind these findings and propose future experiments that may further reveal nuances of the richness of the mapping between speech and motion. This work builds towards a real-time gesture generator which performs gestures that effectively convey rich communicative functions