The Evolution of Assessment: Learning about Culture from a Serious Game

In ill-defined domains, properly assessing learning is, itself, an illdefined problem. Over the last several years, the domain of interest to us has been teaching Americans about Iraqi business culture via a serious-game-based practice environment. We describe this system and the various measures we used in a series of studies to assess its ability to teach. As subsequent studies identified the limits of each measure, we selected additional measures that would let us better understand what and how people were learning, using Bloom’s revised taxonomy as a guide. We relate these and other lessons we learned in the process of refining our solution to this ill-defined problem.</p

Investigating the Influence of Virtual Peers as Dialect Models on Students’ Prosodic Inventory

<p>Children who speak non-standard dialects of English show reduced performance not just in language-oriented topics in school but also in math and science. Technological solutions have been rare exactly because of the nonmainstream nature of their talk, and hence the difficulty in automatically recognizing their speech and responding to it with, for example, computer tutors. In order to work towards overcoming this achievement gap, in this work we investigate African American students’ prosodic inventories in different contexts as a first-step towards building a system that will be able to automatically recognize, and respond to, the dialect in which a child is speaking. We presented children with recordings of a peer (confederate) speaking in either African American English (AAE) or Mainstream American English (MAE) during both a social task and a science task. We found that children showed decreased prosodic variation and peak slopes during speech segments which did not contain AAE features, resulting in more monotone and breathy utterances than when they are speaking in AAE. We also found that children who were speaking with a “peer” who uses AAE have increased articulation rates, energy, and pitch variation. We discuss potential interpretations of these results that are important to the design of a system to support linguistic diversity and decrease the achievement gap.</p

“Love ya, jerkface”: using Sparse Log-Linear Models to Build Positive (and Impolite) Relationships with Teens

<p>One challenge of implementing spoken dialogue systems for long-term interaction is how to adapt the dialogue as user and system become more familiar. We believe this challenge includes evoking and signaling aspects of long-term relationships such as rapport. For tutoring systems, this may additionally require knowing how relationships are signaled among non-adult users. We therefore investigate conversational strategies used by teenagers in peer tutoring dialogues, and how these strategies function differently among friends or strangers. In particular, we use annotated and automatically extracted linguistic devices to predict impoliteness and positivity in the next turn. To take into account the sparse nature of these features in real data we use models including Lasso, ridge estimator, and elastic net. We evaluate the predictive power of our models under various settings, and compare our sparse models with standard non-sparse solutions. Our experiments demonstrate that our models are more accurate than non-sparse models quantitatively, and that teens use unexpected kinds of language to do relationship work such as signaling rapport, but friends and strangers, tutors and tutees, carry out this work in quite different ways from one another.</p