Multimodal Interactive Parsing

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-38628-2_57Probabilistic parsing is a fundamental problem in Computational Linguistics, whose goal is obtaining a syntactic structure associated to a sentence according to a probabilistic grammatical model. Recently, an interactive framework for probabilistic parsing has been introduced, in which the user and the system cooperate to generate error-free parse trees. In an early prototype developed according to this interactive parsing technology, user feedback was provided by means of mouse actions and keyboard strokes. Here we augment the interaction style with support for (non-deterministic) natural handwritten recognition, and provide confidence measures as a visual aid to ease the correction process. Handwriting input seems to be a modality specially suitable for parsing, since the vocabulary size involved in the recognition of syntactic labels is fairly limited and thus intuitively errors should be small. However, errors may increase as handwriting quality (i.e., calligraphy) degrades. To solve this problem, we introduce a late fusion approach that leverages both on-line and off-line information, corresponding to pen strokes and contextual information from the parse trees. We demonstrate that late fusion can effectively help to disambiguate user intention and improve system accuracy.This research has received funding from the EC’s 7th Framework Programme (FP7/2007-13) under grant agreement No.287576- CasMaCat; from the Spanish MEC under the STraDA project (TIN2012-37475- C02-01) and the MITTRAL project (TIN2009-14633-C03-01); from the GV under the Prometeo project; and from the Universidad del Cauca (Colombia)Benedí Ruiz, JM.; Sánchez Peiró, JA.; Leiva, LA.; Sánchez Sáez, R.; Maca, M. (2013). Multimodal Interactive Parsing. En Pattern Recognition and Image Analysis. Springer. 484-491. https://doi.org/10.1007/978-3-642-38628-2_57S484491Afonso, S., Bick, E., Haber, R., Santos, D.: Floresta sintá(c)tica: a treebank for portuguese. In: Proc. LREC, pp. 1698–1703 (2002)Brants, T., Plaehn, O.: Interactive corpus annotation. In: Proc. LREC (2000)Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: UNIPEN project of on-line data exchange and recognizer benchmarks. In: Proc. ICPR, pp. 29–33 (1994)Lease, M., Charniak, E., Johnson, M., McClosky, D.: A look at parsing and its applications. In: Proc. AAAI, pp. 1642–1645 (2006)Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19(2), 313–330 (1993)Ortiz, D., Leiva, L.A., Alabau, V., Casacuberta, F.: Interactive machine translation using a web-based architecture. In: Proc. IUI, pp. 423–425 (2010)Romero, V., Leiva, L.A., Toselli, A.H., Vidal, E.: Interactive multimodal transcription of text images using a web-based demo system. In: Proc. IUI, pp. 477–478 (2009)Sánchez-Sáez, R., Leiva, L.A., Sánchez, J.A., Benedí, J.M.: Interactive predictive parsing using a web-based architecture. In: Proc. NAACL-HLT, pp. 37–40 (2010)Sánchez-Sáez, R., Sánchez, J.A., Benedí, J.M.: Interactive predictive parsing. In: Proc. IWPT, pp. 222–225 (2009)Sánchez-Sáez, R., Sánchez, J.A., Benedí, J.M.: Confidence measures for error discrimination in an interactive predictive parsing framework. In: Proc. COLING, pp. 1220–1228 (2010

Locus of Control in L2 English Listening Assessment

In second language (L2) listening assessment, various factors have the potential to impact the validity of listening test items (Brindley & Slatyer, 2002; Buck & Tatsuoka, 1998; Freedle & Kostin, 1999; Nissan, DeVincenzi, & Tang, 1996; Read, 2002; Shohamy & Inbar, 1991). One relatively unexplored area to date is who controls the aural input. In traditional standardized listening tests, an administrator controlled recording is played once or twice. In real world or classroom listening, however, listeners can sometimes request repetition or clarification. Allowing listeners to control the aural input thus has the potential to add test authenticity but requires careful design of the input and expected response as well as an appropriate computer interface. However, if candidates feel less anxious, allowing control of listening input may enhance examinees\u27 experience and still reflect their listening proficiency. Comparing traditional and self paced (i.e., examinees having the opportunity to start, stop, and move the audio position) delivery of multiple choice comprehension items, my research inquiry is whether self paced listening can be a sufficiently reliable and valid measure of examinees\u27 listening ability. Data were gathered from 100 prospective and current university ESL students. They were administered computer based multiple choice listening tests: 10 identical once played items, followed by 33 items in three different conditions: 1) administrator paced input with no audio player visible, 2) self paced with a short time limit, and 3) self paced with a longer time limit. Many facet Rasch (1960/1980) modeling was used to compare the difficulty and discrimination of the items across conditions. Results indicated items on average were similar difficulty overall but discriminated best in self paced conditions. Furthermore, the vast majority of examinees reported they preferred self paced listening. The quantitative results were complemented by follow up stimulated recall interviews with eight participants who took 22 additional test items using screen capture software to explore whether and when they paused and/or repeated the input. Frequency of and reasons for self pacing did not follow any particular pattern by proficiency level. Examinees tended to play more than once but not two full times through, even without limited time. Implications for listening instruction and classroom assessment, as well as standardized testing, are discussed

Error Signals from the Brain: 7th Mismatch Negativity Conference

The 7th Mismatch Negativity Conference presents the state of the art in methods, theory, and application (basic and clinical research) of the MMN (and related error signals of the brain). Moreover, there will be two pre-conference workshops: one on the design of MMN studies and the analysis and interpretation of MMN data, and one on the visual MMN (with 20 presentations). There will be more than 40 presentations on hot topics of MMN grouped into thirteen symposia, and about 130 poster presentations. Keynote lectures by Kimmo Alho, Angela D. Friederici, and Israel Nelken will round off the program by covering topics related to and beyond MMN

Social Measurement and Causal Inference with Text

The digital age has dramatically increased access to large-scale collections of digitized text documents. These corpora include, for example, digital traces from social media, decades of archived news reports, and transcripts of spoken interactions in political, legal, and economic spheres. For social scientists, this new widespread data availability has potential for improved quantitative analysis of relationships between language use and human thought, actions, and societal structure. However, the large-scale nature of these collections means that traditional manual approaches to analyzing content are extremely costly and do not scale. Furthermore, incorporating unstructured text data into quantitative analysis is difficult due to texts’ high-dimensional nature and linguistic complexity. This thesis blends (a) the computational strengths of natural language processing (NLP) and machine learning to automate and scale-up quantitative text analysis with (b) two themes central to social scientific studies but often under-addressed in NLP: measurement—creating quantifiable summaries of empirical phenomena—and causal inference—estimating the effects of interventions. First, we address measuring class prevalence in document collections; we contribute a generative probabilistic modeling approach to prevalence estimation and show empirically that our model is more robust to shifts in class priors between training and inference. Second, we examine cross- document entity-event measurement; we contribute an empirical pipeline and a novel latent disjunction model to identify the names of civilians killed by police from our corpus of web-scraped news reports. Third, we gather and categorize applications that use text to reduce confounding from causal estimates and contribute a list of open problems as well as guidance about data processing and evaluation decisions in this area. Finally, we contribute a new causal research design to estimate the natural indirect and direct effects of social group signals (e.g. race or gender) on conversational outcomes with separate aspects of language as causal mediators; this chapter is motivated by a theoretical case study of U.S. Supreme Court oral arguments and the effect of an advocate’s gender on interruptions from justices. We conclude by discussing the relationship between measurement and causal inference with text and future work at this intersection

JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction

Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Java’s reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the class’s method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally

Advances in Interactive Speech Transcription

[ES] Novedoso sistema interactivo para la transcripción del habla que compensa el esfuerzo del usuario y el error máximo tolerado en las transcripciones resultantes.[EN] Novel interactive speech transcription system that balances the user effort and the maximum allowed error tolerated for the final resulting transcriptions.Sánchez Cortina, I. (2012). Advances in Interactive Speech Transcription. http://hdl.handle.net/10251/17889Archivo delegad

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

SPLIT DECISIONS: PRACTICAL MACHINE LEARNING FOR EMPIRICAL LEGAL SCHOLARSHIP

Multivariable regression may be the most prevalent and useful task in social science. Empirical legal studies rely heavily on the ordinary least squares method. Conventional regression methods have attained credibility in court, but by no means do they dictate legal outcomes. Using the iconic Boston housing study as a source of price data, this Article introduces machine-learning regression methods. Although decision trees and forest ensembles lack the overt interpretability of linear regression, these methods reduce the opacity of black-box techniques by scoring the relative importance of dataset features. This Article will also address the theoretical tradeoff between bias and variance, as well as the importance of training, cross-validation, and reserving a holdout dataset for testing