Notes from the ISMIR 2012 late-breaking session on evaluation in music information retrieval

During the last day of the ISMIR 2012 conference there were two events related to Music IR Evaluation. A panel took place during the morning to discuss several issues concerning the various evaluation initiatives with the general audience at ISMIR. A late-breaking session during the afternoon kept the discussion alive between a group of researchers who wanted to dig deeper into these issues. This extended abstract reports the main topics covered during this short session and the general thoughts that came up

Summary of the nondestructive evaluation session

Advanced material systems and designs require improved nondestructive evaluation (NDE) techniques for quality control, failure analysis and properties monitoring. Some advanced NDE techniques such as infrared scanning systems, laser holographic applications and the use of the Mossbauer effect are outlined. Applications of these techniques are discussed for evaluating the new space shuttle external insulation materials, aircraft structures and engine components, and precursors to crack initiation and growth

Same query - different results? A study of repeat queries in search sessions

Typically, three main query reformulation types in sessions are considered: generalization, specication, and drift. We show that given the full context of user interactions, repeat queries represent an important reformulation type which should also be addressed in session retrieval evaluation. We investigate dierent query reformulation patterns in logs from The European Library. Using an automatic classification for query reformulations, we found that the most frequent (and presumably the most important) reformulation pattern corresponds to repeat queries. We aim to nd possible explanations for repeat queries in sessions and try to uncover implications for session retrieval evaluation

Session Types in a Linearly Typed Multi-Threaded Lambda-Calculus

We present a formalization of session types in a multi-threaded lambda-calculus (MTLC) equipped with a linear type system, establishing for the MTLC both type preservation and global progress. The latter (global progress) implies that the evaluation of a well-typed program in the MTLC can never reach a deadlock. As this formulated MTLC can be readily embedded into ATS, a full-fledged language with a functional programming core that supports both dependent types (of DML-style) and linear types, we obtain a direct implementation of session types in ATS. In addition, we gain immediate support for a form of dependent session types based on this embedding into ATS. Compared to various existing formalizations of session types, we see the one given in this paper is unique in its closeness to concrete implementation. In particular, we report such an implementation ready for practical use that generates Erlang code from well-typed ATS source (making use of session types), thus taking great advantage of the infrastructural support for distributed computing in Erlang.Comment: This is the original version of the paper on supporting programming with dyadic session types in AT

A pilot study of the Italian adaptation of the Session Evaluation Questionnaire fourth version

Abstract The Session Evaluation Questionnaire (SEQ) evaluates the impact of counselling and psychotherapy sessions; it may be conceived as a bridge between psychotherapy process and outcome. SEQ is a self-report tool asking patients about their experience with the clinical session just ended; it consists of 27 adjectives in semantic differential scale, divided into three thematic parts: evaluation of the session itself, feelings after the session, and evaluation of the therapist. Even if the original American SEQ has been translated into many languages, only a few translations have been validated. This is a pilot study, which attempted to replicate the five-dimensional structure of the fourth version of the Anglo-American SEQ, for the Italian population. Data were collected on 111 outpatients attending the Psychological Assistance Service (SAP-Dynamic Psychotherapy Service) of Padua University, after their first two clinical interviews. Exploratory factor analyses (PCA with varimax rotation) were performed on each of the three parts of the SEQ. Results confirmed the original factorial structure, showing a substantial agree for Depth, Smoothness, Positivity and Arousal dimensions; Good Therapist dimension overlapped perfectly with the original one. The Italian SEQ showed adequate internal consistency (Cronbach\u2019s alphas ranged .65 to .86 for the five subscales). Convergent validity measured with an index of perceived satisfaction in the counselling process, showed significant positive correlations with Good Therapist, Depth and Positivity scores. The preliminary results of this pilot study show that the Italian SEQ results in a reliable instrument to measure the impact of clinical sessions. Of course, validation studies are needed, especially to replicate the factor structure of the instrument and to better assess its validity

Predicting Session Length in Media Streaming

Session length is a very important aspect in determining a user's satisfaction with a media streaming service. Being able to predict how long a session will last can be of great use for various downstream tasks, such as recommendations and ad scheduling. Most of the related literature on user interaction duration has focused on dwell time for websites, usually in the context of approximating post-click satisfaction either in search results, or display ads. In this work we present the first analysis of session length in a mobile-focused online service, using a real world data-set from a major music streaming service. We use survival analysis techniques to show that the characteristics of the length distributions can differ significantly between users, and use gradient boosted trees with appropriate objectives to predict the length of a session using only information available at its beginning. Our evaluation on real world data illustrates that our proposed technique outperforms the considered baseline.Comment: 4 pages, 3 figure

A Critical Review of "Automatic Patch Generation Learned from Human-Written Patches": Essay on the Problem Statement and the Evaluation of Automatic Software Repair

At ICSE'2013, there was the first session ever dedicated to automatic program repair. In this session, Kim et al. presented PAR, a novel template-based approach for fixing Java bugs. We strongly disagree with key points of this paper. Our critical review has two goals. First, we aim at explaining why we disagree with Kim and colleagues and why the reasons behind this disagreement are important for research on automatic software repair in general. Second, we aim at contributing to the field with a clarification of the essential ideas behind automatic software repair. In particular we discuss the main evaluation criteria of automatic software repair: understandability, correctness and completeness. We show that depending on how one sets up the repair scenario, the evaluation goals may be contradictory. Eventually, we discuss the nature of fix acceptability and its relation to the notion of software correctness.Comment: ICSE 2014, India (2014