Crowdsourcing step-by-step information extraction to enhance existing how-to videos

Millions of learners today use how-to videos to master new skills in a variety of domains. But browsing such videos is often tedious and inefficient because video player interfaces are not optimized for the unique step-by-step structure of such videos. This research aims to improve the learning experience of existing how-to videos with step-by-step annotations. We first performed a formative study to verify that annotations are actually useful to learners. We created ToolScape, an interactive video player that displays step descriptions and intermediate result thumbnails in the video timeline. Learners in our study performed better and gained more self-efficacy using ToolScape versus a traditional video player. To add the needed step annotations to existing how-to videos at scale, we introduce a novel crowdsourcing workflow. It extracts step-by-step structure from an existing video, including step times, descriptions, and before and after images. We introduce the Find-Verify-Expand design pattern for temporal and visual annotation, which applies clustering, text processing, and visual analysis algorithms to merge crowd output. The workflow does not rely on domain-specific customization, works on top of existing videos, and recruits untrained crowd workers. We evaluated the workflow with Mechanical Turk, using 75 cooking, makeup, and Photoshop videos on YouTube. Results show that our workflow can extract steps with a quality comparable to that of trained annotators across all three domains with 77% precision and 81% recall

VLSI design of configurable low-power coarse-grained array architecture

Biomedical signal acquisition from in- or on-body sensors often requires local (on-node) low-level pre-processing before the data are sent to a remote node for aggregation and further processing. Local processing is required for many different operations, which include signal cleanup (noise removal), sensor calibration, event detection and data compression. In this environment, processing is subject to aggressive energy consumption restrictions, while often operating under real-time requirements. These conflicting requirements impose the use of dedicated circuits addressing a very specific task or the use of domain-specific customization to obtain significant gains in power efficiency. However, economic and time-to-market constraints often make the development or use of application-specific platforms very risky.One way to address these challenges is to develop a sensor node with a general-purpose architecture combining a low-power, low-performance general microprocessor or micro-controller with a coarse-grained reconfigurable array (CGRA) acting as an accelerator. A CGRA consists of a fixed number of processing units (e.g., ALUs) whose function and interconnections are determined by some configuration data.The objective of this work is to create an RTL-level description of a low-power CGRA of ALUs and produce a low-power VLSI (standard cell) implementation, that supports power-saving features.The CGRA implementation should use as few resources as possible and fully exploit the intended operation environment. The design will be evaluated with a set of simple signal processing task

Development and Validation of a Rule-based Time Series Complexity Scoring Technique to Support Design of Adaptive Forecasting DSS

Evidence from forecasting research gives reason to believe that understanding time series complexity can enable design of adaptive forecasting decision support systems (FDSSs) to positively support forecasting behaviors and accuracy of outcomes. Yet, such FDSS design capabilities have not been formally explored because there exists no systematic approach to identifying series complexity. This study describes the development and validation of a rule-based complexity scoring technique (CST) that generates a complexity score for time series using 12 rules that rely on 14 features of series. The rule-based schema was developed on 74 series and validated on 52 holdback series using well-accepted forecasting methods as benchmarks. A supporting experimental validation was conducted with 14 participants who generated 336 structured judgmental forecasts for sets of series classified as simple or complex by the CST. Benchmark comparisons validated the CST by confirming, as hypothesized, that forecasting accuracy was lower for series scored by the technique as complex when compared to the accuracy of those scored as simple. The study concludes with a comprehensive framework for design of FDSS that can integrate the CST to adaptively support forecasters under varied conditions of series complexity. The framework is founded on the concepts of restrictiveness and guidance and offers specific recommendations on how these elements can be built in FDSS to support complexity

"Customization is Key": Four Characteristics of Textual Affordances for Accessible Data Visualization

Current best practices recommend using textual descriptions to make data visualizations accessible to blind and low vision (BLV) screen reader users. While recent research has explored laying such descriptions out hierarchically to enable reading varying levels of detail, the textual descriptions remain fixed: their syntax and semantics are set by the visualization author or tool, and cannot be changed by a BLV user based on their preferences or task-specific needs. In this paper, we explore four characteristics of customizations for hierarchical textual descriptions of visualizations: presence, or what content is present in the description; verbosity, or the length and conciseness of the content; ordering, or the sequencing of content; and, duration, or how long a particular customization lasts. We instantiate these methods as extensions to Olli, an open source library that converts web-based visualizations into hierarchical textual structures, and evaluate our work through a mixed-methods study with 13 BLV participants. Users reported that customization is crucial to their agency and that being able to change the four characteristics helps them efficiently carry out their desired tasks on the data. However, differences in preferred defaults, prior experiences, and enthusiasm for customization indicate that there is no one-size-fits-all system even for customization itself: both accessible data visualizations and user interfaces for customizing them must be flexible enough to meet a variety of needs.Comment: 24 pages. 6 figures. 2 table

An Improved Approach of Intention Discovery with Machine Learning for POMDP-based Dialogue Management

An Embodied Conversational Agent (ECA) is an intelligent agent that works as the front end of software applications to interact with users through verbal/nonverbal expressions and to provide online assistance without the limits of time, location, and language. To help to improve the experience of human-computer interaction, there is an increasing need to empower ECA with not only the realistic look of its human counterparts but also a higher level of intelligence. This thesis first highlights the main topics related to the construction of ECA, including different approaches of dialogue management, and then discusses existing techniques of trend analysis for its application in user classification. As a further refinement and enhancement to our prior work on ECA, this thesis research proposes a cohesive framework to integrate emotion-based facial animation with improved intention discovery. In addition, a machine learning technique modelled from Q-learning (Quality-Learning) technique is introduced to support sentiment analysis for the adjustment of policy design in POMDP-based dialogue management. It is anticipated that the proposed research work is going to improve the accuracy of intention discovery while reducing the length of dialogues. Un agent de conversation incorporé (ECA) est un agent intelligent fonctionnant en amont des applications logicielles pour interagir avec les utilisateurs par le biais d\u27expressions verbales / non verbales et pour fournir une assistance en ligne sans limite de temps, de lieu et de langage. Pour aider à améliorer l\u27expérience de l\u27interaction homme-machine, il est de plus en plus nécessaire de doter la CEA de droits non seulement vis-à-vis de ses homologues humains, mais également d\u27un niveau de renseignement supérieur. Cette thèse aborde d’abord les principaux sujets liés à la construction de la CEA, y compris différentes approches de la gestion du dialogue, puis aborde les techniques existantes d’analyse des tendances pour son application à la classification des utilisateurs. Pour affiner et améliorer nos travaux antérieurs sur ECA, cette thèse de recherche propose un cadre cohérent pour intégrer une animation faciale basée sur les émotions avec une découverte de l’intention améliorée. En outre, une technique d\u27apprentissage automatique modélisée à partir de la technique Q-learning (Quality-Learning) est introduite pour prendre en charge l\u27analyse des sentiments afin d\u27ajuster la conception des stratégies dans la gestion du dialogue basée sur POMDP. On s’attend à ce que les travaux de recherche proposés améliorent la précision de la découverte de l’intention tout en réduisant la durée des dialogues

Naturalness vs. Predictability: A Key Debate in Controlled Languages

Abstract. In this paper we describe two quite different philosophies used in developing controlled languages (CLs): A "naturalist " approach, in which CL interpretation is treated as a simpler form of full natural language processing; and a "formalist " approach, in which the CL interpretation is “deterministic” (context insensitive) and the CL is viewed more as an English-like formal specification language. Despite the philosophical and practical differences, we suggest that a synthesis can be made in which a deterministic core is embedded in a naturalist CL, and illustrate this with our own controlled language CPL. In the second part of this paper we present a fictitious debate between an ardent “naturalist ” and an ardent “formalist”, each arguing their respective positions, to illustrate the benefits and tradeoffs of these different philosophies in an accessible way. Part I: The Naturalist vs. Formalist Debate