Response type selection for chat-like spoken dialog systems based on LSTM and multi-task learning

We propose a method of automatically selecting appropriate responses in conversational spoken dialog systems by explicitly determining the correct response type that is needed first, based on a comparison of the user’s input utterance with many other utterances. Response utterances are then generated based on this response type designation (back channel, changing the topic, expanding the topic, etc.). This allows the generation of more appropriate responses than conventional end-to-end approaches, which only use the user’s input to directly generate response utterances. As a response type selector, we propose an LSTM-based encoder–decoder framework utilizing acoustic and linguistic features extracted from input utterances. In order to extract these features more accurately, we utilize not only input utterances but also response utterances in the training corpus. To do so, multi-task learning using multiple decoders is also investigated. To evaluate our proposed method, we conducted experiments using a corpus of dialogs between elderly people and an interviewer. Our proposed method outperformed conventional methods using either a point-wise classifier based on Support Vector Machines, or a single-task learning LSTM. The best performance was achieved when our two response type selectors (one trained using acoustic features, and the other trained using linguistic features) were combined, and multi-task learning was also performed

Developing FPGA-based Embedded Controllers Using Matlab/Simulink

Field Programmable Gate Arrays (FPGAs) are emerging as suitable platforms for implementing embedded control systems. FPGAs offer advantages such as high performance and concurrent computing which makes them attractive in many embedded applications. As reconfigurable devices, they can be used to build the hardware and software components of an embedded system on a single chip. Traditional FPGA design flows and tools, requiring the use of Hardware Description Languages (HDLs), are in a different domain than standard control system design tools such as MATLAB/Simulink. This paper illustrates development of FPGA-based controllers by utilizing popular tools such as MATLAB/Simulink available for the design and development of control systems. The capability of DSP Builder is extended by developing a custom library of control system building blocks that facilitates rapid development of FPGA-based controllers in the familiar Matlab/Simulink environment. As a case study, this paper presents how the tools can be utilized to develop a FPGA-based controller for a laboratory scale air levitation system

Is question answering fit for the Semantic Web? A survey

With the recent rapid growth of the Semantic Web (SW), the processes of searching and querying content that is both massive in scale and heterogeneous have become increasingly challenging. User-friendly interfaces, which can support end users in querying and exploring this novel and diverse, structured information space, are needed to make the vision of the SW a reality. We present a survey on ontology-based Question Answering (QA), which has emerged in recent years to exploit the opportunities offered by structured semantic information on the Web. First, we provide a comprehensive perspective by analyzing the general background and history of the QA research field, from influential works from the artificial intelligence and database communities developed in the 70s and later decades, through open domain QA stimulated by the QA track in TREC since 1999, to the latest commercial semantic QA solutions, before tacking the current state of the art in open userfriendly interfaces for the SW. Second, we examine the potential of this technology to go beyond the current state of the art to support end-users in reusing and querying the SW content. We conclude our review with an outlook for this novel research area, focusing in particular on the R&D directions that need to be pursued to realize the goal of efficient and competent retrieval and integration of answers from large scale, heterogeneous, and continuously evolving semantic sources

Seahawk: moving beyond HTML in Web-based bioinformatics analysis

<p>Abstract</p> <p>Background</p> <p>Traditional HTML interfaces for input to and output from Bioinformatics analysis on the Web are highly variable in style, content and data formats. Combining multiple analyses can therfore be an onerous task for biologists. Semantic Web Services allow automated discovery of conceptual links between remote data analysis servers. A shared data ontology and service discovery/execution framework is particularly attractive in Bioinformatics, where data and services are often both disparate and distributed. Instead of biologists copying, pasting and reformatting data between various Web sites, Semantic Web Service protocols such as MOBY-S hold out the promise of seamlessly integrating multi-step analysis.</p> <p>Results</p> <p>We have developed a program (Seahawk) that allows biologists to intuitively and seamlessly chain together Web Services using a data-centric, rather than the customary service-centric approach. The approach is illustrated with a ferredoxin mutation analysis. Seahawk concentrates on lowering entry barriers for biologists: no prior knowledge of the data ontology, or relevant services is required. In stark contrast to other MOBY-S clients, in Seahawk users simply load Web pages and text files they already work with. Underlying the familiar Web-browser interaction is an XML data engine based on extensible XSLT style sheets, regular expressions, and XPath statements which import existing user data into the MOBY-S format.</p> <p>Conclusion</p> <p>As an easily accessible applet, Seahawk moves beyond standard Web browser interaction, providing mechanisms for the biologist to concentrate on the analytical task rather than on the technical details of data formats and Web forms. As the MOBY-S protocol nears a 1.0 specification, we expect more biologists to adopt these new semantic-oriented ways of doing Web-based analysis, which empower them to do more complicated, <it>ad hoc </it>analysis workflow creation without the assistance of a programmer.</p

A common ground for virtual humans: using an ontology in a natural language oriented virtual human architecture

When dealing with large, distributed systems that use state-of-the-art components, individual components are usually developed in parallel. As development continues, the decoupling invariably leads to a mismatch between how these components internally represent concepts and how they communicate these representations to other components: representations can get out of synch, contain localized errors, or become manageable only by a small group of experts for each module. In this paper, we describe the use of an ontology as part of a complex distributed virtual human architecture in order to enable better communication between modules while improving the overall flexibility needed to change or extend the system. We focus on the natural language understanding capabilities of this architecture and the relationship between language and concepts within the entire system in general and the ontology in particular. 1

Sikuli: Using GUI screenshots for search and automation

We present Sikuli, a visual approach to search and automation of graphical user interfaces using screenshots. Sikuli allows users to take a screenshot of a GUI element (such as a toolbar button, icon, or dialog box) and query a help system using the screenshot instead of the element's name. Sikuli also provides a visual scripting API for automating GUI interactions, using screenshot patterns to direct mouse and keyboard events. We report a web-based user study showing that searching by screenshot is easy to learn and faster to specify than keywords. We also demonstrate several automation tasks suitable for visual scripting, such as map navigation and bus tracking, and show how visual scripting can improve interactive help systems previously proposed in the literature

KnowText: Auto-generated Knowledge Graphs for custom domain applications

While industrial Knowledge Graphs enable information extraction from massive data volumes creating the backbone of the Semantic Web, the specialised, custom designed knowledge graphs focused on enterprise specific information are an emerging trend. We present “KnowText”, an application that performs automatic generation of custom Knowledge Graphs from unstructured text and enables fast information extraction based on graph visualisation and free text query methods designed for non-specialist users. An OWL ontology automatically extracted from text is linked to the knowledge graph and used as a knowledge base. A basic ontological schema is provided including 16 Classes and Data type Properties. The extracted facts and the OWL ontology can be downloaded and further refined. KnowText is designed for applications in business (CRM, HR, banking). Custom KG can serve for locally managing existing data, often stored as “sensitive” information or proprietary accounts, which are not on open web access. KnowText deploys a custom KG from a collection of text documents and enable fast information extraction based on its graph based visualisation and text based query methods

Sprout: Using a Garden Metaphor to Visualize and Support Customizable and Collaborative Health Tracking

Self-tracking tools have become increasingly popular, especially with the advent of wearable technology and smartphone applications. However, traditional tracking tools often display data in a quantitative format that can be overwhelming and cause users to abandon their tracking efforts. Additionally, these tools typically provide a generic user experience and are designed from a single-user perspective, lacking external support. To overcome these limitations, we develop Sprout, a mobile data-tracking application that offers a more qualitative, customizable, and collaborative experience for health monitoring and management. Sprout uses a garden metaphor to visually represent health information and allows users to tailor their data experience by customizing data capture types and corresponding visual representation for each element. Furthermore, users in Sprout can collaborate to achieve community goals, unlocking new features for their gardens. We conduct a user study with 22 participants to investigate the impact of qualitative data visualization, customizability, and social support on users\u27 activity levels, goal attainment, engagement, and satisfaction with the self-tracking system. Our results suggest that qualitative visualization of data can help some users maintain their motivation to meet health-related goals, but a mix of quantitative and qualitative data is desired by some users. Customizability requires tailored features to help users develop a sense of ownership over time, and social features are a crucial motivator for users to achieve their health goals. However, tracking with strangers instead of friends can hinder user engagement due to the lack of connection

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Multimodal systems have attained increased attention in recent years, which has made possible important improvements in the technologies for recognition, processing, and generation of multimodal information. However, there are still many issues related to multimodality which are not clear, for example, the principles that make it possible to resemble human-human multimodal communication. This chapter focuses on some of the most important challenges that researchers have recently envisioned for future multimodal interfaces. It also describes current efforts to develop intelligent, adaptive, proactive, portable and affective multimodal interfaces