Correction of Errors in Preference Ratings from Automated Metrics for Text Generation

A major challenge in the field of Text Generation is evaluation: Human evaluations are cost-intensive, and automated metrics often display considerable disagreement with human judgments. In this paper, we propose a statistical model of Text Generation evaluation that accounts for the error-proneness of automated metrics when used to generate preference rankings between system outputs. We show that existing automated metrics are generally over-confident in assigning significant differences between systems in this setting. However, our model enables an efficient combination of human and automated ratings to remedy the error-proneness of the automated metrics. We show that using this combination, we only require about 50% of the human annotations typically used in evaluations to arrive at robust and statistically significant results while yielding the same evaluation outcome as the pure human evaluation in 95% of cases. We showcase the benefits of approach for three text generation tasks: dialogue systems, machine translation, and text summarization

Action Selection for Interaction Management: Opportunities and Lessons for Automated Planning

The central problem in automated planning---action selection---is also a primary topic in the dialogue systems research community, however, the nature of research in that community is significantly different from that of planning, with a focus on end-to-end systems and user evaluations. In particular, numerous toolkits are available for developing speech-based dialogue systems that include not only a method for representing states and actions, but also a mechanism for reasoning and selecting the actions, often combined with a technical framework designed to simplify the task of creating end-to-end systems. We contrast this situation with that of automated planning, and argue that the dialogue systems community could benefit from some of the directions adopted by the planning community, and that there also exist opportunities and lessons for automated planning

Impact of information technology in trade facilitation on small and medium-sized enterprises in Bangladesh

This paper focuses specifically on one particular aspect of trade facilitation in the context of Bangladesh, i.e., impact on SMEs of IT in trade facilitation. It is hoped that the policy recommendations offered in this paper will be useful in furthering the cause of SME internationalization in Bangladesh.Trade Facilitation, Bangladesh, SMEs, Information Technology

Studies in Trade and Investment: The Development Impact of Information Technology in Trade Facilitation

In Bangladesh, SMEs are very important players in the economy. About 90 per cent of all industrial units in Bangladesh are SMEs, which generate some 25 per cent of the gross domestic product (GDP), employ about 31 million people and provide 75 per cent of household income. There is no denying that SMEs act as the driver of the economy and are very important for national economic and social development. They serve as employers creating new jobs and providers of products for daily needs. They also act as stewards over employees and the community. However, SMEs in developing and least developed countries face considerable barriers in running their businesses and are often constrained financially and technologically. This includes inadequate and/or complex sets of policies by the respective governments. Such impediments largely contribute to the under-involvement of SMEs in international markets. This is where the nature and extent of SME participation in the global trading system needs to be highlighted. Trade facilitation is thus a crucial factor in providing SMEs with access to global markets.Trade facilitation, ICT, IT, SMEs, international trade, Bangladesh

The use of artificial intelligence techniques to improve the multiple payload integration process

A maximum return of science and products with a minimum expenditure of time and resources is a major goal of mission payload integration. A critical component then, in successful mission payload integration is the acquisition and analysis of experiment requirements from the principal investigator and payload element developer teams. One effort to use artificial intelligence techniques to improve the acquisition and analysis of experiment requirements within the payload integration process is described

Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time

Crowd-powered conversational assistants have been shown to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. A promising direction is to combine the two approaches for high quality, low latency, and low cost solutions. In this paper, we introduce Evorus, a crowd-powered conversational assistant built to automate itself over time by (i) allowing new chatbots to be easily integrated to automate more scenarios, (ii) reusing prior crowd answers, and (iii) learning to automatically approve response candidates. Our 5-month-long deployment with 80 participants and 281 conversations shows that Evorus can automate itself without compromising conversation quality. Crowd-AI architectures have long been proposed as a way to reduce cost and latency for crowd-powered systems; Evorus demonstrates how automation can be introduced successfully in a deployed system. Its architecture allows future researchers to make further innovation on the underlying automated components in the context of a deployed open domain dialog system.Comment: 10 pages. To appear in the Proceedings of the Conference on Human Factors in Computing Systems 2018 (CHI'18

Design and implementation of a user-oriented speech recognition interface: the synergy of technology and human factors

The design and implementation of a user-oriented speech recognition interface are described. The interface enables the use of speech recognition in so-called interactive voice response systems which can be accessed via a telephone connection. In the design of the interface a synergy of technology and human factors is achieved. This synergy is very important for making speech interfaces a natural and acceptable form of human-machine interaction. Important concepts such as interfaces, human factors and speech recognition are discussed. Additionally, an indication is given as to how the synergy of human factors and technology can be realised by a sketch of the interface's implementation. An explanation is also provided of how the interface might be integrated in different applications fruitfully