Generation Challenges : Results of the Accuracy Evaluation Shared Task

Publisher PD

UNDERSTANDING STUDENT BEHAVIORS USING IMMEDIATE FEEDBACK FEATURES IN A BLENDED LEARNING ENVIRONMENT

Feedback serves to close the gap between learners’ current understanding and the desired understanding. Informative feedback can keep students from holding onto misconceptions, actively engage learners in knowledge acquisition, and increase confidence and motivation to learn. Yet, in the context of higher education, it is usually not possible for instructors to provide timely feedback to every individual student. This is especially difficult in first-year foundational courses due to the large number of students. Online learning platforms offer a solution by providing students immediate feedback during the course of their interactions with formative assessment tools (e.g., online homework, quizzes, embedded questions in lecture videos). However, how students choose to interact with these features and how these features influence students’ learning experiences have not been well understood. Even less is known about student behaviors with these immediate feedback features in a blended learning class

A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

Acknowledgements: Many thanks to the Mechanical Turk annotators who participated in our experiment, and also to David Reiter, Tim Daniels, Rodrigo de Oliveira, and Andrew Smith for serving as pilot annotators when we were developing the methodology described in this paper. We would also like to thank Moray Greig for being our basketball domain expert during development. We are also grateful for the very helpful comments on this paper from the anonymous reviewers, the Aberdeen CLAN group, David Howcroft, Clement Rebuffel, and Chris van ´ der Lee. We would also like to thank Sam Wiseman, Ratish Puduppully, and Clement Rebuffel for pro- viding the generated texts from their respective systems. The work presented here is partially funded by the Engineering and Physical Sciences Research Council (EPSRC), which funds Craig Thomson under a National Productivity Investment Fund Doctoral Studentship (EP/R512412/1).Peer reviewedPublisher PD

Evaluating factual accuracy in complex data-to-text

We are very grateful for the hard work of the Mechanical Turk annotators who did excellent work and provided helpful feedback. We would like to thank all of the participants in the shared task, the combination of their hard work and diverse approaches has been essential to furthering understanding of the factual accuracy problem in NLG. We would also like to thank Sam Wiseman, Ratish Puduppully, and Clément Rebuffel for providing outputs from their respective systems. The constructive and insightful feedback from the two anonymous reviewers was very helpful and we greatly appreciate their input. We would also like to thank Anya Belz for checking the German translation, as well as Moray Greig, our basketball domain expert. Finally, we would like to thank members of the Aberdeen CLAN group for their advice and feedback. Craig Thomson’s work on this project was supported under an EPSRC NPIF studentship grant (EP/R512412/1).Peer reviewedPostprin

Dynamically typed languages

Dynamically typed languages such as Python and Ruby have experienced a rapid grown in popularity in recent times. However, there is much confusion as to what makes these languages interesting relative to statically typed languages, and little knowledge of their rich history. In this chapter I explore the general topic of dynamically typed languages, how they differ from statically typed languages, their history, and their defining features

Formal Analysis of Network Protocols

Today’s Internet is becoming increasingly complex and fragile. Current performance centric techniques on network analysis and runtime verification have became inadequate in the development of robust networks. To cope with these challenges there is a growing interest in the use of formal analysis techniques to reason about network protocol correctness throughout the network development cycle. This talk surveys recent work on the use of formal analysis techniques to aid in design, implementation, and analysis of network protocols. We first present a general framework that covers a majority of existing formal analysis techniques on both the control and routing planes of networks, and present a classification and taxonomy of techniques according to the proposed framework. Using four representative case studies (Metarouting, rcc, axiomatic formulation, and Alloy based analysis), we discuss various aspects of formal network analysis, including formal specification, formal verification, and system validation. Their strengths and limitations are evaluated and compared in detail

Proceedings of International Workshop "Global Computing: Programming Environments, Languages, Security and Analysis of Systems"

According to the IST/ FET proactive initiative on GLOBAL COMPUTING, the goal is to obtain techniques (models, frameworks, methods, algorithms) for constructing systems that are flexible, dependable, secure, robust and efficient. The dominant concerns are not those of representing and manipulating data efficiently but rather those of handling the co-ordination and interaction, security, reliability, robustness, failure modes, and control of risk of the entities in the system and the overall design, description and performance of the system itself. Completely different paradigms of computer science may have to be developed to tackle these issues effectively. The research should concentrate on systems having the following characteristics: • The systems are composed of autonomous computational entities where activity is not centrally controlled, either because global control is impossible or impractical, or because the entities are created or controlled by different owners. • The computational entities are mobile, due to the movement of the physical platforms or by movement of the entity from one platform to another. • The configuration varies over time. For instance, the system is open to the introduction of new computational entities and likewise their deletion. The behaviour of the entities may vary over time. • The systems operate with incomplete information about the environment. For instance, information becomes rapidly out of date and mobility requires information about the environment to be discovered. The ultimate goal of the research action is to provide a solid scientific foundation for the design of such systems, and to lay the groundwork for achieving effective principles for building and analysing such systems. This workshop covers the aspects related to languages and programming environments as well as analysis of systems and resources involving 9 projects (AGILE , DART, DEGAS , MIKADO, MRG, MYTHS, PEPITO, PROFUNDIS, SECURE) out of the 13 founded under the initiative. After an year from the start of the projects, the goal of the workshop is to fix the state of the art on the topics covered by the two clusters related to programming environments and analysis of systems as well as to devise strategies and new ideas to profitably continue the research effort towards the overall objective of the initiative. We acknowledge the Dipartimento di Informatica and Tlc of the University of Trento, the Comune di Rovereto, the project DEGAS for partially funding the event and the Events and Meetings Office of the University of Trento for the valuable collaboration

The Accuracy Evaluation Shared Task as a Retrospective Reproduction Study

Publisher PD