Keyboardless Visual Programming Using Voice, Handwriting, and Gesture

Visual programming languages have facilitated the application development process, improving our ability to express programs, as well as our ability to view, edit and interact with them. Yet even in programming environments, productivity is restricted by the primary input sources: the mouse and the keyboard. As an alternative, we investigate a program development interface which responds to the most natural human communication technologies: voice, handwriting and gesture. Speech- and pen-based systems have yet to find broad acceptance in everyday life because they are insufficiently advantageous to overcome problems with reliability. However, we believe that a visual programming environment with a multimodal user interface properly constrained so as not to exceed the limits of the current technology has the potential to increase programming productivity for not only those people who are manually or visually impaired, but for the general population as well. In this paper we report on such a system

A travel-guidance engine for visually impaired people

Fino a pochi anni fa, usare i trasporti pubblici poteva essere fonte di confusione e richiedere la comprensione del sistema dei trasporti locali. Più tardi, con la diffusione di dispositivi con localizzazione GPS, reti dati cellulare e Google Maps (inizialmente Google Transit), tutto è cambiato, rendendo possibile la pianificazione di un viaggio mentre si è fuori casa. Nonostante Google Maps disponga di indicazioni stradali più o meno in tutto il mondo e mostri molte informazioni, alcune funzionalità, come l’integrazione degli orari in tempo reale, non sono disponibili in tutte le città, ma sono basate su accordi con le agenzie dei trasporti locali. GoGoBus è un’applicazione Android per l’ausilio al trasporto nella città di Bologna. Combinando diversi servizi, GoGoBus si rivolge a svariati tipi di utilizzatori: offre la pianificazione per i meno pratici del sistema e coloro che usano i trasporti pubblici raramente, dispone di orari in tempo reale per chi usa i mezzi frequentemente, e in più traccia la posizione dell’autobus, ha un supporto vocale e un’interfaccia semplice per persone con disabilità. Progettata appositamente per ipovedenti, l’aspetto più innovativo dell’applicazione è il suo supporto durante il percorso sull’autobus, integrato alla pianificazione del tragitto e agli orari aggiornati in tempo reale. Il sistema traccia la posizione dell’autobus attraverso il GPS del dispositivo mobile, la cui posizione è usata sia per riconoscere quando una fermata viene superata, sia per mostrare informazioni utili come la distanza dalla prossima fermata, il numero di fermate e i minuti rimanenti prima di scendere, e soprattutto notificare l’utente quando deve scendere. L’idea dietro GoGoBus è incrementare la fruibilità dei trasporti pubblici per non vedenti, ma anche per persone che li usano di rado, aumentando ampiamente la loro indipendenza, allo stesso tempo migliorando la qualità del servizio per chi usa i mezzi quotidianamente

Extendable framework for graphical block programming systems

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (leaves 60-61).Graphical programming systems have been built to lower the threshold to programming for beginners. However, because these systems were designed to make programming more accessible to novices, they were developed with narrower intentions for their users and applications. For example, in StarLogo TNG, a graphical block programming environment, users may create games and simulations, but they cannot use this same system to create programs that can automate their computer processes, like the text-based scripting system AppleScript. Application developers can create their own programming systems, but doing so can take a significant amount of time to design and implement. This thesis describes an extendable framework called OpenBlocks that enables application developers to build and iterate their own graphical block programming systems by specifying a single XML file. Application developers can focus more on the design of their systems instead of oil the details of implementation. The design and implementation of OpenBlocks are described, along with a user study conducted to test its usability and extendability.by Ricarose Vallarta Roque.M.Eng

Networked programming environment for children

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (leaves 63-64).This thesis introduces NetScratch, a programming environment that enables children to make dynamic digital creations that interact across networks. The work is developed as an extension to Scratch, a multi-media programming environment designed for children to create interactive animations, art, games, and other dynamic creations. Particularly, NetScratch adds websensors, a tool to bring information from the web into a Scratch project, and shareable, a way to share information among projects. These features provide children with the tools to create their own web mashups, experiment with real-time changing data, and connect projects to each other. Using NetScratch, children can create personally meaningful networked projects, while learning important computational and design concepts. And, in designing these creations, children can think about how they want to interact across networks and how their creations affect their friends and communities.by Tamara I. Stern.M.Eng

Characterizing Visual Programming Approaches for End-User Developers: A Systematic Review

Recently many researches have explored the potential of visual programming in robotics, the Internet of Things (IoT), and education. However, there is a lack of studies that analyze the recent evidence-based visual programming approaches that are applied in several domains. This study presents a systematic review to understand, compare, and reflect on recent visual programming approaches using twelve dimensions: visual programming classification, interaction style, target users, domain, platform, empirical evaluation type, test participants\u27 type, number of test participants, test participants\u27 programming skills, evaluation methods, evaluation measures, and accessibility of visual programming tools. The results show that most of the selected articles discussed tools that target IoT and education, while other fields such as data science, robotics are emerging. Further, most tools use abstractions to hide implementation details and use similar interaction styles. The predominant platforms for the tools are web and mobile, while desktop-based tools are on the decline. Only a few tools were evaluated with a formal experiment, whilst the remaining ones were evaluated with evaluation studies or informal feedback. Most tools were evaluated with students with little to no programming skills. There is a lack of emphasis on usability principles in the design stage of the tools. Additionally, only one of the tools was evaluated for expressiveness. Other areas for exploration include supporting end users throughout the life cycle of applications created with the tools, studying the impact of tutorials on improving learnability, and exploring the potential of machine learning to improve debugging solutions developed with visual programming. © 2013 IEEE

Characterizing Visual Programming Approaches for End-User Developers: A Systematic Review

Recently many researches have explored the potential of visual programming in robotics, the Internet of Things (IoT), and education. However, there is a lack of studies that analyze the recent evidence-based visual programming approaches that are applied in several domains. This study presents a systematic review to understand, compare, and reflect on recent visual programming approaches using twelve dimensions: visual programming classification, interaction style, target users, domain, platform, empirical evaluation type, test participants’ type, number of test participants, test participants’ programming skills, evaluation methods, evaluation measures, and accessibility of visual programming tools. The results show that most of the selected articles discussed tools that target IoT and education, while other fields such as data science, robotics are emerging. Further, most tools use abstractions to hide implementation details and use similar interaction styles. The predominant platforms for the tools are web and mobile, while desktop-based tools are on the decline. Only a few tools were evaluated with a formal experiment, whilst the remaining ones were evaluated with evaluation studies or informal feedback. Most tools were evaluated with students with little to no programming skills. There is a lack of emphasis on usability principles in the design stage of the tools. Additionally, only one of the tools was evaluated for expressiveness. Other areas for exploration include supporting end users throughout the life cycle of applications created with the tools, studying the impact of tutorials on improving learnability, and exploring the potential of machine learning to improve debugging solutions developed with visual programming

Cognimates

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2018.Cataloged from PDF version of thesis. "Some pages in the original document contain text that runs off the edge of the page"--Disclaimer Notice page.Includes bibliographical references (pages 199-204).Conversational agents and intelligent toys are present in children's homes. This raises questions as to the impact of Al on their development. In this context, we explore how to educate the children that are growing up with Al and best prepare them for the future. Our prior studies showed that young people consider intelligent agents as friendly and trustworthy, and sometimes even defer to them when making decisions [16, 73]. This thesis explores how children, who are 7 to 14 years old, develop a better understanding of Al concepts and change their perception of smart agents by programming and teaching them with the Cognimates platform we developed. Variations between children of different nationalities and SES backgrounds are discussed together with the influence of their collaboration and communication skills.by Stefania Druga.S.M