PhoneLab: Cloud-Backed Development Environment for Smartphones

We will develop a Scala-based language and a development environment to simplify the construction of cloud-backed smartphone applications, both by professionals and by end users. We will develop programming assistance tools that use cloud analysis (running on http://ecocloud.ch infrastructure) to suggest code fragments, and enable development and customization of applications both from the desktop and directly from smartphones

HILC: Domain-Independent PbD System Via Computer Vision and Follow-Up Questions

Creating automation scripts for tasks involving Graphical User Interface (GUI) interactions is hard. It is challenging because not all software applications allow access to a program’s internal state, nor do they all have accessibility APIs. Although much of the internal state is exposed to the user through the GUI, it is hard to programmatically operate the GUI’s widgets. To that end, we developed a system prototype that learns by demonstration, called HILC (Help, It Looks Confusing). Users, both programmers and non-programmers, train HILC to synthesize a task script by demonstrating the task. A demonstration produces the needed screenshots and their corresponding mouse-keyboard signals. After the demonstration, the user answers follow-up questions. We propose a user-in-the-loop framework that learns to generate scripts of actions performed on visible elements of graphical applications. Although pure programming by demonstration is still unrealistic due to a computer’s limited understanding of user intentions, we use quantitative and qualitative experiments to show that non-programming users are willing and effective at answering follow-up queries posed by our system, to help with confusing parts of the demonstrations. Our models of events and appearances are surprisingly simple but are combined effectively to cope with varying amounts of supervision. The best available baseline, Sikuli Slides, struggled to assist users in the majority of the tests in our user study experiments. The prototype with our proposed approach successfully helped users accomplish simple linear tasks, complicated tasks (monitoring, looping, and mixed), and tasks that span across multiple applications. Even when both systems could ultimately perform a task, ours was trained and refined by the user in less time

The pedagogy of computer programming using cognitive development through an e-learning object

Motivated by the needs of a pedagogy focusing on minimizing the learning difficulties in program semantics knowledge and logical reasoning, this research project develops a cognitive development-based pedagogy for introductory programming to support students in organizing and constructing knowledge to learn computer programming. A pedagogy is described as a practice and learning theory that defines the teaching and learning. Regarding the practice of this pedagogy, it uses a cognitive learning tool, called e-learning object, to support the scaffolding. With regard to the theory, this pedagogy is developed based on Vygotsky’s Zone of Proximal Development and Piaget’s theory for cognitive development. In particular the scaffolding of this pedagogy includes three major learning processes. The first two learning processes focus on supporting students constructing knowledge on program semantics and conceptually map this knowledge to the coding process. The last learning process extends the learning to self-practice by demanding students to complete a set of exercises independently. All of these learning processes are supported by using the e-learning object, which is the major cognitive learning tool used in this pedagogy to support cognitive development. It is called e-learning object as it is designed by organizing a group of learning objects, in which each of them is to deliver the concepts of a specific unit topic of program control. Together with the course materials, these learning objects are accessed through the college’s ‘Blackboard System’. In addition to the major objective of improving students’ learning performance, this cognitive development-based pedagogy also extends from this objective to find out whether the positive learning outcome connects to cognitive development, and also whether this pedagogy can be embraced by teachers for use in their teaching processes. With these objectives, six research questions are defined in two stages of study. Research questions Q1 and Q2 are used to study students’ learning outcomes in year 1 and 2, and research questions Q3 to Q5 are used to find out whether students’ learning outcomes are connected to cognitive development. Research question Q6 focuses on whether this pedagogy matches teachers’ knowledge of using it, based on their knowledge of applying technology-based pedagogy. The research methodology of this project is the triangulation design where quantitative data are enriched by the collection of qualitative data. This mixture of quantitative and qualitative data collection in different research questions enables this study to interpret the values of this cognitive development-based pedagogy with different views from students and teachers. The research methods mainly include the quasi-experimental method, survey method and the rating scale anchoring method. With these methods, data are collected by using pre-test and post-test papers, questionnaires, and a checklist of rating scale anchoring mental specifications. They are analysed by two-tailed t-test, descriptive method with mean analysis and the one- way repeated measure ANOVA. These research and data analysis methods have been proven effective and used widely, in educational research projects. This research project makes four major contributions: (i) the e-learning object used in this pedagogy can be used to improve students’ learning performance in computer programming; (ii) evidence that a pedagogy focusing on cognitive development can be used to improve students’ learning performance without being limited by programming languages; (iii) development of a cognitive development- based pedagogy for wide use in introductory programming without being limited by teachers’ knowledge and programming languages; and (iv) learning with this cognitive development-based pedagogy builds up students’ problem-solving skills and applies them to different subject areas. With these achieved goals, this project therefore provides a conceptual and operational model for a pedagogical approach to Computer Science teachers design and use in their teaching process