Playing, Constructionism, and Music in Early-Stage Software Engineering Education

[EN] Understanding that design involves trade-offs, thinking at multiple levels of abstraction, and considering the cohesion and coupling between sub-components of a larger whole is an important part of software (and other) engineering. It can be challenging to convey such abstract design concepts to novice engineers, especially for materials that are themselves abstract (e.g. software). Such challenges are compounded when teaching at the secondary school stage where students have limited experience of large-scale design problems that motivate the need for abstraction at all. In this paper, we describe a method for introducing these concepts to secondary school students using LEGO® and Raspberry Pi computers, asking them to build musical instruments as an entertaining way of motivating engagement with learning about design through play. The method has been successfully piloted in a series of three classroom sessions and key observations and experiences of using the method are presented.This project received no external funding but was funded by the UCL Department of Computer Science Strategic Research FundGold, NE.; Purves, R.; Himonides, E. (2022). Playing, Constructionism, and Music in Early-Stage Software Engineering Education. Multidisciplinary Journal for Education, Social and Technological Sciences. 9(1):14-38. https://doi.org/10.4995/muse.2022.1645314389

Semiotics, Human-Computer Interaction and End-User Development:Special Issue

Semiotics has been an undercurrent in the discussions on HCI and Programming Languages since the early days of Computer Science. This does not come as surprise: reducing computation to algorithmic symbol manipulation starting with Leibniz can be described as the pre-history of computer science (Krämer 1988; Krämer 1993). The Turing Machine, still one of the basic theoretical models for computation, expresses computation as formal symbol manipulation (Turing 1936). Though rather caused by serendipitous collocation, the re-interpretation of Chomsky”s generative grammars, that were meant to describe human language, has been used as a theoretical base for programming languages and compiler construction (Chomsky 1956; Chomsky & Schützenberger 1963; Knuth 1963). Aspects of programming language were early on denoted using semiotic terms: syntax stands for the rules on how to formulate a computer program, semantics refers to studying the computation described by a program. Also the relationship between natural language and programming language has been the subject of reflection for programming language researchers time and again. Heinz Zemanek”s publications on Wittgenstein and programming languages (1966, 1993) can here stand as an example. When it comes to Human Computer Interaction (HCI), the discussion around computing and Semiotics is more diverse. The semiotic and linguistic theories that are brought into play are as heterogeneous as the computer science subdisciplines. An editorial cannot provide a systematic literature review. Nonetheless we would like to provide the interested reader with a number of pointers that could be a starting point for further reading. We apologize to readers who do not find their favorite articles on Semiotics, HCI and End User Development, since our selection is due to limited space. Winograd and Flores” book “Understanding Computers and Cognition” in 1986 triggered a controversial discussion of speech act theory and software design that went far beyond the US community (see e.g. (Ljungberg & Holm 1995)). The discussion on the politics of (speech act) categories (Suchman 1993) sharpened sensitivity to the descriptive and analytical usage of theoretical concepts and their prescriptive usage when informing software design. In Scandinavia, linguists and computer scientists met to discuss ordinary language in relation to computers (Bøgh Andersen and Halskov Madsen 1988; Bøgh Andersen and Bratteteig 1989). These discussions resulted in a number of monographs and edited volumes exploring the application of semiotic and linguistic theories in the context of information systems development (Bøgh Andersen 1990; Bøgh Andersen et al. 1993; Holmqvist et al. 1996). The edited volumes also provided a base to connect researchers from The Netherlands (Stamper 1996) and the US (Klein & Truex 1996) exploring the connection of linguistics and information systems in a similar way. In Germany, Frieder Nake already in the 70 referred to aesthetic theory connecting semiotics and human computer interaction (Nake 1994). In the article “Human–computer interaction viewed as pseudo-communication” Nake and Grabowski (2001) proposed understanding informatics as technical semiotics and suggested a conceptualization of Human Computer Interaction as an interface between technical symbol manipulation and human sense making. Dittrich (1997; 1998) elaborated the pragmatic dimension of the interaction between ordinary and formal language, when developing and using software referring to Wittgenstein”s Philosophical Investigations and Humboldt”s language philosophy. Christiane Floyd, who, since the 80, had researched participatory and evolutionary software engineering, conceptualized software engineering as being concerned with “Developing and Embedding Autooperational Forms” (1997; 2002), where the concept of autooperational form describes the formal symbol manipulation defined by the program. Besides Peter Bøgh Andersen, Mihai Nadin (Nadin 2011) and Clarisse S De Souza (DeSouza 2005) contributed continuously to the discussion on Semiotics and Computers. De Souza started to develop her semiotic engineering approach in the 90. Semiotic engineering views the whole software program as a message sent from the developer to the user of a software system on how to relate to what the software is about and how to interact through the software system (DeSouza 2005). De Souza emphasises the openness of the interpretation that the software system as a semiotic artefact triggers in ongoing cultural discourse (De Souza 2013). Though the idea of EUD can be traced back to the early days of computer science (Mehandjiev and Bottaci 1995), the systematic research of EUD only dates back to two parallel projects, one is the European Network of Excellence resulting in (Lieberman et al. 2016) and the other is a US NSF project on End User Software Engineering, whose members joined forces to author (Ko 2011). The research on End User Development (EUD) (Lieberman et al. 2016) and End User Software Engineering (EUSE) (Ko 2011) refers both to programming language techniques and principles and to Human Computer Interaction: users are provided with interfaces that allow them to change the software they are using. These interfaces can be described as (domain specific) programming languages. EUSE research encompasses activities that lead to the creation, modification or extension of software artifacts for personal use, only by primarily addressing quality aspects that are “individual EUD activities” (Cabitza et al., 2014). EUD covers methods, situations and socio-technical environments which enable conditions that put owners of problems in charge, by defining the technical and social conditions for broad participation in design activities [Fischer, 2013]. In a wider sense EUD does not focus only on software (as do EUP and EUSE), but it “encompasses methods, techniques, methodologies, situations, and socio-technical environments that allow end users to act as professionals in those domains in which they are not professionals” (Fischer et al., in print). A more recent and updated view on EUD is reported in the new book on End-User Development (Paternò and Wulf, in print). Piero Mussio and his group in Brescia and Milano were to our knowledge the first to explicitly relate End User Development and a semiotic take on HCI (Marcante & Mussio 2016, Valtolina et al. 2012). In particular, they presented a computer semiotic approach to describe the digital communication process and the new characteristics that have evolved from the creation and the maintenance of e-documents and the interaction with them. In this view the HCI process is seen as exchanges of messages – physical representations of e-documents - and contrasts e-document features to those of oral and written ones. Clearly EUD takes advantage of the semiotic character of software: the functionality and appearance of a program is defined by a textual representation, which in turn can be offered, in an adequate form, to the user for manipulation. Referring to De Souza”s (2005) semiotic engineering, EUD allows the user to comment the message of the software developers not only in the same medium, by changing the text, but also by commenting on the message in a different medium, e.g. in oral feedback or by using the system in an innovative manner. In other words, taking the step to become an End User Developer, the user becomes an author, who not only communicates through the system, but changes the program or medium and enters the design discourse on an equal footing. Here EUD provides a specific challenge for the interface design: not only does the interaction with the system to achieve a goal need to be designed, but also the formulation of a program needs to be supported. Can the semiotic take on design of a computer system inform also this design? By editing a special issue connecting semiotics and End User Development, we have invited authors to explore this area and further develop both the understanding of software systems as semiotic artefacts and the design of EUD systems. As the description of the accepted papers below indicates, the contributions provide not only a documentation of existing research, but also provide important innovative stepping stones. Initially we received 20 submissions from authors from 11 different countries all over the world (Italy, Bangladesh, The Netherlands, Brazil, the Russian Federation, Belgium, Australia, Denmark, the US, the UK and Finland). All the papers underwent a rigid review process. After two rounds of reviews, seven papers were accepted for publication in this special issue. Accepted papers come from 3 different countries (Italy 3, Brazil 3 and Belgium 1). The special issue consists of seven excellent papers. Rita Barricelli and Stefano Valtolina develop, implement and evaluate “A Visual Language and Interactive System for End-User Development of Internet of Things Ecosystems” that supports trainers and athletes to make use of wearable interactive devices that allow the formulation of complex temporal rules to support lifestyle change. The language is designed both to relate to the domain of training and to steer the underpinning network of interacting sensors and activators. The evaluation shows that semiotic design leads to understandable and useful interfaces for both End User Developers and users. The paper “Self-expression and discourse continuity in a multilevel EUD environment: The case of Moodle” by Luciana da Silveira Espindola and Milene Selbach Silveira deals with Virtual Learning Environments as a platform, allowing multiple customizations through multiple layers of (distinct) users. In particular, they focus on verifying the discourse continuity after multiple customizations by many hands, looking at how it impacts on the teacher”s self-expression and how this expression is perceived by the students, while interacting with the product of their teacher”s effort. The paper “An analysis of deictic signs in computer interfaces: contributions to the Semiotic Inspection Method” by Aron Daniel Lopes, Vinicius Carvalho Pereira and Cristiano Maciel analyses the concept of deictic signs in digital games. Deictic signs in this case are here signs that establish an indexical relation with the objects they refer to, placing them in terms of space, time and person with a reference to the communication through the interface. They are considered as a component of the conceptual framework of Linguistics and Semiotics that can be added to the Semiotic Inspection Method (SIM) when the method is adapted to approach digital games better. The paper “The semiotics of configurations for the immanent design of interactive computational systems”, by Federico Cabitza and Alvise Mattozzi, proposes a novel semiotic approach to the design of interactive systems and computational systems, called Semiotics of Configurations (SoC). The approach has inspired a platform for user-driven development and the use of electronic documents and forms in cooperative and organizational domains, such as hospital work. The article “Signifying Software Engineering to Computational Thinking Learners with AgentSheets and PoliFacets” by Ingrid Monteiro, Luciana Salgado, Marcelle Mota, Andreia Liborio Sampaio and Clarisse de Souza evaluates an approach to computational thinking acquisition (CTA) that is based on an understanding of programming as communication when it supports the core notions of (End User) Software Engineering: requirements, design, reuse testing and debugging. The result shows that a semiotic approach to End User Programming is compatible with the teaching and implementation of software engineering concepts, though it requires a knowledgeable teacher who can draw the connection between the features of the End User Programming environment and the software engineering concepts. Daniela Fogli in her article “Weaving Semiotic Engineering in Meta-Design: A Case Study Analysis” takes the application of a semantic understanding of programs as communication between software engineers and users a step further. She uses it to explore the relationship between software engineers responsible for the design of an EUD environment (meta design) and the domain expert using that environment to implement concrete applications for other users. The message in meta-design seen as communication is therefore a message about what kind of messages can be designed to be communicated to the (other) users by the end user developer, and how the dialogue between users and end user developers is meant to evolve