Interactive Machine Learning for User-Innovation Toolkits – An Action Design Research approach

Machine learning offers great potential to developers and end users in the creative industries. However, to better support creative software developers' needs and empower them as machine learning users and innovators, the usability of and developer experience with machine learning tools must be considered and better understood. This thesis asks the following research questions: How can we apply a user-centred approach to the design of developer tools for rapid prototyping with Interactive Machine Learning? In what ways can we design better developer tools to accelerate and broaden innovation with machine learning? This thesis presents a three-year longitudinal action research study that I undertook within a multi-institutional consortium leading the EU H2020 -funded Innovation Action RAPID-MIX. The scope of the research presented here was the application of a user-centred approach to the design and evaluation of developer tools for rapid prototyping and product development with machine learning. This thesis presents my work in collaboration with other members of RAPID-MIX, including design and deployment of a user-centred methodology for the project, interventions for gathering requirements with RAPID-MIX consortium stakeholders and end users, and prototyping, development and evaluation of a software development toolkit for interactive machine learning. This thesis contributes with new understanding about the consequences and implications of a user-centred approach to the design and evaluation of developer tools for rapid prototyping of interactive machine learning systems. This includes 1) new understanding about the goals, needs, expectations, and challenges facing creative machine-learning non-expert developers and 2) an evaluation of the usability and design trade-offs of a toolkit for rapid prototyping with interactive machine learning. This thesis also contributes with 3) a methods framework of User-Centred Design Actions for harmonising User-Centred Design with Action Research and supporting the collaboration between action researchers and practitioners working in rapid innovation actions, and 4) recommendations for applying Action Research and User-Centred Design in similar contexts and scale

Protest Images, Collective Portraits

A aparência é uma instância política, uma vez que a representação coletiva é uma possibilidade de emancipação da comunidade, dos “povos” (Didi-Huberman, 2012) e um mecanismo de constituição de memória cultural. A possibilidade de aparência, que também participa da história coletiva é uma relação de poder em si mesma. Na cultura digital, a aparência global banalizou-se, mas também foi dissolvida sob um regime tecnológico digital de produção visual, de “imagem de massa” (Cubitt, 2016). A imaterialidade, a obsolescência programada, as versões de software disponibilizado e a dependência para produzir e criar visualidades sob a influência de regimes corporativos globais, que organizam sistemas e tecnologias, tornam esta afirmação paradoxal e problemática para uma constituição coletiva dessas representações, correndo o risco de mercantilizar a memória. Apesar disso, tem havido uma ligação entre técnicas culturais e memória, onde a “ideologia californiana” (Barbrook, 1999) organiza a cultura digital, num equilíbrio entre uma “cultura participativa” e um “empreendedorismo neoliberal”. No entanto, nunca uma quantidade tão grande de imagens foi produzida e partilhada. Hito Steyerl define estas imagens digitais como “imagens pobres” ou, como defendido neste trabalho, imagens precárias, ao criar uma correspondência entre o atual regime político de trabalho com uma produção visual digital, numa perspectiva não profissional. Mas em momentos de protesto, estas “imagens pobres”, reconhecidas como desvalorizadas, comunicam e criar memória e história, emancipando a noção de vídeo “vernacular” como parte da acção política (Snowdon, 2014). Foram momentos como os movimentos sociais de 2011, como as “Arab Springs”, “Movimento 12M”,“Madrid 15M” e “Occupy Wall Street”, em que as “redes de indignação e esperança”(Castells, 2012) se formaram e surgiram globalmente, num espaço público híbrido de contestação, que estas imagens podem ser novamente convocadas. Conforme analisado, na paisagem visual do Youtube.com, onde “a cultura participativa continua a ser o seu principal negócio” (Burgess e Green, 2009), estas imagens fazem parte do fluxo dos protestos e a sua recuperação constitui um ato de “reaparecimento”. Este reaparecimento é pensado aqui em diferentes gestos, respectivamente, como constituição de um corpus de vídeos de protesto, alinhados pela metáfora das imagens-pirilampos (Didi-Huber man, 2009), em tempos digitais, transportadas para o continente visual do Youtube.com. Também como parte de uma investigação prática, foi desenvolvido um protótipo de um documentário interativo, como um filme plataforma, organizado em torno da representação de uma “articulação do protesto” (Steyerl, 2002) onde as contribuições se organizam, entre linhas temporais individuais e coletivas. Como num editor de vídeo, é proposta uma mesa de montagem como experiência participativa, utilizando os materiais dos protestos. Este estudo propõe experimentar práticas artísticas como num “laboratório crítico” (Hirschhorn, 2013) com um efeito de “coletivos encontrados” presente no arquivo (Baron, 2013) que se constituem neste artefacto multimédia, interativo enquanto hipótese de persistência na memória coletiva, ou retrato coletivo de aparência política, a partir de momentos de protesto de movimentos sociais como os de 2011. Produzir uma intervenção activista e estética, uma intervenção artivista, como “forma de arte” política do século XXI (Weibel, 2014).Appearance is a political instance, as collective representation is a possibility to an emancipation of communities, of the “peuples” (Didi-Huberman, 2012), and a mechanism of constitution of cultural memory. The possibility of appearance that also participates in the history of collectives is a power relation in itself. In digital culture, global appearance seems to have exploded, but it has also been dissolved under a digital technological regime of “mass image”(Cubitt, 2016) visual production. Immateriality, programmed obsolescence, pervasive software and a dependence to produce and visualize under global corporations regimes, that organize systems and technologies, make this affirmation paradoxical and problematic to a collective constitution of these representations with a risk of commodifying memory. Despite this, there has been a connection between cultural techniques and memory, that under the “Californian Ideology” (Barbrook, 1999) digital culture is organized as a balance between “participatory culture” and “neoliberal entrepreneurship”. However, never such a quantity of images have been produced and shared. Hito Steyerl defines these as “poor images” or as stated here, defined as precarious images, making a correspondence between present political regimes of labour with visual and cultural production digitally produced, as non professional video. But in moments of uprisings, these poor images, commonly recognized as devalued, served to communicate and create memory and History, emancipating “vernacular” videos as part of the political actors (Snowdon, 2014) Such were moments as 2011 social movements, like “Arab Springs”, “Moviment12M”, “Madrid15M” and “Occupy Wall Street”, when “networks of outrage and hope” (Manuel Castells, 2012) stepped out globally, in an hybrid public space of insurrection. As analyzed in visual landscapes of Youtube.com where “participatory culture is its core business” (Burgess and Green, 2009), these images were part of the uprisings flow, and their retrieval constitutes an act of “reappearance”. This reappearance is oriented here in different gestures, respectively as a constitution of a corpus of protest videos, aligned through the metaphor of fireflies-images (Didi-Huberman, 2009) in digital times, transported to Youtube.com visual continents. Also, as part of a practice based research, a prototype of an interactive documentary, as a platform film, has been developed, aligned with a representation of an “articulation of protest”(Hito Steyerl, 2002) where, as an editing table, between individual and collective timelines, a participatory interactive experience is proposed, using remnant materials of protests. This study essays how to relate artistic practices of “Critical Laboratory” (Hirschhorn, 2013) with “found collective” effect of archive documentary (Jaimie Baron, 2013) related in this digital, online, multimedia, interactive, audiovisual artifact, produced by individuals with digital images, in a way to persist in collective memory and become a collective portrait of political appearance from historical moments of social movements uprisings as those of 2011. At the same time, an activist and aesthetic intervention, an artivist intervention takes place, as a political “XXIst Century art form” (Weibel, 2014)

09051 Abstracts Collection -- Knowledge representation for intelligent music processing

From the twenty-fifth to the thirtieth of January, 2009, the Dagstuhl Seminar 09051 on ``Knowledge representation for intelligent music processing\u27\u27 was held in Schloss Dagstuhl~--~Leibniz Centre for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations and demos given during the seminar as well as plenary presentations, reports of workshop discussions, results and ideas are put together in this paper. The first section describes the seminar topics and goals in general, followed by plenary `stimulus\u27 papers, followed by reports and abstracts arranged by workshop followed finally by some concluding materials providing views of both the seminar itself and also forward to the longer-term goals of the discipline. Links to extended abstracts, full papers and supporting materials are provided, if available. The organisers thank David Lewis for editing these proceedings

The Effectiveness of Online Simulation with GDL and PBL toward Students’ Digital Literacy Skill

This study aimed to investigate the impact of online simulation in physics learning toward students’ digital literacy skills. There are 106 randomly students Grade XII Science from West Borneo and Riau Province, Indonesia as participants in this study. The level of digital literacy skill of all participants in the classes before and after the learning process was determined to assess the effectiveness of two models with online simulation learning.  We used comparative –descriptive, pre-test and post-test experimental design, and Anova Mixed Design to analysis data. The result of this study was: (1) Students’ digital literacy skill of students in two groups initially are at very low level and increased after learning; (2) there was a significant difference in the level of digital literacy skill of the students online simulation with GDL class; (3) in this study, the online simulation with GDL model is more effective than online simulation with PBL model in enhancing students’ digital literacy skill in senior high school. For the conclusion, in this study students’ digital literacy skill in Indonesia is rather below. Model PBL with online simulation is one of the teaching models that can be used in enhancing students’ digital literacy skills

Worldwide Infrastructure for Neuroevolution: A Modular Library to Turn Any Evolutionary Domain into an Online Interactive Platform

Across many scientific disciplines, there has emerged an open opportunity to utilize the scale and reach of the Internet to collect scientific contributions from scientists and non-scientists alike. This process, called citizen science, has already shown great promise in the fields of biology and astronomy. Within the fields of artificial life (ALife) and evolutionary computation (EC) experiments in collaborative interactive evolution (CIE) have demonstrated the ability to collect thousands of experimental contributions from hundreds of users across the glob. However, such collaborative evolutionary systems can take nearly a year to build with a small team of researchers. This dissertation introduces a new developer framework enabling researchers to easily build fully persistent online collaborative experiments around almost any evolutionary domain, thereby reducing the time to create such systems to weeks for a single researcher. To add collaborative functionality to any potential domain, this framework, called Worldwide Infrastructure for Neuroevolution (WIN), exploits an important unifying principle among all evolutionary algorithms: regardless of the overall methods and parameters of the evolutionary experiment, every individual created has an explicit parent-child relationship, wherein one individual is considered the direct descendant of another. This principle alone is enough to capture and preserve the relationships and results for a wide variety of evolutionary experiments, while allowing multiple human users to meaningfully contribute. The WIN framework is first validated through two experimental domains, image evolution and a new two-dimensional virtual creature domain, Indirectly Encoded SodaRace (IESoR), that is shown to produce a visually diverse variety of ambulatory creatures. Finally, an Android application built with WIN, filters, allows users to interactively evolve custom image effects to apply to personalized photographs, thereby introducing the first CIE application available for any mobile device. Together, these collaborative experiments and new mobile application establish a comprehensive new platform for evolutionary computation that can change how researchers design and conduct citizen science online

Data Musicalization

Data musicalization is the process of automatically composing music based on given data, as an approach to perceptualizing information artistically. The aim of data musicalization is to evoke subjective experiences in relation to the information, rather than merely to convey unemotional information objectively. This paper is written as a tutorial for readers interested in data musicalization. We start by providing a systematic characterization of musicalization approaches, based on their inputs, methods and outputs. We then illustrate data musicalization techniques with examples from several applications: one that perceptualizes physical sleep data as music, several that artistically compose music inspired by the sleep data, one that musicalizes on-line chat conversations to provide a perceptualization of liveliness of a discussion, and one that uses musicalization in a game-like mobile application that allows its users to produce music. We additionally provide a number of electronic samples of music produced by the different musicalization applications.Peer reviewe

Extempore: The design, implementation and application of a cyber-physical programming language

There is a long history of experimental and exploratory programming supported by systems that expose interaction through a programming language interface. These live programming systems enable software developers to create, extend, and modify the behaviour of executing software by changing source code without perceptual breaks for recompilation. These live programming systems have taken many forms, but have generally been limited in their ability to express low-level programming concepts and the generation of efficient native machine code. These shortcomings have limited the effectiveness of live programming in domains that require highly efficient numerical processing and explicit memory management. The most general questions addressed by this thesis are what a systems language designed for live programming might look like and how such a language might influence the development of live programming in performance sensitive domains requiring real-time support, direct hardware control, or high performance computing. This thesis answers these questions by exploring the design, implementation and application of Extempore, a new systems programming language, designed specifically for live interactive programming

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds