Identifying Engagement in Children's Interaction whilst Composing Digital Music at Home

Identifying points of engagement from a person’s interaction with computers could be used to assess their experience and to adapt user interfaces in real-time. However, it is difficult to identify points of engagement unobtrusively; HCI studies typically use retrospective protocols or rely on cumbersome sensors for real-time analysis. We present a case study on how children compose digital music at home in which we remotely identify points of engagement from patterns of interaction with a musical interface. A mixed-methods approach is contributed in which video recordings of children’s interactions whilst composing are labelled for engagement and linked to i) interaction logs from the interface to identify indicators of engagement in interaction, and ii) interview data gathered using a remote video-cued recall technique to understand the experiential qualities of engaging interactions directly from users. We conclude by speculating on how the suggested indicators of engagement inform the design of adaptive music systems

Evaluating Animation Parameters for Morphing Edge Drawings

Partial edge drawings (PED) of graphs avoid edge crossings by subdividing each edge into three parts and representing only its stubs, i.e., the parts incident to the end-nodes. The morphing edge drawing model (MED) extends the PED drawing style by animations that smoothly morph each edge between its representation as stubs and the one as a fully drawn segment while avoiding new crossings. Participants of a previous study on MED (Misue and Akasaka, GD19) reported eye straining caused by the animation. We conducted a user study to evaluate how this effect is influenced by varying animation speed and animation dynamic by considering an easing technique that is commonly used in web design. Our results provide indications that the easing technique may help users in executing topology-based tasks accurately. The participants also expressed appreciation for the easing and a preference for a slow animation speed.Comment: Appears in the Proceedings of the 31st International Symposium on Graph Drawing and Network Visualization (GD 2023

One Deep Music Representation to Rule Them All? : A comparative analysis of different representation learning strategies

Inspired by the success of deploying deep learning in the fields of Computer Vision and Natural Language Processing, this learning paradigm has also found its way into the field of Music Information Retrieval. In order to benefit from deep learning in an effective, but also efficient manner, deep transfer learning has become a common approach. In this approach, it is possible to reuse the output of a pre-trained neural network as the basis for a new learning task. The underlying hypothesis is that if the initial and new learning tasks show commonalities and are applied to the same type of input data (e.g. music audio), the generated deep representation of the data is also informative for the new task. Since, however, most of the networks used to generate deep representations are trained using a single initial learning source, their representation is unlikely to be informative for all possible future tasks. In this paper, we present the results of our investigation of what are the most important factors to generate deep representations for the data and learning tasks in the music domain. We conducted this investigation via an extensive empirical study that involves multiple learning sources, as well as multiple deep learning architectures with varying levels of information sharing between sources, in order to learn music representations. We then validate these representations considering multiple target datasets for evaluation. The results of our experiments yield several insights on how to approach the design of methods for learning widely deployable deep data representations in the music domain.Comment: This work has been accepted to "Neural Computing and Applications: Special Issue on Deep Learning for Music and Audio

A Simple Deterministic Algorithm for Systems of Quadratic Polynomials over F2\mathbb{F}_2

This article discusses a simple deterministic algorithm for solving quadratic Boolean systems which is essentially a special case of more sophisticated methods. The main idea fits in a single sentence: guess enough variables so that the remaining quadratic equations can be solved by linearization (i.e. by considering each remaining monomial as an independent variable and solving the resulting linear system) and restart until the solution is found. Under strong heuristic assumptions, this finds all the solutions of $m$ quadratic polynomials in $n$ variables with $\mathcal{\tilde O}({2^{n-\sqrt{2m}}})$ operations. Although the best known algorithms require exponentially less time, the present technique has the advantage of being simpler to describe and easy to implement. In strong contrast with the state-of-the-art, it is also quite efficient in practice

Machine learning for function synthesis

Function synthesis is the process of automatically constructing functions that satisfy a given specification. The space of functions as well as the format of the specifications vary greatly with each area of application. In this thesis, we consider synthesis in the context of satisfiability modulo theories. Within this domain, the goal is to synthesise mathematical expressions that adhere to abstract logical formulas. These types of synthesis problems find many applications in the field of computer-aided verification. One of the main challenges of function synthesis arises from the combinatorial explosion in the number of potential candidates within a certain size. The hypothesis of this thesis is that machine learning methods can be applied to make function synthesis more tractable. The first contribution of this thesis is a Monte-Carlo based search method for function synthesis. The search algorithm uses machine learned heuristics to guide the search. This is part of a reinforcement learning loop that trains the machine learning models with data generated from previous search attempts. To increase the set of benchmark problems to train and test synthesis methods, we also present a technique for generating synthesis problems from pre-existing satisfiability modulo theories problems. We implement the Monte-Carlo based synthesis algorithm and evaluate it on standard synthesis benchmarks as well as our newly generated benchmarks. An experimental evaluation shows that the learned heuristics greatly improve on the baseline without trained models. Furthermore, the machine learned guidance demonstrates comparable performance to CVC5 and, in some experiments, even surpasses it. Next, this thesis explores the application of machine learning to more restricted function synthesis domains. We hypothesise that narrowing the scope enables the use of machine learning techniques that are not possible in the general setting. We test this hypothesis by considering the problem of ranking function synthesis. Ranking functions are used in program analysis to prove termination of programs by mapping consecutive program states to decreasing elements of a well-founded set. The second contribution of this dissertation is a novel technique for synthesising ranking functions, using neural networks. The key insight is that instead of synthesising a mathematical expression that represents a ranking function, we can train a neural network to act as a ranking function. Hence, the synthesis procedure is replaced by neural network training. We introduce Neural Termination Analysis as a framework that leverages this. We train neural networks from sampled execution traces of the program we want to prove terminating. We enforce the synthesis specifications of ranking functions using the loss function and network design. After training, we use symbolic reasoning to formally verify that the resulting function is indeed a correct ranking function for the target program. We demonstrate that our method succeeds in synthesising ranking functions for programs that are beyond the reach of state-of-the-art tools. This includes programs with disjunctions and non-linear expressions in the loop guards

IGATY: an archetype-based interactive generative abstraction system focusing on museum interior archetypes

Archetype in Greek means an original model that prevails in all later forms of variations, combinations, and transformations. In the field of design, types and archetypes have been used as an analytical tool; unfortunately, archetypes have not been perceived as promising prospects in the search for creative ideas, and the dynamic transformative quality embedded in archetypes has not been fully utilized among students and designers. Despite its inherent potential as sources of ideas for future invention, a number of scholars have criticized the typological approach to design for its exclusive nature primarily due to a misunderstanding of its fundamental structure. This dissertation aims at clarifying this misconception and explores a method that involves taking advantage of the malleable structure of archetypes. In Part 1 of this dissertation, I redefine the malleable structure of archetypes as a dual structure in which two contrasting yet equally crucial elements coexist: a core signal and a set of peripherals. The study focuses on verification of this dual structure and identification of core signals and peripherals in the six selected museum interior archetypes as a test set. In Part 2 I explore the archetype’s transformative quality using the interactive genetic algorithm (IGA). The dual structure of museum interior archetypes defined in Part 1 was mapped into the genetic algorithms to design an archetype-based generative abstraction system integrated with the Unity game engine, named IGATY-beta. The focus was to develop a system that would serve as an interactive ideation partner, not as a single-solution-oriented optimization tool. In Part 3 a quasi-experiment was conducted to examine the proposed IGATY-beta system’s educational potential in enhancing creativity in the ideation process. Three teaching scenarios based on three instructional materials were compared: (a) manual sketch-based archetypes exercise; (b) archetypes exercise using the IGATY-beta system displayed on a computer screen; and (c) archetypes exercise using the IGATY-beta system with an opportunity of viewing design in a virtual environment via a HMD. The results suggest the proposed archetype-based generative abstraction system’s positive educational potentials in enhancing creativity in the ideation process. Finally, the implications of the proposed generative abstraction system in the field of design are discussed