5 research outputs found
LIPIcs, Volume 261, ICALP 2023, Complete Volume
LIPIcs, Volume 261, ICALP 2023, Complete Volum
Supporting multimedia user interface design using mental models and representational expressiveness
This thesis addresses the problem of output media allocation in the design of
multimedia user interfaces. The literature survey identifies a formal definition of the
representational capabilities of different media.as important in this task. Equally
important, though less prominent in the literature, is that the correct mental model of
a domain is paramount for the successful completion of tasks.
The thesis proposes an original linguistic and cognitive based descriptive framework,
in two parts. The first part defines expressiveness, the amount of representational
abstraction a medium provides over any domain. The second part describes how this
expressiveness is linked to the mental models that media induce, and how this in turn
affects task performance. It is postulated that the mental models induced by different
media, will reflect the abstractive representation those media offer over the task
domain. This must then be matched to the abstraction required by tasks to allow them
to be effectively accomplished.
A 34 subject experiment compares five media, of two levels of expressiveness, over a
range of tasks, in a complex and dynamic domain. The results indicate that
expressiveness may allow media to be matched more closely to tasks, if the mental
models they are known to induce are considered.
Finally, the thesis proposes a tentative framework for media allocation, and two
example interfaces are designed using this framework. This framework is based on
the matching of expressiveness to the abstraction of a domain required by tasks. The
need for the methodology to take account of the user's cognitive capabilities is
stressed, and the experimental results are seen as the beginning of this procedure
Statistical models for natural scene data
This thesis considers statistical modelling of natural image data. Obtaining advances in this field can have significant impact for both engineering applications, and for the understanding of the human visual system. Several recent advances in natural image modelling have been obtained with the use of unsupervised feature learning. We consider a class of such models, restricted Boltzmann machines (RBMs), used in many recent state-of-the-art image models. We develop extensions of these stochastic artificial neural networks, and use them as a basis for building more effective image models, and tools for computational vision. We first develop a novel framework for obtaining Boltzmann machines, in which the hidden unit activations co-transform with transformed input stimuli in a stable and predictable way throughout the network. We define such models to be transformation equivariant. Such properties have been shown useful for computer vision systems, and have been motivational for example in the development of steerable filters, a widely used classical feature extraction technique. Translation equivariant feature sharing has been the standard method for scaling image models beyond patch-sized data to large images. In our framework we extend shallow and deep models to account for other kinds of transformations as well, focusing on in-plane rotations. Motivated by the unsatisfactory results of current generative natural image models, we take a step back, and evaluate whether they are able to model a subclass of the data, natural image textures. This is a necessary subcomponent of any credible model for visual scenes. We assess the performance of a state- of-the-art model of natural images for texture generation, using a dataset and evaluation techniques from in prior work. We also perform a dissection of the model architecture, uncovering the properties important for good performance. Building on this, we develop structured extensions for more complicated data comprised of textures from multiple classes, using the single-texture model architecture as a basis. These models are shown to be able to produce state-of-the-art texture synthesis results quantitatively, and are also effective qualitatively. It is demonstrated empirically that the developed multiple-texture framework provides a means to generate images of differently textured regions, more generic globally varying textures, and can also be used for texture interpolation, where the approach is radically dfferent from the others in the area. Finally we consider visual boundary prediction from natural images. The work aims to improve understanding of Boltzmann machines in the generation of image segment boundaries, and to investigate deep neural network architectures for learning the boundary detection problem. The developed networks (which avoid several hand-crafted model and feature designs commonly used for the problem), produce the fastest reported inference times in the literature, combined with state-of-the-art performance
Actes de la 6e confĂ©rence conjointe JournĂ©es d'Ătudes sur la Parole (JEP, 33e Ă©dition), Traitement Automatique des Langues Naturelles (TALN, 27e Ă©dition), Rencontre des Ătudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RĂCITAL, 22e Ă©dition. Volume 2 : Traitement Automatique des Langues Naturelles
@ 6Úme conférence conjointe: JEP-TALN-RECITAL 2020no abstrac