Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Child development and the aims of road safety education

Pedestrian accidents are one of the most prominent causes of premature injury, handicap and death in the modern world. In children, the problem is so severe that pedestrian accidents are widely regarded as the most serious of all health risks facing children in developed countries. Not surprisingly, educational measures have long been advocated as a means of teaching children how to cope with traffic and substantial resources have been devoted to their development and provision. Unfortunately, there seems to be a widespread view at the present time that education has not achieved as much as had been hoped and that there may even be quite strict limits to what can be achieved through education. This would, of course, shift the emphasis away from education altogether towards engineering or urban planning measures aimed at creating an intrinsically safer environment in which the need for education might be reduced or even eliminated. However, whilst engineering measures undoubtedly have a major role to play in the effort to reduce accidents, this outlook is both overly optimistic about the benefits of engineering and overly pessimistic about the limitations of education. At the same time, a fresh analysis is clearly required both of the aims and methods of contemporary road safety education. The present report is designed to provide such an analysis and to establish a framework within which further debate and research can take place

Evaluating humanoid embodied conversational agents in mobile guide applications

Evolution in the area of mobile computing has been phenomenal in the last few years. The exploding increase in hardware power has enabled multimodal mobile interfaces to be developed. These interfaces differ from the traditional graphical user interface (GUI), in that they enable a more “natural” communication with mobile devices, through the use of multiple communication channels (e.g., multi-touch, speech recognition, etc.). As a result, a new generation of applications has emerged that provide human-like assistance in the user interface (e.g., the Siri conversational assistant (Siri Inc., visited 2010)). These conversational agents are currently designed to automate a number of tedious mobile tasks (e.g., to call a taxi), but the possible applications are endless. A domain of particular interest is that of Cultural Heritage, where conversational agents can act as personalized tour guides in, for example, archaeological attractions. The visitors to historical places have a diverse range of information needs. For example, casual visitors have different information needs from those with a deeper interest in an attraction (e.g., - holiday learners versus students). A personalized conversational agent can access a cultural heritage database, and effectively translate data into a natural language form that is adapted to the visitor’s personal needs and interests. The present research aims to investigate the information needs of a specific type of visitors, those for whom retention of cultural content is important (e.g., students of history, cultural experts, history hobbyists, educators, etc.). Embodying a conversational agent enables the agent to use additional modalities to communicate this content (e.g., through facial expressions, deictic gestures, etc.) to the user. Simulating the social norms that guide the real-world human-to-human interaction (e.g., adapting the story based on the reactions of the users), should at least theoretically optimize the cognitive accessibility of the content. Although a number of projects have attempted to build embodied conversational agents (ECAs) for cultural heritage, little is known about their impact on the users’ perceived cognitive accessibility of the cultural heritage content, and the usability of the interfaces they support. In particular, there is a general disagreement on the advantages of multimodal ECAs in terms of users’ task performance and satisfaction over nonanthropomorphised interfaces. Further, little is known about what features influence what aspects of the cognitive accessibility of the content and/or usability of the interface. To address these questions I studied the user experiences with ECA interfaces in six user studies across three countries (Greece, UK and USA). To support these studies, I introduced: a) a conceptual framework based on well-established theoretical models of human cognition, and previous frameworks from the literature. The framework offers a holistic view of the design space of ECA systems b) a research technique for evaluating the cognitive accessibility of ECA-based information presentation systems that combine data from eye tracking and facial expression recognition. In addition, I designed a toolkit, from which I partially developed its natural language processing component, to facilitate rapid development of mobile guide applications using ECAs. Results from these studies provide evidence that an ECA, capable of displaying some of the communication strategies (e.g., non-verbal behaviours to accompany linguistic information etc.) found in the real-world human guidance scenario, is not affecting and effective in enhancing the user’s ability to retain cultural content. The findings from the first two studies, suggest than an ECA has no negative/positive impact on users experiencing content that is similar (but not the same) across different locations (see experiment one, in Chapter 7), and content of variable difficulty (see experiment two, in Chapter 7). However, my results also suggest that improving the degree of content personalization and the quality of the modalities used by the ECA can result in both effective and affecting human-ECA interactions. Effectiveness is the degree to which an ECA facilitates a user in accomplishing the navigation and information tasks. Similarly, affecting is the degree to which the ECA changes the quality of the user’s experience while accomplishing the navigation and information tasks. By adhering to the above rules, I gradually improved my designs and built ECAs that are affecting. In particular, I found that an ECA can affect the quality of the user’s navigation experience (see experiment three in Chapter 7), as well as how a user experiences narrations of cultural value (see experiment five, in Chapter 8). In terms of navigation, I found sound evidence that the strongest impact of the ECAs nonverbal behaviours is on the ability of users to correctly disambiguate the navigation of an ECA instructions provided by a tour guide system. However, my ECAs failed to become effective, and to elicit enhanced navigation or retention performances. Given the positive impact of ECAs on the disambiguation of navigation instructions, the lack of ECA-effectiveness in navigation could be attributed to the simulated mobile conditions. In a real outdoor environment, where users would have to actually walk around the castle, an ECA could have elicited better navigation performance, than a system without it. With regards to retention performance, my results suggest that a designer should not solely consider the impact of an ECA, but also the style and effectiveness of the question-answering (Q&A) with the ECA, and the type of user interacting with the ECA (see experiments four and six, in Chapter 8). I found that that there is a correlation between how many questions participants asked per location for a tour, and the information they retained after the completion of the tour. When participants were requested to ask the systems a specific number of questions per location, they could retain more information than when they were allowed to freely ask questions. However, the constrained style of interaction decreased their overall satisfaction with the systems. Therefore, when enhanced retention performance is needed, a designer should consider strategies that should direct users to ask a specific number of questions per location for a tour. On the other hand, when maintaining the positive levels of user experiences is the desired outcome of an interaction, users should be allowed to freely ask questions. Then, the effectiveness of the Q&A session is of importance to the success/failure of the user’s interaction with the ECA. In a natural-language question-answering system, the system often fails to understand the user’s question and, by default, it asks the user to rephrase again. A problem arises when the system fails to understand a question repeatedly. I found that a repetitive request to rephrase the same question annoys participants and affects their retention performance. Therefore, in order to ensure effective human-ECA Q&A, the repeat messages should be built in a way to allow users to figure out how to ask the system questions to avoid improper responses. Then, I found strong evidence that an ECA may be effective for some type of users, while for some others it may be not. I found that an ECA with an attention-grabbing mechanism (see experiment six, in Chapter 8), had an inverse effect on the retention performance of participants with different gender. In particular, it enhanced the retention performance of the male participants, while it degraded the retention performance of the female participants. Finally, a series of tentative design recommendations for the design of both affecting and effective ECAs in mobile guide applications in derived from the work undertaken. These are aimed at ECA researchers and mobile guide designers

​​The Effects of Viewpoint, Motion, and Affordance Priming on Perceptual Learning of Feelies​

According to Gibson and Gibson (1955) perceptual learning is a process of developing the skill to differentiate previously undifferentiated but available information. The initial investigations focused on object identification, lacking a behaviorally relevant functional task. In the current study participants learned to differentiate between novel objects (feelies). To test the role of visual exploration objects were viewed from either a side or a top view and were displayed as either static pictures or rotating about a vertical axis. In Experiment 1 a simple object discrimination task was used. Perfect accuracy was achieved sooner in static conditions compared to motion conditions regardless of viewpoint, suggesting that visual exploration was not necessary. Experiment 2 investigated if a functionally relevant task would necessitate the usage of exploratory activity for perceptual learning. Three priming conditions were included to provide task contexts of varying behavioral relevance. Participants were required to 1) think of potential uses (i.e., affordances) for the feelies, or 2) think of one specific use provided by the experimenter, or 3) were asked to describe the object’s physical appearance using semantic labels. The opportunity to visually explore objects in varied ways benefited learning the most in the condition in which observers had to come up with potential uses for the objects. This prime promoted functionally relevant, deep levels of processing. The most efficient and stable pattern of learning was observed when participants actively generated uses for moving objects that were shown from the side view. It was concluded that exploratory activity facilitates perceptual learning of affordances

Node-Based Native Solution to Procedural Game Level Generation

A Geração Procedural de Conteúdo (PCG) aplicada ao domínio do desenvolvimento de jogos tem se tornado um tópico proeminente, com um número crescente de implementações e aplicações. Soluções de PCG standalone e plugin, regidas por interfaces baseadas em nós e outros modelos de alto nível, enfrentam limitações em termos de integração, interatividade e responsividade quando inseridas no processo de desenvolvimento de jogos. Essas limitações afetam a experiência do utilizador e inibem o verdadeiro potencial que estes sistemas podem oferecer. Adotando uma metodologia de Action-Research, realizou-se um estudo preliminar com entrevistas a especialistas da área. A avaliação da pertinência desta metodologia nativa e da abordagem visual mais adequada para a sua interface foi efetuada através de uma série de protótipos. Posteriormente, foi implementado um protótipo funcional e conduzido um estudo de caso com uma amostra constituída por um grupo de especialistas em PCG e de desenvolvedores de jogos. Os participantes realizaram uma série de exercícios que estavam documentados com os respetivos tutoriais. Após a conclusão dos exercícios propostos, os participantes avaliaram a relevância da solução e da experiência do utilizador através de um questionário. No desenvolvimento de uma metodologia nativa de PCG baseado em nós, integrado no motor de jogo, identificamos limitações e concluímos que existem diversos desafios ainda por superar no que diz respeito a uma implementação completa de um sistema complexo e amplo.Procedural Content Generation (PCG) applied to game development has become a prominent topic with increasing implementations and use cases. However, existing standalone and plugin PCG solutions, which use Node-based interfaces and other high-level approaches, face limitations in integration, interactivity, and responsiveness within the game development pipeline. These limitations hinder the overall user experience and restrain the true potential of PCG systems. Adopting an Action-Research methodology, a preliminary interview was conducted with experts in the field. The relevance assessment of this native methodology and the most suitable visual approach for its interface was carried out through a series of prototypes. Subsequently, a functional prototype was implemented, and a case study was conducted using a sample consisting of a group of PCG experts and game developers. The participants performed a series of exercises documented with the respective tutorials. After completing the exercises, the solution's relevancy and user experience were evaluated through a questionnaire. In developing a native node-based PCG methodology integrated into the game engine, we identified limitations. We concluded that several challenges are yet to be overcome regarding fully implementing a complex and extensive system

Applying psychological science to the CCTV review process: a review of cognitive and ergonomic literature

As CCTV cameras are used more and more often to increase security in communities, police are spending a larger proportion of their resources, including time, in processing CCTV images when investigating crimes that have occurred (Levesley & Martin, 2005; Nichols, 2001). As with all tasks, there are ways to approach this task that will facilitate performance and other approaches that will degrade performance, either by increasing errors or by unnecessarily prolonging the process. A clearer understanding of psychological factors influencing the effectiveness of footage review will facilitate future training in best practice with respect to the review of CCTV footage. The goal of this report is to provide such understanding by reviewing research on footage review, research on related tasks that require similar skills, and experimental laboratory research about the cognitive skills underpinning the task. The report is organised to address five challenges to effectiveness of CCTV review: the effects of the degraded nature of CCTV footage, distractions and interrupts, the length of the task, inappropriate mindset, and variability in people’s abilities and experience. Recommendations for optimising CCTV footage review include (1) doing a cognitive task analysis to increase understanding of the ways in which performance might be limited, (2) exploiting technology advances to maximise the perceptual quality of the footage (3) training people to improve the flexibility of their mindset as they perceive and interpret the images seen, (4) monitoring performance either on an ongoing basis, by using psychophysiological measures of alertness, or periodically, by testing screeners’ ability to find evidence in footage developed for such testing, and (5) evaluating the relevance of possible selection tests to screen effective from ineffective screener

Evaluating visually grounded language capabilities using microworlds

Deep learning has had a transformative impact on computer vision and natural language processing. As a result, recent years have seen the introduction of more ambitious holistic understanding tasks, comprising a broad set of reasoning abilities. Datasets in this context typically act not just as application-focused benchmark, but also as basis to examine higher-level model capabilities. This thesis argues that emerging issues related to dataset quality, experimental practice and learned model behaviour are symptoms of the inappropriate use of benchmark datasets for capability-focused assessment. To address this deficiency, a new evaluation methodology is proposed here, which specifically targets in-depth investigation of model performance based on configurable data simulators. This focus on analysing system behaviour is complementary to the use of monolithic datasets as application-focused comparative benchmarks. Visual question answering is an example of a modern holistic understanding task, unifying a range of abilities around visually grounded language understanding in a single problem statement. It has also been an early example for which some of the aforementioned issues were identified. To illustrate the new evaluation approach, this thesis introduces ShapeWorld, a diagnostic data generation framework. Its design is guided by the goal to provide a configurable and extensible testbed for the domain of visually grounded language understanding. Based on ShapeWorld data, the strengths and weaknesses of various state-of-the-art visual question answering models are analysed and compared in detail, with respect to their ability to correctly handle statements involving, for instance, spatial relations or numbers. Finally, three case studies illustrate the versatility of this approach and the ShapeWorld generation framework: an investigation of multi-task and curriculum learning, a replication of a psycholinguistic study for deep learning models, and an exploration of a new approach to assess generative tasks like image captioning.Qualcomm Award Premium Research Studentship, Engineering and Physical Sciences Research Council Doctoral Training Studentshi

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD