Miraikan SC corpus : A trial for data collection in a semi-open and semi-controlled environment

National Institute of Informatics/SOKENDAINational Institute of InformaticsWaseda UniversityThe University of Shiga PrefectureLREC 2018 Special Speech Sessions "Speech Resources Collection in Real-World Situations"; Phoenix Seagaia Conference Center, Miyazaki; 2018-05-09This paper shows the concept and design of our Miraikan SC Corpus. A well-structured and well-prepared corpus would be useful to engineers for understanding the mechanism of speech production and the nature of social interaction with regard to informing the design of their systems. Applications of the corpus range from speech recognition and dialogue processing to human-agent interaction systems, among others. We started collecting audio-visual data using multiple video cameras and microphones in October 2012 at a science museum in Tokyo, Japan. In this paper, we describe the reason why we chose the museum as a research field for data collection, how we audio-video-recorded the interactions, and how we dealt with personal information in the data set, such as participants’ names, jobs, and places of residence

Engaging socially-vulnerable communities in science: exploring Science&Art approaches

Social inclusion in science is a complex issue. During the past decades, research centres, science centres, museums and other institutions invested in science communication aiming to promote cultural activities to diverse audiences. Despite this investment, science communicators from all over the world face the same challenge: how to reach citizens that are not interested in science? The main goals for this project were to explore innovative techniques to engage socially-vulnerable communities with science, and propose a model of science communication built on this practice-based research. The project, named “Embodying Memories”, was developed in a collaborative way between science partners (IGC - Instituto Gulbenkian de Ciência, iNOVA Media Lab), art partners (museum from FCG – Fundação Calouste Gulbenkian) and administrative partners (Câmara Municipal de Oeiras). The target audience, a senior community of women, most illiterate and migrant from Africa, was involved on the project plan since early stages, starting with the topic choice - Memory. The project implementation consisted of eight sessions that took place over a period of more than two months in 2018, covering several themes related to memory and brain. Diverse formats were used for the session’s activities, from scientific presentations, neuroscience stories or study cases, community memories sharing, to more interactive activities stimulating body movement, abstraction and self-expression. Besides in-door sessions at the migrant support centre, a visit to the FCG museum and a visit to IGC laboratories were organized, and a project public presentation was performed. The project was qualitatively evaluated to identify changes in awareness, knowledge, engagement, attitude and social inclusion, which was made by the analysis of field notes, attendance record, pre/post assessment focus group, community project evaluation, project narrative, and public presentation content. Overall, it was considered that the project had a moderate achievement, from a balance between very high attendance and willingness to participate in new cultural experiences, high engagement with the project, moderate increase in knowledge about neuroscience, and some increase in awareness and engagement with science, stimulation of curiosity, abstraction and self-expression. To achieve a high level of engagement, a dynamic equilibrium was constantly in a trial between the six axes of the project (science education, art education, cultural entertainment, social inclusion, mental health promotion, institutional advertising), and respective institutions. The most important project achievements were the fluidity and fruition of the project itself, and the opportunity given to participants to engage with Science & Art, to visit the museum and laboratories, to meet scientists and science instruments. A relevant asset of the project, was the existence of the boundary spanner, which was developed along pre- and during sessions by taking actions, visits, share experiences and events to inhabit the laboratory sphere, the museum sphere, and the community world. The role of the boundary spanner was crucial, yet challenging to balance between how much would be desirable for each partner to stay in and out of their comfort zones and territories. Based on insights gained from the project development and evaluation, a model was proposed to guide science communication projects using Science & Art approaches to promote social inclusion. The model entails the following phases: Phase 1. Design, plan and collaboration; Phase 2. Implementation; and Phase 3. Evaluation.A inclusão social em ciência é um tema complexo. Todavia tem-se assistido nas últimas décadas a um esforço crescente por parte das instituições de investigação científica, dos centros de ciência, museus, e outras organizações, na promoção de atividades culturais dirigidas a públicos diversificados. Apesar deste investimento, por todo o mundo os comunicadores de ciência deparam-se com o mesmo desafio: como chegar a cidadão que não estão interessados em ciência. Os objetivos principais deste projeto foram a exploração de técnicas inovadoras de envolvimento de comunidades socialmente vulneráveis em ciência e a proposta de um modelo de comunicação de ciência decorrente desta investigação de base-prática. O projeto, denominado “Dar Corpo às Memórias”, desenrolou-se de forma colaborativa entre os parceiros científicos (IGC - Instituto Gulbenkian de Ciência, iNOVA Media Lab), artísticos (museu da FCG – Fundação Calouste Gulbenkian) e administrativos (Câmara Municipal de Oeiras). O público-alvo, uma comunidade sénior de mulheres maioritariamente iletradas e migrantes de África, foi envolvido no projeto desde as fases iniciais, começando na própria escolha do tema – Memória. A fase de implementação do projeto consistiu num conjunto de oito sessões, ao longo de mais de dois meses, durante as quais foram abordados vários temas ligados à memória e ao cérebro. As atividades tiveram natureza diversa desde a apresentação de informação científica, narrativa de histórias da neurociência ou casos de estudo interessantes, partilha de memórias das participantes, até atividades mais interativas de estímulo ao movimento, à abstração e autoexpressão. Além das sessões que decorreram no centro de apoio a migrantes, foram também efetuadas duas visitas (ao museu da FCG e aos laboratórios do IGC) e uma apresentação pública do projeto. O projeto foi qualitativamente avaliado para identificar mudanças de consciencialização, conhecimento, envolvimento, atitude e inclusão social, com recurso à análise das notas de campo, registo de assiduidade, pré/pós grupos de foco, avaliação qualitativa feita pela comunidade, narrativa do projeto feita pela comunidade e conteúdo da apresentação pública. De forma global, considerou-se que o impacto do projeto foi moderado, com níveis de participação e abertura a novas experiências culturais muito elevados, elevado envolvimento com o projeto, moderado aumento de conhecimentos nas áreas das neurociências, e algum aumento de consciencialização, envolvimento com a ciência, estímulo da curiosidade, abstração e autoexpressão. Para atingir elevados níveis de envolvimento, foram efetuadas constantes tentativas de equilíbrio dinâmico entre os seis eixos do projeto (educação científica, educação artística, animação cultural, inclusão social, promoção da saúde mental, publicidade institucional) e respetivas instituições. Os maiores sucessos do projeto foram a sua própria fluidez e fruição e a oportunidade dada às participantes de envolvimento com a ciência e a arte, participação numa visita ao museu e noutra aos laboratórios, o encontro com cientistas e os instrumentos da ciência. Uma mais-valia relevante do projeto foi a existência de uma “boundary spanner” – uma pessoa facilitadora de várias valências - , que se foi desenvolvendo durante as fases pré-sessões e durante as sessões, através de ações, visitas, partilha de experiências e eventos para habitar a esfera do laboratório, a esfera do museu e o universo da comunidade. O papel do “boundary spanner” foi crucial, mas também desafiante, na medida em que requereu uma avaliação do quão cada parceiro estava disponível para permanecer ou sair da sua zona de conforto e territórios. Com base nos conhecimentos ganhos durante o desenvolvimento e avaliação deste projeto, foi proposto um modelo para projetos de comunicação de ciência para a promoção da inclusão social com recurso a abordagens de ciência & arte. O modelo consiste nas seguintes fases: Fase 1. Conceção, planeamento e colaboração; Fase 2. Implementação, e Fase 3. Avaliação

Experimental Museology

Experimental Museology scrutinizes innovative endeavours to transform museum interactions with the world. Analysing cutting-edge cases from around the globe, the volume demonstrates how museums can design, apply and assess new modes of audience engagement and participation. Written by an interdisciplinary group of researchers and research-led professionals, the book argues that museum transformations must be focused on conceptualizing and documenting the everyday challenges and choices facing museums, especially in relation to wider social, political and economic ramifications. In order to illuminate the complexity of these challenges, the volume is structured into three related key dimensions of museum practice - namely institutions, representations and users. Each chapter is based on a curatorial design proposed and performed in collaboration between university-based academics and a museum. Taken together, the chapters provide insights into a diversity of geographical contexts, fields and museums, thus building a comprehensive and reflexive repository of design practices and formative experiments that can help strengthen future museum research and design. Experimental Museology will be of great value to academics and students in the fields of museum, gallery and heritage studies, as well as architecture, design, communication and cultural studies. It will also be of interest to museum professionals and anyone else who is interested in learning more about experimentation and design as resources in museums

The Making of Midnight Oil: Exhibition design and the translation of rock music from stage to museum

The ways in which museums construct and negotiate visitor experience are being challenged by expanding and converging modes of spectatorship and representation. This thesis models the relationship between live performance and its exhibition through an examination of the recently emerged genre of the rock music exhibition. The study explores historical antecedents, contemporary practices and explanatory models relating to this phenomenon, arguing that while museums continue to be gathering places for shared encounters and immersive spectatorship, design choices for modes of representation are subject to rapid changes in sound and imaging technologies, and associated meanings of fidelity. Exhibition development practices in museums lack a shared language that accounts for the changing paradigms of knowledge production driving the popular music exhibition phenomenon, meaning that new approaches are required from designers. A case study analysis of the travelling exhibition The Making of Midnight Oil (2014–2017) examines one such project from my designer-researcher’s perspective. Interviews with members of the creative team in different venues, analyses of visitor contributions and design documentation, and a reflexive approach to the theory-practice nexus produce a granular account of the mediation of experience from production to reception. The rock exhibition is examined as a site of negotiable discourses of liveness, authenticity and power, in which design is a key agent of meaning-making. A blended social semiotic framework, newly applied to exhibition design, is used to describe the re-presentation of source phenomena in this emblematic multimodal space. The outcome is an explanatory model of possible responses to the challenge of mediating liveness in the museum, in which exhibition experience is modelled as a dialogic co-construction of gradable meanings between members of a multidisciplinary team, visitors, and institutional and social contexts that are distributed across on-site, off-site, and online places. By identifying how different design choices select certain truth criteria for fidelity of representation, this thesis provides a shared design lexicon of broadened choices for the mediation of live events: a re-imagining of the exhibition design brief as a Modality question. It is designed to foster increased understanding and cross-modal experimentation in interdisciplinary creative teams

Interdisciplinary Insights for Digital Touch Communication

Communication is increasingly moving beyond ‘ways of seeing’ to ‘ways of feeling’. This Open Access book provides social design insights and implications for HCI research and design exploring digitally mediated touch communication. It offers a socially orientated map to help navigate the complex social landscape of digitally mediated touch for communication: from everyday touch-screens, tangibles, wearables, haptics for virtual reality, to the tactile internet of skin. Drawing on literature reviews, new case-study vignettes, and exemplars of digital touch, the book examines the major social debates provoked by digital touch, and investigates social themes central to the communicative potential and societal consequences of digital touch: · Communication environments, capacities and practices · Norms associations and expectations · Presence, absence and connection · Social imaginaries of digital touch · Digital touch ethics and values The book concludes with a discussion of the significance of social understanding and methods in the context of Interdisciplinary collaborations to explore touch, towards the design of digital touch communication, ‘ways of feeling’, that are useable, appropriate, ethical and socially aware

Analysis of visual images in examples of ‘encountered’ science communication

Visual images are often used in science communication activities and events for different purposes and in different contexts. However, there is limited research on this area and the decision made for choosing a visual image often relies on common sense or the personal judgement of a science communicator. Previous studies have highlighted potential effects with respect to a specific visual image property. This research aims to expand on existing research by examining visual images and their accompanying written texts utilised in ‘encountered’ science communication events. A case study approach was adopted to examine selected examples occurring within the years 2014-2018. The analysis is based on discourse analysis of the visual image and accompanying written texts which scrutinises the construction of an intended concept or ideology. The analyses of exemplified cases demonstrate several relationships between a visual image and accompanying elements to express an ideology. As such, there are four distinct, though occasionally overlapping, ways accompanying elements contribute to a visual image namely identification, expansion, implication, and reinforcement. The case studies then serve as the basis for synthesising the research findings in a science communication context. The findings indicate that a visual image or series of visual images can increase the opportunity for initial engagement through large physical size or occupied space, colour and brightness, recognisable elements, and the coherence with other collaborative visual images. It may also prolong the initial attention by being part of a series or continuation, displaying aspects of relevance or applicability, resonating with emotions, and being intriguing or challenging with respect to the interpretation. With regards to achieving learning outcomes, textual/verbal information is often required for the audience to process scientific knowledge relevant to the images. The interdependent combination of visual and textual/verbal information may enable the promotion of an attitude/value or ideological standpoint. Meanwhile, the use of a visual image can provide enjoyment or inspiration, as well as support the improvement of visual skills by default. These findings are summarised as a guide for science communication practitioners to use in the selection of visual images. In addition, there is a suggestion that the use of a visual image analysis framework could be a useful tool for a science communicator in selecting visual images for use in science communication

Recontextualisation in museum displays: refracting discourses over time

This research investigates the representation of people in museums, focusing on the San, South Africa’s first nation. Using a multimodal social semiotic framework, it analyses three exhibitions of the San that were mounted over a period of a hundred years in a natural history museum from 1911 to the present. The research takes into consideration the socio-political background in which the exhibitions were designed, and examines how this manifests in the ways in which the San were represented. The analysis surfaces three dominant discourses, namely evolutionary, ecological as well as a discourse of transformation. These discourses are complex and always in dialogue with one another. The research entailed working with and analysing photographs and drawing on secondary texts of two exhibitions that are no longer open to the public, and analysing an existing exhibition. The data analysis was framed by the semiotic principles of recontextualisation as posited by Bezemer and Kress (2008): selection, social relations and arrangement. Selection refers to the choice of meaning materials for an exhibition. Arrangement refers to the decisions made in the display of the meaning materials (including layout, framing, and foregrounding), and social relations pertain to the social repositioning that takes place in the process of recontextualisation. The research showed how discourses shifted across time, but that dominant discourses such as an evolutionary discourse persisted through the ages and the various exhibitions. By analysing exhibitions of the San against the political backdrop of colonialism, apartheid and post-apartheid this research contributes to an understanding of colonial museums and their exhibitions. It provides suggestions to South African museum practitioners dealing with colonial collections on how to bring a decolonial perspective to exhibitions. The insights gained through this research may enable museum professionals to better understand meaning making and representation in museum display and to contribute to current debates on representation, including ways in which dominant discourses are reflected and refracted in museums. The dialogue between discourses and traces of discourse is of interest within the museum context as well as other contexts of transformation. The research shows that it is possible to map a re-imagining of museum display on the three principles of recontextualisation – selection, arrangement and social relations – in order to see what forms transformation in museum display could take