Multi-modal surrogates for retrieving and making sense of videos: is synchronization between the multiple modalities optimal?

Video surrogates can help people quickly make sense of the content of a video before downloading or seeking more detailed information. Visual and audio features of a video are primary information carriers and might become important components of video retrieval and video sense-making. In the past decades, most research and development efforts on video surrogates have focused on visual features of the video, and comparatively little work has been done on audio surrogates and examining their pros and cons in aiding users' retrieval and sense-making of digital videos. Even less work has been done on multi-modal surrogates, where more than one modality are employed for consuming the surrogates, for example, the audio and visual modalities. This research examined the effectiveness of a number of multi-modal surrogates, and investigated whether synchronization between the audio and visual channels is optimal. A user study was conducted to evaluate six different surrogates on a set of six recognition and inference tasks to answer two main research questions: (1) How do automatically-generated multi-modal surrogates compare to manually-generated ones in video retrieval and video sense-making? and (2) Does synchronization between multiple surrogate channels enhance or inhibit video retrieval and video sense-making? Forty-eight participants participated in the study, in which the surrogates were measured on the the time participants spent on experiencing the surrogates, the time participants spent on doing the tasks, participants' performance accuracy on the tasks, participants' confidence in their task responses, and participants' subjective ratings on the surrogates. On average, the uncoordinated surrogates were more helpful than the coordinated ones, but the manually-generated surrogates were only more helpful than the automatically-generated ones in terms of task completion time. Participants' subjective ratings were more favorable for the coordinated surrogate C2 (Magic A + V) and the uncoordinated surrogate U1 (Magic A + Storyboard V) with respect to usefulness, usability, enjoyment, and engagement. The post-session questionnaire comments demonstrated participants' preference for the coordinated surrogates, but the comments also revealed the value of having uncoordinated sensory channels

“A new cartography of the world” : Of participatory sense-making within ecological sound art and interdisciplinary collaboration

This presentation discusses two examples of participatory sense-making, that fall within the ecological-enactive trajectory of my artistic research. The first work is Tapeshavet, a collaboration between composer Anders Hultqvist, poet Gunnar D. Hansson and ecological sound artist Stefan Östersjö and I, with the first output premiered at GAS festival in Sweden in autumn 2018 and a larger work set for premiere in 2019-20. The second is The cartographer; a staged performance created with actress Liv Kaastrup Vesterskov at Lund University‘s Inter Arts Center, Sweden. The work falls under my PhD research which draws on exploration of agency, and the dynamics between the practices of performer, composer and curator in the field of contemporary music and sound art. Through various case studies I look at how an ecological-enactive perspective on musical practice can challenge current understandings of the aforementioned agencies and what artistic methods can be employed to explore a fluid understanding of these roles. The collaborations draw on field-recording as a methodological tool, as well as activation as a method. I have developed the latter since winter 2016-2017, through which “a musician's multi-modal listening is employed in a fleshy listening” (Stefánsdóttir & Östersjö, forthcoming). Through analysis this presentation is set to unpack the processes of such participatory sense-making. Based on this I put forth and argue for the concept of multi-entity performance (Rawlings 2019, Stefánsdóttir 2019) as a technique as well as analytical and conceptual stance

When fiction trumps truth : what ‘post-truth’ and ‘alternative facts’ mean for management studies

In this essay, we explore the notions of ‘post-truth’ and ‘alternative facts’ for management studies. Adopting a pragmatist perspective, we argue that there is no intrinsically accurate language in terms of which to refer to reality. Language, rather, is a tool that enables agents to grab hold of causal forces and intervene in the world. ‘Alternative facts’ can be created by multimodal communication to highlight different aspects of the world for the purpose of political mobilization and legitimacy. ‘Post-truth’ politics reveals the fragmentation of the language game in which mainstream politics has been hitherto conducted. Using the communicative acts of businessman-turned-politician President Trump and his aides, as a prompt, we explore the implications that ‘alternative facts’ and ‘post-truth’ have for today’s management scholarship. We argue that management scholars should unpack how managers navigate strategic action and communication, and how the creation of alternative realities is accomplished in conditions of informational abundance and multimodal communication

Free Search of real value or how to make computers think

This book introduces in detail Free Search - a novel advanced method for search and optimisation. It also deals with some essential questions that have been raised in a strong debate following the publication of this method in journal and conference papers. In the light of this debate, Free Search deserves serious attention, as it appears to be superior to other competitive methods in the context of the experimental results obtained. This superiority is not only quantitative in terms of the actual optimal value found but also qualitative in terms of independence from initial conditions and adaptation capabilities in an unknown environment

The Disunity of Consciousness

It is commonplace for both philosophers and cognitive scientists to express their allegiance to the "unity of consciousness". This is the claim that a subject’s phenomenal consciousness, at any one moment in time, is a single thing. This view has had a major influence on computational theories of consciousness. In particular, what we call single-track theories dominate the literature, theories which contend that our conscious experience is the result of a single consciousness-making process or mechanism in the brain. We argue that the orthodox view is quite wrong: phenomenal experience is not a unity, in the sense of being a single thing at each instant. It is a multiplicity, an aggregate of phenomenal elements, each of which is the product of a distinct consciousness-making mechanism in the brain. Consequently, cognitive science is in need of a multi-track theory of consciousness; a computational model that acknowledges both the manifold nature of experience, and its distributed neural basis

Modal Linear Logic in Higher Order Logic, an experiment in Coq

The sequent calculus of classical modal linear logic KDT 4lin is coded in the higher order logic using the proof assistant COQ. The encoding has been done using two-level meta reasoning in Coq. KDT 4lin has been encoded as an object logic by inductively defining the set of modal linear logic formulas, the sequent relation on lists of these formulas, and some lemmas to work with lists.This modal linear logic has been argued to be a good candidate for epistemic applications. As examples some epistemic problems have been coded and proven in our encoding in Coq::the problem of logical omniscience and an epistemic puzzle: ’King, three wise men and five hats’