Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning

Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.Comment: To appear in ICCV 2017. Total 17 pages including the supplementary materia

"Scholarly Hypertext: Self-Represented Complexity"

Scholarly hypertexts involve argument and explicit selfquestioning, and can be distinguished from both informational and literary hypertexts. After making these distinctions the essay presents general principles about attention, some suggestions for self-representational multi-level structures that would enhance scholarly inquiry, and a wish list of software capabilities to support such structures. The essay concludes with a discussion of possible conflicts between scholarly inquiry and hypertext

Material Sight: A Sensorium for Fundamental Physics

Often our attempts to connect to the spatial and temporal scales of fundamental physics - from the subatomic to the multiverse - provoke a form of perceptual vertigo, especially for non-scientists. When we approach ideas of paralysing abstraction through the perceptual range of our sensing bodies, a ‘phenomenological dissonance’ can be said to be invoked, between material presence and radical remoteness. This relational dynamic, between materiality and remoteness, formed the conceptual springboard for 'Material Sight' (2016-2018), a research project based at three world-leading facilities for fundamental physics, that brought to fruition a body of photographic objects, film works and immersive soundscape that re-presented the spaces of fundamental physics as sites of material encounter. The research was premised on a paradoxical desire to create a sensorium for fundamental physics, asking if photography, film and sound can embody the spaces of experimental science and present them back to scientists and non-scientists alike, not as illustrations of the technical sublime but as sites of phenomenological encounter. This article plots the key conceptual coordinates of 'Material Sight' and looks at how the project’s methodological design – essentially the production of knowledge through the 'act of looking' – emphatically resisted the gravitational pull of art to be instrumentalised as an illustrative device within scientific contexts