Building Scene Models by Completing and Hallucinating Depth and Semantics

Building 3D scene models has been a longstanding goal of computer vision. The great progress in depth sensors brings us one step closer to achieving this in a single shot. However, depth sensors still produce imperfect measurements that are sparse and contain holes. While depth completion aims at tackling this issue, it ignores the fact that some regions of the scene are occluded by the foreground objects. Building a scene model would therefore require to hallucinate the depth behind these objects. In contrast with existing methods that either rely on manual input, or focus on the indoor scenario, we introduce a fully-automatic method to jointly complete and hallucinate depth and semantics in challenging outdoor scenes. To this end, we develop a two-layer model representing both the visible information and the hidden one. At the heart of our approach lies a formulation based on the Mumford-Shah functional, for which we derive an effective optimization strategy. Our experiments evidence that our approach can accurately fill the large holes in the input depth maps, segment the different kinds of objects in the scene, and hallucinate the depth and semantics behind the foreground objects

Using Agent-Based Modelling to Inform Policy – What Could Possibly Go Wrong?

© 2019, Springer Nature Switzerland AG. Scientific modelling can make things worse, as in the case of the North Atlantic Cod Fisheries Collapse. Some of these failures have been attributed to the simplicity of the models used compared to what they are trying to model. MultiAgent-Based Simulation (MABS) pushes the boundaries of what can be simulated, prompting many to assume that it can usefully inform policy, even in the face of complexity. That said, MABS also brings with it new difficulties and potential confusions. This paper surveys some of the pitfalls that can arise when MABS analysts try to do this. Researchers who claim (or imply) that MABS can reliably predict are criticised in particular. However, an alternative is suggested – that of using MABS for a kind of uncertainty analysis – identifying some of the possible ways a policy can go wrong (or indeed go right). A fisheries example is given. This alternative may widen, rather than narrow, the range of evidence and possibilities that are considered, which could enrich the policy-making process. We call this Reflexive Possibilistic Modelling

Capturing the sounds of an urban greenspace

Acoustic data can be a source of important information about events and the environment in modern cities. To date, much of the focus has been on monitoring noise pollution, but the urban soundscape contains a rich variety of signals about both human and natural phenomena. We describe the CitySounds project, which has installed enclosed sensor kits at several locations across a heavily used urban greenspace in the city of Edinburgh. The acoustic monitoring components regularly capture short clips in real-time of both ultrasonic and audible noises, for example encompassing bats, birds and other wildlife, traffic, and human. The sounds are complemented by collecting other data from sensors, such as temperature and relative humidity. To ensure privacy and compliance with relevant legislation, robust methods render completely unintelligible any traces of voice or conversation that may incidentally be overheard by the sensors. We have adopted a variety of methods to encourage community engagement with the audio data and to communicate the richness of urban soundscapes to a general audience

Mapping the global network of fisheries science collaboration

As socio‐environmental problems have proliferated over the past decades, one narrative which has captured the attention of policymakers and scientists has been the need for collaborative research that spans traditional boundaries. Collaboration, it is argued, is imperative for solving these problems. Understanding how collaboration is occurring in practice is important, however, and may help explain the idea space across a field. In an effort to make sense of the shape of fisheries science, here we construct a co‐authorship network of the field, from a data set comprising 73,240 scientific articles, drawn from 50 journals and published between 2000 and 2017. Using a combination of social network analysis and machine learning, the work first maps the global structure of scientific collaboration amongst fisheries scientists at the author, country and institutional levels. Second, it uncovers the hidden subgroups—here country clusters and communities of authors—within the network, detailing also the topical focus, publication outlets and relative impact of the largest fisheries science communities. We find that whilst the fisheries science network is becoming more geographically extensive, it is simultaneously becoming more intensive. The uncovered network exhibits characteristics suggestive of a thin style of collaboration, and groupings that are more regional than they are global. Although likely shaped by an array of overlapping micro‐ and macro‐level factors, the analysis reveals a number of political–economic patterns that merit reflection by both fisheries scientists and policymakers

Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames

Abstract. Optical flow research has made significant progress in recent years and it can now be computed efficiently and accurately for many images. How-ever, complex motions, large displacements, and difficult imaging conditions are still problematic. In this paper, we present a framework for estimating optical flow which leads to improvements on these difficult cases by 1) estimating occlusions and 2) using additional temporal information. First, we divide the image into dis-crete triangles and show how this allows for occluded regions to be naturally esti-mated and directly incorporated into the optimization algorithm. We additionally propose a novel method of dealing with temporal information in image sequences by using “inertial estimates ” of the flow. These estimates are combined using a classifier-based fusion scheme, which significantly improves results. These con-tributions are evaluated on three different optical flow datasets, and we achieve state-of-the-art results on MPI-Sintel.

Bat detective—Deep learning tools for bat acoustic signal detection

Passive acoustic sensing has emerged as a powerful tool for quantifying anthropogenic impacts on biodiversity, especially for echolocating bat species. To better assess bat population trends there is a critical need for accurate, reliable, and open source tools that allow the detection and classification of bat calls in large collections of audio recordings. The majority of existing tools are commercial or have focused on the species classification task, neglecting the important problem of first localizing echolocation calls in audio which is particularly problematic in noisy recordings. We developed a convolutional neural network based open-source pipeline for detecting ultrasonic, full-spectrum, search-phase calls produced by echolocating bats. Our deep learning algorithms were trained on full-spectrum ultrasonic audio collected along road-transects across Europe and labelled by citizen scientists from www.batdetective.org. When compared to other existing algorithms and commercial systems, we show significantly higher detection performance of search-phase echolocation calls with our test sets. As an example application, we ran our detection pipeline on bat monitoring data collected over five years from Jersey (UK), and compared results to a widely-used commercial system. Our detection pipeline can be used for the automatic detection and monitoring of bat populations, and further facilitates their use as indicator species on a large scale. Our proposed pipeline makes only a small number of bat specific design decisions, and with appropriate training data it could be applied to detecting other species in audio. A crucial novelty of our work is showing that with careful, non-trivial, design and implementation considerations, state-of-the-art deep learning methods can be used for accurate and efficient monitoring in audio

Towards Eco-Centric Interaction: Urban Playful Interventions in the Anthropocene

The twin crises of nature and climate is supported by overwhelming scientific evidence and increasing public concern about long-term and potentially irreversible consequences of the Anthropocene. People and wildlife living today, as well as future generations, are at risk unless urgent action is taken to reverse the loss of plants, insects and other creatures on which we depend for food, clean water, and a stable climate. As urban dwellings become the main concentration of citizens, we thought of addressing these issues through a series of urban interventions aiming at educating and enhancing the user\u2019s biodiversity and sustainability awareness through play and reflection. We provide a list of preliminary insights, discussed, and shared trying to frame how these new interventions can contribute to a panorama where playful interactions in smart cities can inspire sustainable and respectful attitudes towards nature. We conclude with a constructive conversation about playful urban approaches aimed at understanding how the interaction process could be re-centered to promote environmental protection and ecological consciousness on the part of technology users. Our case studies strive to reconcile concepts and theories, about ecological computing, more than human design, biodiversity actions and nature awareness and conservation in urban settings, for the design of urban playful and enjoyable systems that promote environmental protection and ecological consciousness on the part of technology users