Wildbook: Crowdsourcing, computer vision, and data science for conservation

Photographs, taken by field scientists, tourists, automated cameras, and incidental photographers, are the most abundant source of data on wildlife today. Wildbook is an autonomous computational system that starts from massive collections of images and, by detecting various species of animals and identifying individuals, combined with sophisticated data management, turns them into high resolution information database, enabling scientific inquiry, conservation, and citizen science. We have built Wildbooks for whales (flukebook.org), sharks (whaleshark.org), two species of zebras (Grevy's and plains), and several others. In January 2016, Wildbook enabled the first ever full species (the endangered Grevy's zebra) census using photographs taken by ordinary citizens in Kenya. The resulting numbers are now the official species census used by IUCN Red List: http://www.iucnredlist.org/details/7950/0. In 2016, Wildbook partnered up with WWF to build Wildbook for Sea Turtles, Internet of Turtles (IoT), as well as systems for seals and lynx. Most recently, we have demonstrated that we can now use publicly available social media images to count and track wild animals. In this paper we present and discuss both the impact and challenges that the use of crowdsourced images can have on wildlife conservation.Comment: Presented at the Data For Good Exchange 201

Synesthesia: Detecting Screen Content via Remote Acoustic Side Channels

We show that subtle acoustic noises emanating from within computer screens can be used to detect the content displayed on the screens. This sound can be picked up by ordinary microphones built into webcams or screens, and is inadvertently transmitted to other parties, e.g., during a videoconference call or archived recordings. It can also be recorded by a smartphone or "smart speaker" placed on a desk next to the screen, or from as far as 10 meters away using a parabolic microphone. Empirically demonstrating various attack scenarios, we show how this channel can be used for real-time detection of on-screen text, or users' input into on-screen virtual keyboards. We also demonstrate how an attacker can analyze the audio received during video call (e.g., on Google Hangout) to infer whether the other side is browsing the web in lieu of watching the video call, and which web site is displayed on their screen

To transduce a zebra finch: interrogating behavioral mechanisms in a model system for speech.

The ability to alter neuronal gene expression, either to affect levels of endogenous molecules or to express exogenous ones, is a powerful tool for linking brain and behavior. Scientists continue to finesse genetic manipulation in mice. Yet mice do not exhibit every behavior of interest. For example, Mus musculus do not readily imitate sounds, a trait known as vocal learning and a feature of speech. In contrast, thousands of bird species exhibit this ability. The circuits and underlying molecular mechanisms appear similar between disparate avian orders and are shared with humans. An advantage of studying vocal learning birds is that the neurons dedicated to this trait are nested within the surrounding brain regions, providing anatomical targets for relating brain and behavior. In songbirds, these nuclei are known as the song control system. Molecular function can be interrogated in non-traditional model organisms by exploiting the ability of viruses to insert genetic material into neurons to drive expression of experimenter-defined genes. To date, the use of viruses in the song control system is limited. Here, we review prior successes and test additional viruses for their capacity to transduce basal ganglia song control neurons. These findings provide a roadmap for troubleshooting the use of viruses in animal champions of fascinating behaviors-nowhere better featured than at the 12th International Congress

A low-cost hyperspectral scanner for natural imaging and the study of animal colour vision above and under water

Hyperspectral imaging is a widely used technology for industrial and scientific purposes, but the high cost and large size of commercial setups have made them impractical for most basic research. Here, we designed and implemented a fully open source and low-cost hyperspectral scanner based on a commercial spectrometer coupled to custom optical, mechanical and electronic components. We demonstrate our scanner's utility for natural imaging in both terrestrial and underwater environments. Our design provides sub-nm spectral resolution between 350-950 nm, including the UV part of the light spectrum which has been mostly absent from commercial solutions and previous natural imaging studies. By comparing the full light spectra from natural scenes to the spectral sensitivity of animals, we show how our system can be used to identify subtle variations in chromatic details detectable by different species. In addition, we have created an open access database for hyperspectral datasets collected from natural scenes in the UK and India. Together with comprehensive online build- and use-instructions, our setup provides an inexpensive and customisable solution to gather and share hyperspectral imaging data

Landmark detection in 2D bioimages for geometric morphometrics: a multi-resolution tree-based approach

The detection of anatomical landmarks in bioimages is a necessary but tedious step for geometric morphometrics studies in many research domains. We propose variants of a multi-resolution tree-based approach to speed-up the detection of landmarks in bioimages. We extensively evaluate our method variants on three different datasets (cephalometric, zebrafish, and drosophila images). We identify the key method parameters (notably the multi-resolution) and report results with respect to human ground truths and existing methods. Our method achieves recognition performances competitive with current existing approaches while being generic and fast. The algorithms are integrated in the open-source Cytomine software and we provide parameter configuration guidelines so that they can be easily exploited by end-users. Finally, datasets are readily available through a Cytomine server to foster future research

Songbird organotypic culture as an in vitro model for interrogating sparse sequencing networks

Sparse sequences of neuronal activity are fundamental features of neural circuit computation; however, the underlying homeostatic mechanisms remain poorly understood. To approach these questions, we have developed a method for cellular-resolution imaging in organotypic cultures of the adult zebra finch brain, including portions of the intact song circuit. These in vitro networks can survive for weeks, and display mature neuron morphologies. Neurons within the organotypic slices exhibit a diversity of spontaneous and pharmacologically induced activity that can be easily monitored using the genetically encoded calcium indicator GCaMP6. In this study, we primarily focus on the classic song sequence generator HVC and the surrounding areas. We describe proof of concept experiments including physiological, optical, and pharmacological manipulation of these exposed networks. This method may allow the cellular rules underlying sparse, stereotyped neural sequencing to be examined with new degrees of experimental control

Monitoring of Single-Cell Responses in the Optic Tectum of Adult Zebrafish with Dextran-Coupled Calcium Dyes Delivered via Local Electroporation

Kassing V, Engelmann J, Kurtz R. Monitoring of Single-Cell Responses in the Optic Tectum of Adult Zebrafish with Dextran-Coupled Calcium Dyes Delivered via Local Electroporation. PLoS ONE. 2013;8(5): e62846.The zebrafish (Danio rerio) has become one of the major animal models for in vivo examination of sensory and neuronal computation. Similar to Xenopus tadpoles neural activity in the optic tectum, the major region controlling visually guided behavior, can be examined in zebrafish larvae by optical imaging. Prerequisites of these approaches are usually the transparency of larvae up to a certain age and the use of two-photon microscopy. This principle of fluorescence excitation was necessary to suppress crosstalk between signals from individual neurons, which is a critical issue when using membrane-permeant dyes. This makes the equipment to study neuronal processing costly and limits the approach to the study of larvae. Thus there is lack of knowledge about the properties of neurons in the optic tectum of adult animals. We established a procedure to circumvent these problems, enabling in vivo calcium imaging in the optic tectum of adult zebrafish. Following local application of dextran-coupled dyes single-neuron activity of adult zebrafish can be monitored with conventional widefield microscopy, because dye labeling remains restricted to tens of neurons or less. Among the neurons characterized with our technique we found neurons that were selective for a certain pattern orientation as well as neurons that responded in a direction-selective way to visual motion. These findings are consistent with previous studies and indicate that the functional integrity of neuronal circuits in the optic tectum of adult zebrafish is preserved with our staining technique. Overall, our protocol for in vivo calcium imaging provides a useful approach to monitor visual responses of individual neurons in the optic tectum of adult zebrafish even when only widefield microscopy is available. This approach will help to obtain valuable insight into the principles of visual computation in adult vertebrates and thus complement previous work on developing visual circuits