Object Referring in Videos with Language and Human Gaze

We investigate the problem of object referring (OR) i.e. to localize a target object in a visual scene coming with a language description. Humans perceive the world more as continued video snippets than as static images, and describe objects not only by their appearance, but also by their spatio-temporal context and motion features. Humans also gaze at the object when they issue a referring expression. Existing works for OR mostly focus on static images only, which fall short in providing many such cues. This paper addresses OR in videos with language and human gaze. To that end, we present a new video dataset for OR, with 30, 000 objects over 5, 000 stereo video sequences annotated for their descriptions and gaze. We further propose a novel network model for OR in videos, by integrating appearance, motion, gaze, and spatio-temporal context into one network. Experimental results show that our method effectively utilizes motion cues, human gaze, and spatio-temporal context. Our method outperforms previousOR methods. For dataset and code, please refer https://people.ee.ethz.ch/~arunv/ORGaze.html.Comment: Accepted to CVPR 2018, 10 pages, 6 figure

Flexible human-robot cooperation models for assisted shop-floor tasks

The Industry 4.0 paradigm emphasizes the crucial benefits that collaborative robots, i.e., robots able to work alongside and together with humans, could bring to the whole production process. In this context, an enabling technology yet unreached is the design of flexible robots able to deal at all levels with humans' intrinsic variability, which is not only a necessary element for a comfortable working experience for the person but also a precious capability for efficiently dealing with unexpected events. In this paper, a sensing, representation, planning and control architecture for flexible human-robot cooperation, referred to as FlexHRC, is proposed. FlexHRC relies on wearable sensors for human action recognition, AND/OR graphs for the representation of and reasoning upon cooperation models, and a Task Priority framework to decouple action planning from robot motion planning and control.Comment: Submitted to Mechatronics (Elsevier

The Oral and Skin Microbiomes of Captive Komodo Dragons Are Significantly Shared with Their Habitat.

Examining the way in which animals, including those in captivity, interact with their environment is extremely important for studying ecological processes and developing sophisticated animal husbandry. Here we use the Komodo dragon (Varanus komodoensis) to quantify the degree of sharing of salivary, skin, and fecal microbiota with their environment in captivity. Both species richness and microbial community composition of most surfaces in the Komodo dragon's environment are similar to the Komodo dragon's salivary and skin microbiota but less similar to the stool-associated microbiota. We additionally compared host-environment microbiome sharing between captive Komodo dragons and their enclosures, humans and pets and their homes, and wild amphibians and their environments. We observed similar host-environment microbiome sharing patterns among humans and their pets and Komodo dragons, with high levels of human/pet- and Komodo dragon-associated microbes on home and enclosure surfaces. In contrast, only small amounts of amphibian-associated microbes were detected in the animals' environments. We suggest that the degree of sharing between the Komodo dragon microbiota and its enclosure surfaces has important implications for animal health. These animals evolved in the context of constant exposure to a complex environmental microbiota, which likely shaped their physiological development; in captivity, these animals will not receive significant exposure to microbes not already in their enclosure, with unknown consequences for their health. IMPORTANCE Animals, including humans, have evolved in the context of exposure to a variety of microbial organisms present in the environment. Only recently have humans, and some animals, begun to spend a significant amount of time in enclosed artificial environments, rather than in the more natural spaces in which most of evolution took place. The consequences of this radical change in lifestyle likely extend to the microbes residing in and on our bodies and may have important implications for health and disease. A full characterization of host-microbe sharing in both closed and open environments will provide crucial information that may enable the improvement of health in humans and in captive animals, both of which experience a greater incidence of disease (including chronic illness) than counterparts living under more ecologically natural conditions

Can Artificail Entities Assert?

There is an existing debate regarding the view that technological instruments, devices, or machines can assert ‎or testify. A standard view in epistemology is that only humans can testify. However, the notion of quasi-‎testimony acknowledges that technological devices can assert or testify under some conditions, without ‎denying that humans and machines are not the same. Indeed, there are four relevant differences between ‎humans and instruments. First, unlike humans, machine assertion is not imaginative or playful. Second, ‎machine assertion is prescripted and context restricted. As such, computers currently cannot easily switch ‎contexts or make meaningful relevant assertions in contexts for which they were not programmed. Third, ‎while both humans and computers make errors, they do so in different ways. Computers are very sensitive to ‎small errors in input, which may cause them to make big errors in output. Moreover, automatic error control ‎is based on finding irregularities in data without trying to establish whether they make sense. Fourth, ‎testimony is produced by a human with moral worth, while quasi-testimony is not. Ultimately, the notion of ‎quasi-testimony can serve as a bridge between different philosophical fields that deal with instruments and ‎testimony as sources of knowledge, allowing them to converse and agree on a shared description of reality, ‎while maintaining their distinct conceptions and ontological commitments about knowledge, humans, and ‎nonhumans.

QuAC : Question Answering in Context

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text. QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation. We also report results for a number of reference models, including a recently state-of-the-art reading comprehension architecture extended to model dialog context. Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data. Dataset, baseline, and leaderboard available at http://quac.ai.Comment: EMNLP Camera Read

Pro-social motive promotes early understanding of false belief

Ever since Premack & Woodruff's classic article^1^, which introduced the term "theory of mind", researchers have claimed that strategic deception is the most natural behavioural consequence of understanding false belief. Here we challenge that claim, and provide evidence for the first time that the earliest manifestation of false belief understanding in human development is found in young children's emerging pro-social behaviours. In a modified false belief task, children were asked either to choose one protagonist they should help to find the object (the pro-social context), or to choose one they need to deceive so that none of the protagonists can find the object (the competitive context). The results show that the pro-social motive, but not the competitive motive, boosts early false belief understanding. This is most clearly contrasted with findings that apes, our closest living relatives, are capable of intentionally manipulating others by concealing information only under competitive motives, not under cooperative alternatives. Thus, the current findings are the strongest to date that sophisticated understanding of others' belief in humans has its unique origin, separate from the primate origin at some point in recent evolution, when cooperative and communicative motives played an essential role for their survival