Human and Animal Behavior Understanding

Human and animal behavior understanding is an important yet challenging task in computer vision. It has a variety of real-world applications including human computer interaction (HCI), video surveillance, pharmacology, genetics, etc. We first present an evaluation of spatiotemporal interest point features (STIPs) for depth-based human action recognition, and then propose a framework call TriViews for 3D human action recognition with RGB-D data. Finally, we investigate a new approach for animal behavior recognition based on tracking, video content extraction and data fusion.;STIPs features are widely used with good performance for action recognition using the visible light videos. Recently, with the advance of depth imaging technology, a new modality has appeared for human action recognition. It is important to assess the performance and usefulness of the STIPs features for action analysis on the new modality of 3D depth map. Three detectors and six descriptors are combined to form various STIPs features in this thesis. Experiments are conducted on four challenging depth datasets.;We present an effective framework called TriViews to utilize 3D information for human action recognition. It projects the 3D depth maps into three views, i.e., front, side, and top views. Under this framework, five features are extracted from each view, separately. Then the three views are combined to derive a complete description of the 3D data. The five features characterize action patterns from different aspects, among which the top three best features are selected and fused based on a probabilistic fusion approach (PFA). We evaluate the proposed framework on three challenging depth action datasets. The experimental results show that the proposed TriViews framework achieves the most accurate results for depth-based action recognition, better than the state-of-the-art methods on all three databases.;Compared to human actions, animal behaviors exhibit some different characteristics. For example, animal body is much less expressive than human body, so some visual features and frameworks which are widely used for human action representation, cannot work well for animals. We investigate two features for mice behavior recognition, i.e., sparse and dense trajectory features. Sparse trajectory feature relies on tracking heavily. If tracking fails, the performance of sparse trajectory feature may deteriorate. In contrast, dense trajectory features are much more robust without relying on the tracking, thus the integration of these two features could be of practical significance. A fusion approach is proposed for mice behavior recognition. Experimental results on two public databases show that the integration of sparse and dense trajectory features can improve the recognition performance

Virtual infection modeling for Aspergillus fumigatus in human and murine alveoli

Der Der filamentöse pathogene Pilz Aspergillus fumigatus kann schwere Infektionen wie die invasive pulmonale Aspergillose in immungeschwächten Patienten verursachen. Verbunden mit einer hohen Mortalität und einer steigenden Inzidenz der letzten Jahrzehnte bezeugt dies die Notwendigkeit zur Erforschung seines opportunistischen Verhaltens sowie zur Entwicklung effizienter Behandlungsstrategien, um Menschenleben zu retten. Da die Lunge, als primäres Ziel von A. fumigatus Infektionen, nur begrenzt experimentell in vivo studiert werden kann, verfolgt diese Arbeit den Ansatz agenten-basierter Simulation. Die kumulative Dissertation basiert auf 4 veröffentlichten Manuskripten. Untersucht wurden dabei die Vergleichbarkeit von natürlichen Infektionen im Menschen und künstlichen Infektionen im etablierten Mausmodell. Eine zweite Veröffentlichung untersucht den Einfluss von Kohn'schen Poren auf die Dynamik der Immunabwehr gegen Aspergillus fumigatus. Eine dritte Veröffentlichung untersucht die Anwendbarkeit von dynamischen Kugeloberflächenfunktionen - Spherical Harmonics - als Werkzeug der Klassifikation und Beschreibung von beweglichen Zellen. Die vierte Veröffentlichung präsentiert erstmals einen Aspergillose Chip auf Mikrofluidikchips. Dies erlaubt es, die Pathogen-Wirt-Beziehungen unter realistischen Bedingungen zu untersuchen sowie das Wachstum der Pilzhyphen zu quantifizieren

Rational Design of Pathogen-Mimicking Amphiphilic Materials as Nanoadjuvants

An opportunity exists today for cross-cutting research utilizing advances in materials science, immunology, microbial pathogenesis, and computational analysis to effectively design the next generation of adjuvants and vaccines. This study integrates these advances into a bottom-up approach for the molecular design of nanoadjuvants capable of mimicking the immune response induced by a natural infection but without the toxic side effects. Biodegradable amphiphilic polyanhydrides possess the unique ability to mimic pathogens and pathogen associated molecular patterns with respect to persisting within and activating immune cells, respectively. The molecular properties responsible for the pathogen-mimicking abilities of these materials have been identified. The value of using polyanhydride nanovaccines was demonstrated by the induction of long-lived protection against a lethal challenge of Yersinia pestis following a single administration ten months earlier. This approach has the tantalizing potential to catalyze the development of next generation vaccines against diseases caused by emerging and re-emerging pathogens

DESIGN OF A GAIT ACQUISITION AND ANALYSIS SYSTEM FOR ASSESSING THE RECOVERY OF MICE POST-SPINAL CORD INJURY

Current methods of determining spinal cord recovery in mice, post-directed injury, are qualitative measures. This is due to the small size and quickness of mice. This thesis presents a design for a gait acquisition and analysis system able to capture the footfalls of a mouse, extract position and timing data, and report quantitative gait metrics to the operator. These metrics can then be used to evaluate the recovery of the mouse. This work presents the design evolution of the system, from initial sensor design concepts through prototyping and testing to the final implementation. The system utilizes a machine vision camera, a well-designed walkway enclosure, and image processing techniques to capture and analyze paw strikes. Quantitative results gained from live animal experiments are presented, and it is shown how the measurements can be used to determine healthy, injured, and recovered gait

Ancient Urban Ecology Reconstructed from Archaeozoological Remains of Small Mammals in the Near East

Acknowledgments We especially thank the many archaeologists who collaborated closely with our project and invested pioneering efforts in intensive fine-scale retrieval of the archaeozoological samples that provided the basis for this study: Shai Bar, Amnon Ben-Tor, Amit Dagan, Yosef Garfinkel, Ayelet Gilboa, Zvi Greenhut, Amihai Mazar, Stefan Munger, Ronny Reich, Itzhaq Shai, Ilan Sharon, Joe Uziel, Sharon Zuckerman, and additional key excavation personnel who were instrumental in collection of the samples or in assisting the work including: Shimrit Bechar, Jacob Dunn, Norma Franklin, Egon Lass and Yiftah Shalev. Funding:The research was funded by a post-doctoral grant awarded to L.W. from the European Research Council under the European Community’s Seventh Framework Program (FP7/2007e2013)/ERC grant agreement number 229418. The laboratory work was also supported by funding by the Israel Science Foundation (Grant 52/10). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Peer reviewedPublisher PD

Towards solving computer vision problems: datasets, labels, algorithms, and applications

The solution to a supervised computer vision problem consists of an application, algorithm, input data, and a set of human generated labels. Solving these kinds of tasks involves collecting large quantities of data, collecting appropriate labels, and developing machine vision algorithms tailored to the application. Progress on these problems has often benefited from large scale datasets with high fidelity labels. Successful algorithms display a synergy between application goals and the size and quality of the dataset. This thesis presents work highlighting the importance of each component of a supervised vision task.First, the problem of automatically classifying groups of people into social categories is introduced. This problem is called Urban Tribe Classification. To tackle this problem, each individual and the entire group of individuals are modeled. Since this was a newly introduced computer vision problem, a dataset for this task was created. On this dataset, the combined representation of group and individuals outperforms using only the person representations. This model showed promising results for automatic subculture classification.Second, the problem of creating perceptual embeddings based on human similarity judgements is tackled. This work focuses on triplet similarity comparisons of the form ``Is object $i$ more similar to $j$ or $k$?'', which have been useful for computer vision and machine learning applications. Unfortunately, triplet similarity comparisons, like many human labeling efforts, can be prohibitively expensive. This work proposes two techniques for dealing with this obstacle. First, an alternative display for collecting triplets is designed. This display shows a probe image and a grid of query images, allowing the user to collect multiple triplets simultaneously. The display is shown to reduce the cost and time of triplet collection. In addition, higher quality embeddings are created with the improved triplet collection UI. A 10,000-food item dataset of human taste similarity was created using this UI. Second, ``SNaCK,'' a low-dimensional perceptual embedding algorithm that combines human expertise with automatic machine kernels, is introduced. Both parts are complementary: human insight can capture relationships that are not apparent from the object's visual similarity and the machine can help relieve the human from having to exhaustively specify many constraints. Finally, the precise localization of key frames of an action is explored. This work focuses on detecting the exact starting frame of a behavior, an important task for neuroscience research. To address this problem, a loss designed to penalize extra and missed action start detections over small misalignments. Recurrent neural networks (RNN) are trained to optimize this loss. The model is shown to reduce the number of false positives, an important criteria defined by the neuroscientist. The performance of the model is evaluated on a new dataset, the Mouse Reach Dataset, a large, annotated video dataset of mice performing a sequence of actions. The dataset was created for neuroscience research. On this dataset, the proposed model outperforms related approaches and baseline methods using an unstructured loss