Classification of bird species from video using appearance and motion features

The monitoring of bird populations can provide important information on the state of sensitive ecosystems; however, the manual collection of reliable population data is labour-intensive, time-consuming, and potentially error prone. Automated monitoring using computer vision is therefore an attractive proposition, which could facilitate the collection of detailed data on a much larger scale than is currently possible. A number of existing algorithms are able to classify bird species from individual high quality detailed images often using manual inputs (such as a priori parts labelling). However, deployment in the field necessitates fully automated in-flight classification, which remains an open challenge due to poor image quality, high and rapid variation in pose, and similar appearance of some species. We address this as a fine-grained classification problem, and have collected a video dataset of thirteen bird classes (ten species and another with three colour variants) for training and evaluation. We present our proposed algorithm, which selects effective features from a large pool of appearance and motion features. We compare our method to others which use appearance features only, including image classification using state-of-the-art Deep Convolutional Neural Networks (CNNs). Using our algorithm we achieved a 90% correct classification rate, and we also show that using effectively selected motion and appearance features together can produce results which outperform state-of-the-art single image classifiers. We also show that the most significant motion features improve correct classification rates by 7% compared to using appearance features alone

Automatic classification of flying bird species using computer vision techniques [forthcoming]

Bird populations are identified as important biodiversity indicators, so collecting reliable population data is important to ecologists and scientists. However, existing manual monitoring methods are labour-intensive, time-consuming, and potentially error prone. The aim of our work is to develop a reliable automated system, capable of classifying the species of individual birds, during flight, using video data. This is challenging, but appropriate for use in the field, since there is often a requirement to identify in flight, rather than while stationary. We present our work, which uses a new and rich set of appearance features for classification from video. We also introduce motion features including curvature and wing beat frequency. Combined with Normal Bayes classifier and a Support Vector Machine classifier, we present experimental evaluations of our appearance and motion features across a data set comprising 7 species. Using our appearance feature set alone we achieved a classification rate of 92% and 89% (using Normal Bayes and SVM classifiers respectively) which significantly outperforms a recent comparable state-of-the-art system. Using motion features alone we achieved a lower-classification rate, but motivate our on-going work which we seeks to combine these appearance and motion feature to achieve even more robust classification

Automatic classification of flying bird species using computer vision techniques

Bird species are recognised as important biodiversity indicators: they are responsive to changes in sensitive ecosystems, whilst populations-level changes in behaviour are both visible and quantifiable. They are monitored by ecologists to determine factors causing population fluctuation and to help conserve and manage threatened and endangered species. Every five years, the health of bird population found in the UK are reviewed based on data collected from various surveys. Currently, techniques used in surveying species include manual counting, Bioacoustics and computer vision. The latter is still under development by researchers. Hitherto, no computer vision technique has fully been deployed in the field for counting species as these techniques use high-quality and detailed images of stationary birds, which make them impractical for deployment in the field, as most species in the field are in-flight and sometimes distant from the cameras field of view. Techniques such as manual and bioacoustics are the most frequently used but they can also become impractical, particularly when counting densely populated migratory species. Manual techniques are labour intensive whilst bioacoustics may be unusable when deployed for species that emit little or no sound. There is the need for automated systems for identifying species using computer vision and machine learning techniques, specifically for surveying densely populated migratory species. However, currently, most systems are not fully automated and use only appearance-based features for identification of species. Moreover, in the field, appearance-based features like colour may fade at a distance whilst motion-based features will remain discernible. Thus to achieve full automation, existing systems will have to combine both appearance and motion features. The aim of this thesis is to contribute to this problem by developing computer vision techniques which combine appearance and motion features to robustly classify species, whilst in flight. It is believed that once this is achieved, with additional development, it will be able to support the surveying of species and their behaviour studies. The first focus of this research was to refine appearance features previously used in other related works for use in automatic classification of species in flight. The bird appearances were described using a group of seven proposed appearance features, which have not previously been used for bird species classification. The proposed features improved the classification rate when compared to state-of-the-art systems that were based on appearance features alone (colour features). The second step was to extract motion features from videos of birds in flight, which were used for automatic classification. The motion of birds was described using a group of six features, which have not previously been used for bird species classification. The proposed motion features, when combined with the appearance features improved classification rates compared with only appearance or motion features. The classification rates were further improved using feature selection techniques. There was an increase of between 2-6% of correct classification rates across all classifiers, which may be attributable directly to the use of motion features. The only motion features selected are the wing beat frequency and vicinity features irrespective of the method used. This shows how important these groups of features were to species classification. Further analysis also revealed specific improvements in identifying species with similar visual appearance and that using the optimal motion features improve classification accuracy significantly. We attempt a further improvement in classification accuracy, using majority voting. This was used to aggregate classification results across a set of video sub-sequences, which improved classification rates considerably. The results using the combined features with majority voting outperform those without majority voting by 3% and 6% on the seven species and thirteen classes dataset respectively. Finally, a video dataset against which future work can be benchmarked has been collated. This data set enables the evaluation of work against a set of 13 species, enabling effective evaluation of automated species identification to date and a benchmark for further work in this area of research. The key contribution of this research is that a species classification system was developed, which combines motion and appearance features and evaluated it against existing appearance-only-based methods. This is not only the first work to combine features in this way but also the first to apply a voting technique to improve classification performance across an entire video sequence

A computer vision approach to classification of birds in flight from video sequences

Bird populations are an important bio-indicator; so collecting reliable data is useful for ecologists helping conserve and manage fragile ecosystems. However, existing manual monitoring methods are labour-intensive, time-consuming, and error-prone. The aim of our work is to develop a reliable system, capable of automatically classifying individual bird species in flight from videos. This is challenging, but appropriate for use in the field, since there is often a requirement to identify in flight, rather than when stationary. We present our work in progress, which uses combined appearance and motion features to classify and present experimental results across seven species using Normal Bayes classifier with majority voting and achieving a classification rate of 86%

Automatic Lesser Kestrel’s Gender Identification using Video Processing

Traditionally, animal surveillance is a common task for biologists. However, this task is often accompanied by the inspection of huge amounts of video. In this sense, this paper proposes an automatic video processing algorithm to identify the gender of a kestrel species. It is based on optical flow and texture analysis. This algorithm makes it possible to identify the important information and therefore, minimizing the analysis time for biologists. Finally, to validate this algorithm, it has been tested against a set of videos, getting good classification results.Junta de Andalucía P10-TIC-570

One-to-many face recognition with bilinear CNNs

The recent explosive growth in convolutional neural network (CNN) research has produced a variety of new architectures for deep learning. One intriguing new architecture is the bilinear CNN (B-CNN), which has shown dramatic performance gains on certain fine-grained recognition problems [15]. We apply this new CNN to the challenging new face recognition benchmark, the IARPA Janus Benchmark A (IJB-A) [12]. It features faces from a large number of identities in challenging real-world conditions. Because the face images were not identified automatically using a computerized face detection system, it does not have the bias inherent in such a database. We demonstrate the performance of the B-CNN model beginning from an AlexNet-style network pre-trained on ImageNet. We then show results for fine-tuning using a moderate-sized and public external database, FaceScrub [17]. We also present results with additional fine-tuning on the limited training data provided by the protocol. In each case, the fine-tuned bilinear model shows substantial improvements over the standard CNN. Finally, we demonstrate how a standard CNN pre-trained on a large face database, the recently released VGG-Face model [20], can be converted into a B-CNN without any additional feature training. This B-CNN improves upon the CNN performance on the IJB-A benchmark, achieving 89.5% rank-1 recall.Comment: Published version at WACV 201

Iterative Object and Part Transfer for Fine-Grained Recognition

The aim of fine-grained recognition is to identify sub-ordinate categories in images like different species of birds. Existing works have confirmed that, in order to capture the subtle differences across the categories, automatic localization of objects and parts is critical. Most approaches for object and part localization relied on the bottom-up pipeline, where thousands of region proposals are generated and then filtered by pre-trained object/part models. This is computationally expensive and not scalable once the number of objects/parts becomes large. In this paper, we propose a nonparametric data-driven method for object and part localization. Given an unlabeled test image, our approach transfers annotations from a few similar images retrieved in the training set. In particular, we propose an iterative transfer strategy that gradually refine the predicted bounding boxes. Based on the located objects and parts, deep convolutional features are extracted for recognition. We evaluate our approach on the widely-used CUB200-2011 dataset and a new and large dataset called Birdsnap. On both datasets, we achieve better results than many state-of-the-art approaches, including a few using oracle (manually annotated) bounding boxes in the test images.Comment: To appear in ICME 2017 as an oral pape

Improving the efficiency and accuracy of nocturnal bird Surveys through equipment selection and partial automation

This thesis was submitted for the degree of Engineering Doctorate and awarded by Brunel University.Birds are a key environmental asset and this is recognised through comprehensive legislation and policy ensuring their protection and conservation. Many species are active at night and surveys are required to understand the implications of proposed developments such as towers and reduce possible conflicts with these structures. Night vision devices are commonly used in nocturnal surveys, either to scope an area for bird numbers and activity, or in remotely sensing an area to determine potential risk. This thesis explores some practical and theoretical approaches that can improve the accuracy, confidence and efficiency of nocturnal bird surveillance. As image intensifiers and thermal imagers have operational differences, each device has associated strengths and limitations. Empirical work established that image intensifiers are best used for species identification of birds against the ground or vegetation. Thermal imagers perform best in detection tasks and monitoring bird airspace usage. The typically used approach of viewing bird survey video from remote sensing in its entirety is a slow, inaccurate and inefficient approach. Accuracy can be significantly improved by viewing the survey video at half the playback speed. Motion detection efficiency and accuracy can be greatly improved through the use of adaptive background subtraction and cumulative image differencing. An experienced ornithologist uses bird flight style and wing oscillations to identify bird species. Changes in wing oscillations can be represented in a single inter-frame similarity matrix through area-based differencing. Bird species classification can then be automated using singular value decomposition to reduce the matrices to one-dimensional vectors for training a feed-forward neural network