11 research outputs found
Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data
This book gives a start-to-finish overview of the whole Fish4Knowledge project, in 18 short chapters, each describing one aspect of the project. The Fish4Knowledge project explored the possibilities of big video data, in this case from undersea video. Recording and analyzing 90 thousand hours of video from ten camera locations, the project gives a 3 year view of fish abundance in several tropical coral reefs off the coast of Taiwan. The research system built a remote recording network, over 100 Tb of storage, supercomputer processing, video target detection and
Feedback-control & queueing theory-based resource management for streaming applications
Recent advances in sensor technologies and instrumentation have led to an extraordinary growth of data sources and streaming applications. A wide variety of devices, from smart phones to dedicated sensors, have the capability of collecting and streaming large amounts of data at unprecedented rates. A number of distinct streaming data models have been proposed. Typical applications for this include smart cites & built environments for instance, where sensor-based infrastructures continue to increase in scale and variety. Understanding how such streaming content can be processed within some time threshold remains a non-trivial and important research topic. We investigate how a cloud-based computational infrastructure can autonomically respond to such streaming content, offering Quality of Service guarantees. We propose an autonomic controller (based on feedback control and queueing theory) to elastically provision virtual machines to meet performance targets associated with a particular data stream. Evaluation is carried out using a federated Cloud-based infrastructure (implemented using CometCloud) – where the allocation of new resources can be based on: (i) differences between sites, i.e. types of resources supported (e.g. GPU vs. CPU only), (ii) cost of execution; (iii) failure rate and likely resilience, etc. In particular, we demonstrate how Little’s Law –a widely used result in queuing theory– can be adapted to support dynamic control in the context of such resource provisioning
Developing deep learning methods for aquaculture applications
Alzayat Saleh developed a computer vision framework that can aid aquaculture experts in analyzing fish habitats. In particular, he developed a labelling efficient method of training a CNN-based fish-detector and also developed a model that estimates the fish weight directly from its image
Model-driven development of data intensive applications over cloud resources
The proliferation of sensors over the last years has generated large amounts of raw data, forming data streams that need to be processed. In many cases, cloud resources are used for such processing, exploiting their flexibility, but these sensor streaming applications often need to support operational and control actions that have real-time and low-latency requirements that go beyond the cost effective and flexible solutions supported by existing cloud frameworks, such as Apache Kafka, Apache Spark Streaming, or Map-Reduce Streams. In this paper, we describe a model-driven and stepwise refinement methodological approach for streaming applications executed over clouds. The central role is assigned to a set of Petri Net models for specifying functional and non-functional requirements. They support model reuse, and a way to combine formal analysis, simulation, and approximate computation of minimal and maximal boundaries of non-functional requirements when the problem is either mathematically or computationally intractable. We show how our proposal can assist developers in their design and implementation decisions from a performance perspective. Our methodology allows to conduct performance analysis: The methodology is intended for all the engineering process stages, and we can (i) analyse how it can be mapped onto cloud resources, and (ii) obtain key performance indicators, including throughput or economic cost, so that developers are assisted in their development tasks and in their decision taking. In order to illustrate our approach, we make use of the pipelined wavefront array
Balance-guaranteed optimized tree with reject option for live fish recognition
This thesis investigates the computer vision application of live fish recognition, which
is needed in application scenarios where manual annotation is too expensive, when
there are too many underwater videos. This system can assist ecological surveillance
research, e.g. computing fish population statistics in the open sea. Some pre-processing
procedures are employed to improve the recognition accuracy, and then 69 types of
features are extracted. These features are a combination of colour, shape and texture
properties in different parts of the fish such as tail/head/top/bottom, as well as
the whole fish. Then, we present a novel Balance-Guaranteed Optimized Tree with
Reject option (BGOTR) for live fish recognition. It improves the normal hierarchical
method by arranging more accurate classifications at a higher level and keeping the
hierarchical tree balanced. BGOTR is automatically constructed based on inter-class
similarities. We apply a Gaussian Mixture Model (GMM) and Bayes rule as a reject
option after the hierarchical classification to evaluate the posterior probability of being
a certain species to filter less confident decisions. This novel classification-rejection
method cleans up decisions and rejects unknown classes. After constructing the tree
architecture, a novel trajectory voting method is used to eliminate accumulated errors
during hierarchical classification and, therefore, achieves better performance. The proposed
BGOTR-based hierarchical classification method is applied to recognize the 15
major species of 24150 manually labelled fish images and to detect new species in
an unrestricted natural environment recorded by underwater cameras in south Taiwan
sea. It achieves significant improvements compared to the state-of-the-art techniques.
Furthermore, the sequence of feature selection and constructing a multi-class SVM
is investigated. We propose that an Individual Feature Selection (IFS) procedure can
be directly exploited to the binary One-versus-One SVMs before assembling the full
multiclass SVM. The IFS method selects different subsets of features for each Oneversus-
One SVM inside the multiclass classifier so that each vote is optimized to discriminate
the two specific classes. The proposed IFS method is tested on four different
datasets comparing the performance and time cost. Experimental results demonstrate
significant improvements compared to the normal Multiclass Feature Selection (MFS)
method on all datasets
Internet of Underwater Things and Big Marine Data Analytics -- A Comprehensive Survey
The Internet of Underwater Things (IoUT) is an emerging communication
ecosystem developed for connecting underwater objects in maritime and
underwater environments. The IoUT technology is intricately linked with
intelligent boats and ships, smart shores and oceans, automatic marine
transportations, positioning and navigation, underwater exploration, disaster
prediction and prevention, as well as with intelligent monitoring and security.
The IoUT has an influence at various scales ranging from a small scientific
observatory, to a midsized harbor, and to covering global oceanic trade. The
network architecture of IoUT is intrinsically heterogeneous and should be
sufficiently resilient to operate in harsh environments. This creates major
challenges in terms of underwater communications, whilst relying on limited
energy resources. Additionally, the volume, velocity, and variety of data
produced by sensors, hydrophones, and cameras in IoUT is enormous, giving rise
to the concept of Big Marine Data (BMD), which has its own processing
challenges. Hence, conventional data processing techniques will falter, and
bespoke Machine Learning (ML) solutions have to be employed for automatically
learning the specific BMD behavior and features facilitating knowledge
extraction and decision support. The motivation of this paper is to
comprehensively survey the IoUT, BMD, and their synthesis. It also aims for
exploring the nexus of BMD with ML. We set out from underwater data collection
and then discuss the family of IoUT data communication techniques with an
emphasis on the state-of-the-art research challenges. We then review the suite
of ML solutions suitable for BMD handling and analytics. We treat the subject
deductively from an educational perspective, critically appraising the material
surveyed.Comment: 54 pages, 11 figures, 19 tables, IEEE Communications Surveys &
Tutorials, peer-reviewed academic journa
Novel deep learning architectures for marine and aquaculture applications
Alzayat Saleh's research was in the area of artificial intelligence and machine learning to autonomously recognise fish and their morphological features from digital images. Here he created new deep learning architectures that solved various computer vision problems specific to the marine and aquaculture context. He found that these techniques can facilitate aquaculture management and environmental protection. Fisheries and conservation agencies can use his results for better monitoring strategies and sustainable fishing practices
Semantics and planning based workflow composition and execution for video processing
Traditional workflow systems have several drawbacks, e.g. in their inabilities to rapidly
react to changes, to construct workflow automatically (or with user involvement) and
to improve performance autonomously (or with user involvement) in an incremental
manner according to specified goals. Overcoming these limitations would be highly
beneficial for complex domains where such adversities are exhibited. Video processing
is one such domain that increasingly requires attention as larger amounts of images and
videos are becoming available to persons who are not technically adept in modelling
the processes that are involved in constructing complex video processing workflows.
Conventional video and image processing systems, on the other hand, are developed
by programmers possessing image processing expertise. These systems are tailored
to produce highly specialised hand-crafted solutions for very specific tasks, making
them rigid and non-modular. The knowledge-based vision community have attempted
to produce more modular solutions by incorporating ontologies. However,
they have not been maximally utilised to encompass aspects such as application context
descriptions (e.g. lighting and clearness effects) and qualitative measures.
This thesis aims to tackle some of the research gaps yet to be addressed by the
workflow and knowledge-based image processing communities by proposing a novel
workflow composition and execution approach within an integrated framework. This
framework distinguishes three levels of abstraction via the design, workflow and processing
layers. The core technologies that drive the workflow composition mechanism
are ontologies and planning. Video processing problems provide a fitting domain for
investigating the effectiveness of this integratedmethod as tackling such problems have
not been fully explored by the workflow, planning and ontological communities despite
their combined beneficial traits to confront this known hard problem. In addition, the
pervasiveness of video data has proliferated the need for more automated assistance
for image processing-naive users, but no adequate support has been provided as of yet.
A video and image processing ontology that comprises three sub-ontologies was
constructed to capture the goals, video descriptions and capabilities (video and image
processing tools). The sub-ontologies are used for representation and inference. In
particular, they are used in conjunction with an enhanced Hierarchical Task Network
(HTN) domain independent planner to help with performance-based selection of solution
steps based on preconditions, effects and postconditions. The planner, in turn,
makes use of process models contained in a process library when deliberating on the
steps and then consults the capability ontology to retrieve a suitable tool at each step. Two key features of the planner are the ability to support workflow execution (interleaves
planning with execution) and can perform in automatic or semi-automatic
(interactive) mode. The first feature is highly desirable for video processing problems
because execution of image processing steps yield visual results that are intuitive
and verifiable by the human user, as automatic validation is non trivial. In the semiautomaticmode,
the planner is interactive and prompts the user tomake a tool selection
when there is more than one tool available to perform a task. The user makes the tool
selection based on the recommended descriptions provided by the workflow system.
Once planning is complete, the result of applying the tool of their choice is presented
to the user textually and visually for verification. This plays a pivotal role in providing
the user with control and the ability to make informed decisions. Hence, the planner
extends the capabilities of typical planners by guiding the user to construct more
optimal solutions. Video processing problems can also be solved in more modular,
reusable and adaptable ways as compared to conventional image processing systems.
The integrated approach was evaluated on a test set consisting of videos originating
from open sea environment of varying quality. Experiments to evaluate the efficiency,
adaptability to user’s changing needs and user learnability of this approach were conducted
on users who did not possess image processing expertise. The findings indicate
that using this integrated workflow composition and execution method: 1) provides a
speed up of over 90% in execution time for video classification tasks using full automatic
processing compared to manual methods without loss of accuracy; 2) is more
flexible and adaptable in response to changes in user requests (be it in the task, constraints
to the task or descriptions of the video) than modifying existing image processing
programs when the domain descriptions are altered; 3) assists the user in selecting
optimal solutions by providing recommended descriptions