Semantics and planning based workflow composition and execution for video processing

Traditional workflow systems have several drawbacks, e.g. in their inabilities to rapidly react to changes, to construct workflow automatically (or with user involvement) and to improve performance autonomously (or with user involvement) in an incremental manner according to specified goals. Overcoming these limitations would be highly beneficial for complex domains where such adversities are exhibited. Video processing is one such domain that increasingly requires attention as larger amounts of images and videos are becoming available to persons who are not technically adept in modelling the processes that are involved in constructing complex video processing workflows. Conventional video and image processing systems, on the other hand, are developed by programmers possessing image processing expertise. These systems are tailored to produce highly specialised hand-crafted solutions for very specific tasks, making them rigid and non-modular. The knowledge-based vision community have attempted to produce more modular solutions by incorporating ontologies. However, they have not been maximally utilised to encompass aspects such as application context descriptions (e.g. lighting and clearness effects) and qualitative measures. This thesis aims to tackle some of the research gaps yet to be addressed by the workflow and knowledge-based image processing communities by proposing a novel workflow composition and execution approach within an integrated framework. This framework distinguishes three levels of abstraction via the design, workflow and processing layers. The core technologies that drive the workflow composition mechanism are ontologies and planning. Video processing problems provide a fitting domain for investigating the effectiveness of this integratedmethod as tackling such problems have not been fully explored by the workflow, planning and ontological communities despite their combined beneficial traits to confront this known hard problem. In addition, the pervasiveness of video data has proliferated the need for more automated assistance for image processing-naive users, but no adequate support has been provided as of yet. A video and image processing ontology that comprises three sub-ontologies was constructed to capture the goals, video descriptions and capabilities (video and image processing tools). The sub-ontologies are used for representation and inference. In particular, they are used in conjunction with an enhanced Hierarchical Task Network (HTN) domain independent planner to help with performance-based selection of solution steps based on preconditions, effects and postconditions. The planner, in turn, makes use of process models contained in a process library when deliberating on the steps and then consults the capability ontology to retrieve a suitable tool at each step. Two key features of the planner are the ability to support workflow execution (interleaves planning with execution) and can perform in automatic or semi-automatic (interactive) mode. The first feature is highly desirable for video processing problems because execution of image processing steps yield visual results that are intuitive and verifiable by the human user, as automatic validation is non trivial. In the semiautomaticmode, the planner is interactive and prompts the user tomake a tool selection when there is more than one tool available to perform a task. The user makes the tool selection based on the recommended descriptions provided by the workflow system. Once planning is complete, the result of applying the tool of their choice is presented to the user textually and visually for verification. This plays a pivotal role in providing the user with control and the ability to make informed decisions. Hence, the planner extends the capabilities of typical planners by guiding the user to construct more optimal solutions. Video processing problems can also be solved in more modular, reusable and adaptable ways as compared to conventional image processing systems. The integrated approach was evaluated on a test set consisting of videos originating from open sea environment of varying quality. Experiments to evaluate the efficiency, adaptability to user’s changing needs and user learnability of this approach were conducted on users who did not possess image processing expertise. The findings indicate that using this integrated workflow composition and execution method: 1) provides a speed up of over 90% in execution time for video classification tasks using full automatic processing compared to manual methods without loss of accuracy; 2) is more flexible and adaptable in response to changes in user requests (be it in the task, constraints to the task or descriptions of the video) than modifying existing image processing programs when the domain descriptions are altered; 3) assists the user in selecting optimal solutions by providing recommended descriptions

SWAV: Semantics-Based Workflows for Automatic Video Analysis

Workflows for Automatic Video Analysis. SWAV utilises ontologies and planning as core technologies to gear the composition and execution of video processing workflows. It is tailored for users without image processing expertise who have specific goals (tasks) and restrictions on these goals but not the ability to choose appropriate video processing software to solve their goals. An evaluation on a set of ecological videos has indicated that SWAV: 1) is more time-efficient at solving video classification tasks than manual processing; 2) is more adaptable in response to changes in user requests (task restrictions and video descriptions) than modifying existing image processing programs; and 3) assists the user in selecting optimal solutions by providing recommended descriptions

Semantics and Planning Based Workflow Composition for Video Processing

This work proposes a novel workflow composition approach that hinges upon ontologies and planning as its core technologies within an integrated framework. Video processing problems provide a fitting domain for investigating the effectiveness of this integrated method as tackling such problems have not been fully explored by the workflow, planning and ontological communities despite their combined beneficial traits to confront this known hard problem. In addition, the pervasiveness of video data has proliferated the need for more automated assistance for image processing-naive users, but no adequate support has been provided as of yet. The integrated approach was evaluated on a video set originating from open sea environment of varying quality. Experiments to evaluate the efficiency, adaptability to user’s changing needs and user learnability of this approach were conducted on users who did not possess image processing expertise. The findings indicate that using this integrated workflow composition and execution method: 1) provides a speed up of over 90 % in execution time for video classification tasks using full automatic processing compared to manual methods without loss of accuracy; 2) is more flexible and adaptable in response to changes in user requests than modifying existing image processing programs when the domain descriptions are altered; 3) assists the user in selecting optimal solutions by providing recommended descriptions

Burnout, anxiety and depression in healthcare workers during the early COVID-19 period in Singapore

acceptedVersionPeer reviewe

A Modular Approach for Automating Video Analysis

International audienceAutomating the steps involved in video processing has yet to be tackled with much success by vision developers and knowledge engineers. This is due to the difficulty in formulating vision problems and their solutions in a generalised manner. In this collaborated work, we introduce a modular approach that utilises ontologies to capture the goals, domain description and capabilities for performing video analysis. This modularisation is tested on real-world videos from an ecological source and proves useful in conceptualising and generalising video processing tasks. On a more significant note, this could be used in a framework for automatic video analysis in emerging infrastructures such as the Grid

Translating a Typical Business Process Modelling Language to a Web Services Ontology through Lightweight Mapping

Abstract: Bridging the gap between Enterprise Modelling methods and Semantic Web services is an important yet challenging task. For organisations with business goals, the automation of business processes as Web services is increasingly important, especially with many business transactions taking place within the Web today. Taking one approach to address this problem, a lightweight mapping between Fundamental Business Process Modelling Language (FBPML) and the Web Services Ontology (OWL-S) is outlined. The framework entails a data model translation and a process model translation via the use of ontologies and mapping principles. Several working examples of the process model translations are presented together with an implementation of an automated translator. FBPML constructs and process models that could not be translated to OWL-S equivalents highlight the differences between the languages of the two domains. It also implies that evolving Semantic Web technologies, in particular OWL-S, are not adequate for all service modelling needs and could thus benefit from the more traditional and mature BPM methods. On a more interesting note, this is effectively the first step towards enabling a semantic-based business workflow system

Enterprise Modelling methods and Semantic Web services,

This paper presents an ontology-based conceptual mapping framework that translates a formal and visually rich business process modeling (BPM) language, Fundamenta