Towards Collaborative Scientific Workflow Management System

The big data explosion phenomenon has impacted several domains, starting from research areas to divergent of business models in recent years. As this intensive amount of data opens up the possibilities of several interesting knowledge discoveries, over the past few years divergent of research domains have undergone the shift of trend towards analyzing those massive amount data. Scientific Workflow Management System (SWfMS) has gained much popularity in recent years in accelerating those data-intensive analyses, visualization, and discoveries of important information. Data-intensive tasks are often significantly time-consuming and complex in nature and hence SWfMSs are designed to efficiently support the specification, modification, execution, failure handling, and monitoring of the tasks in a scientific workflow. As far as the complexity, dimension, and volume of data are concerned, their effective analysis or management often become challenging for an individual and requires collaboration of multiple scientists instead. Hence, the notion of 'Collaborative SWfMS' was coined - which gained significant interest among researchers in recent years as none of the existing SWfMSs directly support real-time collaboration among scientists. In terms of collaborative SWfMSs, consistency management in the face of conflicting concurrent operations of the collaborators is a major challenge for its highly interconnected document structure among the computational modules - where any minor change in a part of the workflow can highly impact the other part of the collaborative workflow for the datalink relation among them. In addition to the consistency management, studies show several other challenges that need to be addressed towards a successful design of collaborative SWfMSs, such as sub-workflow composition and execution by different sub-groups, relationship between scientific workflows and collaboration models, sub-workflow monitoring, seamless integration and access control of the workflow components among collaborators and so on. In this thesis, we propose a locking scheme to facilitate consistency management in collaborative SWfMSs. The proposed method works by locking workflow components at a granular attribute level in addition to supporting locks on a targeted part of the collaborative workflow. We conducted several experiments to analyze the performance of the proposed method in comparison to related existing methods. Our studies show that the proposed method can reduce the average waiting time of a collaborator by up to 36% while increasing the average workflow update rate by up to 15% in comparison to existing descendent modular level locking techniques for collaborative SWfMSs. We also propose a role-based access control technique for the management of collaborative SWfMSs. We leverage the Collaborative Interactive Application Methodology (CIAM) for the investigation of role-based access control in the context of collaborative SWfMSs. We present our proposed method with a use-case of Plant Phenotyping and Genotyping research domain. Recent study shows that the collaborative SWfMSs often different sets of opportunities and challenges. From our investigations on existing research works towards collaborative SWfMSs and findings of our prior two studies, we propose an architecture of collaborative SWfMSs. We propose - SciWorCS - a Collaborative Scientific Workflow Management System as a proof of concept of the proposed architecture; which is the first of its kind to the best of our knowledge. We present several real-world use-cases of scientific workflows using SciWorCS. Finally, we conduct several user studies using SciWorCS comprising different real-world scientific workflows (i.e., from myExperiment) to understand the user behavior and styles of work in the context of collaborative SWfMSs. In addition to evaluating SciWorCS, the user studies reveal several interesting facts which can significantly contribute in the research domain, as none of the existing methods considered such empirical studies, and rather relied only on computer generated simulated studies for evaluation

A Study of Implementation Methodologies for Distributed Real Time Collaboration

Collaboration drives our world and is almost unavoidable in the programming industry. From higher education to the top technological companies, people are working together to drive discovery and innovation. Software engineers must work with their peers to accomplish goals daily in their workplace. When working with others there are a variety of tools to choose from such as Google Docs, Google Colab and Overleaf. Each of the aforementioned collaborative tools utilizes the Operational Transform (OT) technique in order to implement their real time collaboration functionality. Operational transform is the technique seen amongst most if not all major collaborative tools in our industry today. However, there is another way of implementing real time collaboration through a data structure called Conflict-free Replicated Data Type (CRDT) which has made claims of superiority over OT. Previous studies have taken place with the focus on comparing the theory behind OT and CRDT\u27s, but as far as we know, there have not been studies which compare real time collaboration performance using an OT implementation versus a CRDT implementation in a popularly used product such as Google Docs or Overleaf. Our work will focus on comparing OT and CRDT\u27s real time collaborative performance in Overleaf, an academic authorship tool, which allows for easy collaboration on academic and professional papers. Overleaf\u27s current published version implements real time collaboration using operational transform. This thesis will contribute an analysis of the current real time collaboration performance of operational transform in Overleaf, an implementation of CRDT\u27s for real time collaboration in Overleaf and an analysis of the performance of real time collaboration through the CRDT implementation in Overleaf. This thesis describes the main advantages and disadvantages of OT vs CRDTs, as well as, to our knowledge, the first results of a non-theoretical attempt at implementing CRDTs for handling document edits in a collaborative environment which was originally operating using an OT implementation

Developing Collaborative XML Editing Systems

In many areas the eXtensible Mark-up Language (XML) is becoming the standard exchange and data format. More and more applications not only support XML as an exchange format but also use it as their data model or default file format for graphic, text and database (such as spreadsheet) applications. Computer Supported Cooperative Work is an interdisciplinary field of research dealing with group work, cooperation and their supporting information and communication technologies. One part of it is Real-Time Collaborative Editing, which investigates the design of systems which allow several persons to work simultaneously in real-time on the same document, without the risk of inconsistencies. Existing collaborative editing research applications specialize in one or at best, only a small number of document types; for example graphic, text or spreadsheet documents. This research investigates the development of a software framework which allows collaborative editing of any XML document type in real-time. This presents a more versatile solution to the problems of real-time collaborative editing. This research contributes a new software framework model which will assist software engineers in the development of new collaborative XML editing applications. The devised framework is flexible in the sense that it is easily adaptable to different workflow requirements covering concurrency control, awareness mechanisms and optional locking of document parts. Additionally this thesis contributes a new framework integration strategy that enables enhancements of existing single-user editing applications with real-time collaborative editing features without changing their source code

A component-based collaboration infrastructure

Groupware applications allow geographically distributed users to collaborate on shared tasks. However, it is widely recognized that groupware applications are expensive to build due to coordination services and group dynamics, neither of which is present in single-user applications. Previous collaboration transparency systems reuse existing single-user applications as a whole for collaborative work, often at the price of inflexible coordination. Previous collaboration awareness systems, on the other hand, provide reusable coordination services and multi-user widgets, but often with two weaknesses: (1) the multi-user widgets provided are special-purpose and limited in number, while no guidelines are provided for developing multi-user interface components in general; and (2) they often fail to reach the desired level of flexibility in coordination by tightly binding shared data and coordination services. In this dissertation, we propose a component-based approach to developing group- ware applications that addresses the above two problems. To address the first prob- lem, we propose a shared component model for modeling data and graphic user inter- face(GUI) components of groupware applications. As a result, the myriad of existing single-user components can be re-purposed as shared GUI or data components. An adaptation tool is developed to assist the adaptation process. To address the second problem, we propose a coordination service framework which systematically model the interaction between user, data, and coordination protocols. Due to the clean separation of data and control and the capability to dynamically "glue" them together, the framework provides reusable services such as data distribution, persistence, and adaptable consistency control. The association between data and coordination services can be dynamically changed at runtime. An Evolvable and eXtensible Environment for Collaboration (EXEC) is built to evaluate the proposed approach. In our experiments, we demonstrate two benefits of our approach: (1) a group of common groupware features adapted from existing single- user components are plugged in to extend the functionalities of the environment itself; and (2)coordination services can be dynamically attached to and detached from these shared components at different granules to support evolving collaboration needs

CyberLiveApp: a secure sharing and migration approach for live virtual desktop applications in a cloud environment

In recent years we have witnessed the rapid advent of cloud computing, in which the remote software is delivered as a service and accessed by users using a thin client over the Internet. In particular, the traditional desktop application can execute in the remote virtual machines without re-architecture providing a personal desktop experience to users through remote display technologies. However, existing cloud desktop applications mainly achieve isolation environments using virtual machines (VMs), which cannot adequately support application-oriented collaborations between multiple users and VMs. In this paper, we propose a flexible collaboration approach, named CyberLiveApp, to enable live virtual desktop applications sharing based on a cloud and virtualization infrastructure. The CyberLiveApp supports secure application sharing and on-demand migration among multiple users or equipment. To support VM desktop sharing among multiple users, a secure access mechanism is developed to distinguish view privileges allowing window operation events to be tracked to compute hidden window areas in real time. A proxy-based window filtering mechanism is also proposed to deliver desktops to different users. To support application sharing and migration between VMs, we use the presentation streaming redirection mechanism and VM cloning service. These approaches have been preliminary evaluated on an extended MetaVNC. Results of evaluations have verified that these approaches are effective and useful

OntoCS: A Web-Based System for Collaborative Ontology Construction

A number of studies on ontology editing tools and ontology-based applications have been proposed for automatically processing knowledge and information. However, the existing methodologies and tools for dealing with ontologies have assumed that the system is restricted to a single user. Main motivation of this paper is to foster collaborations between users, because ontology building is an expensive task. Thereby, in this paper, we present a web-based ontology construction and integration system, which is called OntoCS, to support collaborative interactions between people during creating ontologies. Particularly, inexpert users can collect available language resources from the web to describe concepts in a (even unfamiliar) domain. We believe that this collaborative process is implementing collective intelligence. In conclusion, we have shown that the proposed OntoCS system can efficiently edit and manage multiple ontologies over time

Applied Research Through Partnership: the Experience of the Yorkshire and Humberside Regional Research Observatory

Paper presented at a seminar on ‘Los Observatorios Regionales de Políticas Públicas como Herramientas de Gestión de Información: Una Aproximación al Estudio del Rendimiento Autonómico, at the Centro de Estudios de Gestión, Análisis y Información, Campus de Somosaguas, La Universidad Complutense, Madrid, 23-24 November, 2000 Ten years ago, a Regional Research Observatory (ReRO) was established to provide ‘clients’ in Yorkshire and Humberside with a single point access to a region-wide data and analysis service. The Observatory’s portfolio covered activities relating to applied research and consultancy, intelligence, education and training, publications and networking. The first part of the paper explains the concept of the Observatory as it was initially conceived as a form of partnership across all the universities in the region, outlines the structure of the organization that was created, explains the arrangements for operating the Observatory as a partnership initiative, and exemplifies the outputs and achievements during the first half of the decade. In order to facilitate its regional monitoring activities, ReRO constructed a Regional Intelligence Centre (RIC), a customised geographical information system in which to store key data sets and generate a range of statistical indicators for the region as a whole or its constituent parts. The second part of the paper explains the structure of the RIC and its contents. It argues that the main advantage that derives from the construction of such a centre is the value that is added to raw information through data handling and integration, through skilled interpretation and through the provision of new information, maybe in the form of forecasts of what is likely to happen in the future, as well as analyses of what has happened in the past. The third and final part of the paper explores some of the key issues and difficulties relating to the operation of the Observatory and considers some of the reasons that have accounted for its loss of momentum in the last few years. This has occurred over a period of increased political attention to regional administration and planning in the UK, exemplified by the creation of Scottish and Welsh Assemblies and the emergence of Regional Development Agencies and Regional Assemblies across England. A retrospective evaluation demonstrates a number of lessons that have been learnt and provides a number of useful guidelines to those attempting to establish similar structures elsewhere in the developed world