Predictable execution of scientific workflows using advance resource reservations

Scientific Workflows are long-running and data intensive, and may encompass operations provided by multiple physically distributed service providers. The traditional approach to execute such workflows is to employ a single workflow engine which orchestrates the entire execution of a workflow instance, while being mostly agnostic about the state of the infrastructure it operates in (e.g., host or network load). Therefore, such centralized best-effort execution may use resources inefficiently -- for instance, repeatedly shipping large data volumes over slow network connections -- and cannot provide Quality of Service (QoS) guarantees. In particular, independent parallel executions might cause an overload of some resources, resulting in a performance degradation affecting all involved parties. In order to provide predictable behavior, we propose an approach where resources are managed proactively (i.e., reserved before being used), and where workflow execution is handled by multiple distributed and cooperating workflow engines. This allows to efficiently use the existing resources (for instance, using the most suitable provider for operations, and considering network locality for large data transfers) without overloading them, while at the same time providing predictability -- in terms of resource usage, execution timing, and cost -- for both service providers and customers. The contributions of this thesis are as follows. First, we present a system model which defines the concepts and operations required to formally represent a system where service providers are aware of the resource requirements of the operations they make available, and where (planned) workflow executions are adapted to the state of the infrastructure. Second, we describe our prototypical implementation of such a system, where a workflow execution comprises two main phases. In the planning phase, the resources to reserve for an upcoming workflow execution must be determined; this is realized using a Genetic Algorithm. We present conceptual and implementation details of the chromosome layout, and the fitness functions employed to plan executions according to one or more user-defined optimization goals. During the execution phase, the system must ensure that the actual resource usages abide to the reservations made. We present details on how such enforcement can be performed for various resource types. Third, we describe how these parts work together, and how the entire prototype system is deployed on an infrastructure based on WSDL/SOAP Web Services, UDDI Registries, and Glassfish Application Servers. Finally, we discuss the results of various evaluations, encompassing both the planning and runtime enforcement

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

We are recently witnessing an increased adoption of microservice architectures by the industry for achieving scalability by functional decomposition, fault-tolerance by deployment of small and independent services, and polyglot persistence by the adoption of different database technologies specific to the needs of each service. Despite the accelerating industrial adoption and the extensive research on microservices, there is a lack of thorough investigation on the state of the practice and the major challenges faced by practitioners with regard to data management. To bridge this gap, this paper presents a detailed investigation of data management in microservices. Our exploratory study is based on the following methodology: we conducted a systematic literature review of articles reporting the adoption of microservices in industry, where more than 300 articles were filtered down to 11 representative studies; we analyzed a set of 9 popular open-source microservice-based applications, selected out of more than 20 open-source projects; furthermore, to strengthen our evidence, we conducted an online survey that we then used to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 practitioners and researchers. Through this process, we were able to categorize the state of practice and reveal several principled challenges that cannot be solved by software engineering practices, but rather need system-level support to alleviate the burden of practitioners. Based on the observations we also identified a series of research directions to achieve this goal. Fundamentally, novel database systems and data management tools that support isolation for microservices, which include fault isolation, performance isolation, data ownership, and independent schema evolution across microservices must be built to address the needs of this growing architectural style

Consortium Proposal NFDI-MatWerk

This is the official proposal the NFDI-consortium NFDI-MatWerk submitted to the DFG within the request for funding the project. Visit www.dfg.de/nfdi for more infos on the German National Research Data Infrastructure (Nationale Forschungsdateninfrastruktur - NFDI) initiative. Visit www.nfdi-matwerk.de for last infos about the project NFDI-MatWerk

Content warehouses

Nowadays, content management systems are an established technology. Based on the experiences from several application scenarios we discuss the points of contact between content management systems and other disciplines of information systems engineering like data warehouses, data mining, and data integration. We derive a system architecture called "content warehouse" that integrates these technologies and defines a more general and more sophisticated view on content management. As an example, a system for the collection, maintenance, and evaluation of biological content like survey data or multimedia resources is shown as a case study

Proceedings of the 12th International Conference on Digital Preservation

The 12th International Conference on Digital Preservation (iPRES) was held on November 2-6, 2015 in Chapel Hill, North Carolina, USA. There were 327 delegates from 22 countries. The program included 12 long papers, 15 short papers, 33 posters, 3 demos, 6 workshops, 3 tutorials and 5 panels, as well as several interactive sessions and a Digital Preservation Showcase

Proceedings of the 12th International Conference on Digital Preservation

The 12th International Conference on Digital Preservation (iPRES) was held on November 2-6, 2015 in Chapel Hill, North Carolina, USA. There were 327 delegates from 22 countries. The program included 12 long papers, 15 short papers, 33 posters, 3 demos, 6 workshops, 3 tutorials and 5 panels, as well as several interactive sessions and a Digital Preservation Showcase

A Model-Driven Approach for the Design, Implementation, and Execution of Software Development Methods

[EN] Software development projects are diverse in nature. For this reason, software companies are often forced to define their methods in-house. In order to define methods efficiently and effectively, software companies require systematic solutions that are built upon sound methodical foundations. Providing these solutions is the main goal of the Method Engineering discipline. Method Engineering is the discipline to design, construct, and adapt methods, techniques, and tools for the development of information systems. Over the last two decades, a lot of research work has been performed in this area. However, despite its potential benefits, Method Engineering is not widely used in industrial settings. Some of the causes of this reality are the high theoretical complexity of Method Engineering and the lack of adequate software support. In this thesis, we aim to mitigate some of the problems that affect Method Engineering by providing a novel methodological approach that is built upon Model-Driven Engineering (MDE) foundations. The use of MDE enables a rise in abstraction, automation, and reuse that allows us to alleviate the complexity of our Method Engineering approach. Furthermore, by leveraging MDE techniques (such as metamodeling, model transformations, and models at runtime), our approach supports three phases of the Method Engineering lifecycle: design, implementation, and execution. This is unlike traditional Method Engineering approaches, which, in general, only support one of these phases. In order to provide software support for our proposal, we developed a Computer-Aided Method Engineering (CAME) environment that is called MOSKitt4ME. To ensure that MOSKitt4ME offered the necessary functionality, we identified a set of functional requirements prior to developing the tool. Then, after these requirements were identified, we defined the architecture of our CAME environment, and, finally, we implemented the architecture in the context of Eclipse. The thesis work was evaluated by means of a study that involved the participation of end users. In this study, MOSKitt4ME was assessed by means of the Technology Acceptance Model (TAM) and the Think Aloud method. While the TAM allowed us to measure usefulness and ease of use in a subjective manner, the Think Aloud method allowed us to analyze these measures objectively. Overall, the results were favorable. MOSKitt4ME was highly rated in perceived usefulness and ease of use; we also obtained positive results with respect to the users' actual performance and the difficulty experienced.[ES] Los proyectos de desarrollo de software son diversos por naturaleza. Por este motivo, las compañías de software se ven forzadas frecuentemente a definir sus métodos de manera interna. Para poder definir métodos de forma efectiva y eficiente, las compañías necesitan soluciones sistemáticas que estén definidas sobre unos fundamentos metodológicos sólidos. Proporcionar estas soluciones es el principal objetivo de la Ingeniería de Métodos. La Ingeniería de Métodos es la disciplina que aborda el diseño, la construcción y la adaptación de métodos, técnicas y herramientas para el desarrollo de sistemas de información. Durante las dos últimas décadas, se ha llevado a cabo mucho trabajo de investigación en esta área. Sin embargo, pese a sus potenciales beneficios, la Ingeniería de Métodos no se aplica ampliamente en contextos industriales. Algunas de las principales causas de esta situación son la alta complejidad teórica de la Ingeniería de Métodos y la falta de un apropiado soporte software. En esta tesis, pretendemos mitigar algunos de los problemas que afectan a la Ingeniería de Métodos proporcionando una propuesta metodológica innovadora que está basada en la Ingeniería Dirigida por Modelos (MDE). El uso de MDE permite elevar el nivel de abstracción, automatización y reuso, lo que posibilita una reducción de la complejidad de nuestra propuesta. Además, aprovechando técnicas de MDE (como por ejemplo el metamodelado, las transformaciones de modelos y los modelos en tiempo de ejecución), nuestra aproximación da soporte a tres fases del ciclo de vida de la Ingeniería de Métodos: diseño, implementación y ejecución. Esto es a diferencia de las propuestas existentes, las cuales, por lo general, sólo dan soporte a una de estas fases. Con el objetivo de proporcionar soporte software para nuestra propuesta, implementamos una herramienta CAME (Computer-Aided Method Engineering) llamada MOSKitt4ME. Para garantizar que MOSKitt4ME proporcionaba la funcionalidad necesaria, definimos un conjunto de requisitos funcionales como paso previo al desarrollo de la herramienta. Tras la definción de estos requisitos, definimos la arquitectura de la herramienta CAME y, finalmente, implementamos la arquitectura en el contexto de Eclipse. El trabajo desarrollado en esta tesis se evaluó por medio de un estudio donde participaron usuarios finales. En este estudio, MOSKitt4ME se evaluó por medio del Technology Acceptance Model (TAM) y del método Think Aloud. Mientras que el TAM permitió medir utilidad y facilidad de uso de forma subjetiva, el método Think Aloud permitió analizar estas medidas objetivamente. En general, los resultados obtenidos fueron favorables. MOSKitt4ME fue valorado de forma positiva en cuanto a utilidad y facilidad de uso percibida; además, obtuvimos resultados positivos en cuanto al rendimiento objetivo de los usuarios y la dificultad experimentada.[CA] Els projectes de desenvolupament de programari són diversos per naturalesa. Per aquest motiu, les companyies es veuen forçades freqüenment a definir els seus mètodes de manera interna. Per poder definir mètodes de forma efectiva i eficient, les companyies necessiten solucions sistemàtiques que estiguin definides sobre uns fundaments metodològics sòlids. Proporcionar aquestes solucions és el principal objectiu de l'Enginyeria de Mètodes. L'Enginyeria de Mètodes és la disciplina que aborda el diseny, la construcció i l'adaptació de mètodes, tècniques i eines per al desenvolupament de sistemes d'informació. Durant les dues últimes dècades, s'ha dut a terme molt de treball de recerca en aquesta àrea. No obstant, malgrat els seus potencials beneficis, l'Enginyeria de Mètodes no s'aplica àmpliament en contextes industrials. Algunes de les principals causes d'aquesta situació són l'alta complexitat teòrica de l'Enginyeria de Mètodes i la falta d'un apropiat suport de programari. En aquesta tesi, pretenem mitigar alguns dels problemes que afecten a l'Enginyeria de Mètodes proporcionant una proposta metodològica innovadora que està basada en l'Enginyeria Dirigida per Models (MDE). L'ús de MDE ens permet elevar el nivell d'abstracció, automatització i reutilització, possibilitant una reducció de la complexitat de la nostra proposta. A més a més, aprofitant tècniques de MDE (com per exemple el metamodelat, les transformacions de models i els models en temps d'execució), la nostra aproximació suporta tres fases del cicle de vida de l'Enginyeria de Mètodes: diseny, implementació i execució. Açò és a diferència de les propostes existents, les quals, en general, només suporten una d'aquestes fases. Amb l'objectiu de proporcionar suport de programari per a la nostra proposta, implementàrem una eina CAME (Computer-Aided Method Engineering) anomenada MOSKitt4ME. Per garantir que MOSKitt4ME oferia la funcionalitat necessària, definírem un conjunt de requisits funcionals com a pas previ al desenvolupament de l'eina. Després de la definició d'aquests requisits, definírem la arquitectura de l'eina CAME i, finalment, implementàrem l'arquitectura en el contexte d'Eclipse. El treball desenvolupat en aquesta tesi es va avaluar per mitjà d'un estudi on van participar usuaris finals. En aquest estudi, MOSKitt4ME es va avaluar per mitjà del Technology Acceptance Model (TAM) i el mètode Think Aloud. Mentre que el TAM va permetre mesurar utilitat i facilitat d'ús de manera subjectiva, el mètode Think Aloud va permetre analitzar aquestes mesures objectivament. En general, els resultats obtinguts van ser favorables. MOSKitt4ME va ser valorat de forma positiva pel que fa a utilitat i facilitat d'ús percebuda; a més a més, vam obtenir resultats positius pel que fa al rendiment objectiu dels usuaris i a la dificultat experimentada.Cervera Úbeda, M. (2015). A Model-Driven Approach for the Design, Implementation, and Execution of Software Development Methods [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/53931TESI

Computer Vision for Multimedia Geolocation in Human Trafficking Investigation: A Systematic Literature Review

The task of multimedia geolocation is becoming an increasingly essential component of the digital forensics toolkit to effectively combat human trafficking, child sexual exploitation, and other illegal acts. Typically, metadata-based geolocation information is stripped when multimedia content is shared via instant messaging and social media. The intricacy of geolocating, geotagging, or finding geographical clues in this content is often overly burdensome for investigators. Recent research has shown that contemporary advancements in artificial intelligence, specifically computer vision and deep learning, show significant promise towards expediting the multimedia geolocation task. This systematic literature review thoroughly examines the state-of-the-art leveraging computer vision techniques for multimedia geolocation and assesses their potential to expedite human trafficking investigation. This includes a comprehensive overview of the application of computer vision-based approaches to multimedia geolocation, identifies their applicability in combating human trafficking, and highlights the potential implications of enhanced multimedia geolocation for prosecuting human trafficking. 123 articles inform this systematic literature review. The findings suggest numerous potential paths for future impactful research on the subject