A Web Service Composition Method Based on OpenAPI Semantic Annotations

Automatic Web service composition is a research direction aimed to improve the process of aggregating multiple Web services to create some new, specific functionality. The use of semantics is required as the proper semantic model with annotation standards is enabling the automation of reasoning required to solve non-trivial cases. Most previous models are limited in describing service parameters as concepts of a simple hierarchy. Our proposed method is increasing the expressiveness at the parameter level, using concept properties that define attributes expressed by name and type. Concept properties are inherited. The paper also describes how parameters are matched to create, in an automatic manner, valid compositions. Additionally, the composition algorithm is practically used on descriptions of Web services implemented by REST APIs expressed by OpenAPI specifications. Our proposal uses knowledge models (ontologies) to enhance these OpenAPI constructs with JSON-LD semantic annotations in order to obtain better compositions for involved services. We also propose an adjusted composition algorithm that extends the semantic knowledge defined by our model.Comment: International Conference on e-Business Engineering (ICEBE) 9 page

SLA-Driven Governance of RESTful Systems

The Software as a Service (SaaS) paradigm has become entrenched in the industry as a deployment model, bringing flexibility to the customers and a recurring revenue to the business. The main architectural paradigm of SaaS systems is the service-oriented one since it provides numerous advantages in terms of elasticity, fault tolerance, and flexible architectural design. Currently, the RESTful paradigm, a layer of abstraction on the server created by defining resources and entities that can be accessed by means of a URI, is the preferred choice for the construction of SaaS, as it promotes the deployment, isolation and integration of microservices through APIs. Nowadays, APIs are regarded as a new form of business product and ever more organizations are publicly opening up access to their APIs as a way to create new business opportunities. In the same way, other organizations also consume a number of third-party APIs as part of their business. We henceforth define the concept of a RESTful System as an information system following the RESTful paradigm to shape the integration model between both its own components as well as other information systems. Furthermore, understanding governance as the way in which a component is directed and controlled, in RESTful Systems, those components will be the RESTful APIs and what we aim to control or regulate is their behavior (i.e., how an API is being consumed or provided). As APIs are increasingly regarded as business products, a crucial activity is to describe the set of plans (i.e., the pricing) that depicts the functionality and performance being offered to clients. API providers usually define certain limitations in each instance of a plan (e.g., quotas and rates); for example, a free plan might be limited to having one hundred monthly requests, and a professional plan to have five hundred monthly requests. However, although API providers use the Service Level Agreement (SLA) concept to delimit the functionality and guarantees to which they commit to their customers, there is no standard model used by API providers for modeling API pricing (including the plans and limitations). Although some providers do model the information regarding the API pricing and API limitations with an ad hoc approach, there is no widely accepted model in the industry. Wherefore answering questions regarding API limitations (e.g., determining whether or not a certain pricing is valid) is still a manual or non-interoperable process coming along with some inconveniences (being tedious, time-consuming, error-prone, etc.). Understating governance as to how a system is directed and controlled, we translate this concept to meet the SLA-driven approach: we consider the SLA (i.e., API pricing) as the element that will drive the directions, policies and rules to deliver and maintain the RESTful System. Adding the SLA to the idea of governance of RESTful systems leads to the main hypothesis of this dissertation: there is no well-established model for describing API pricings)in RESTful systems, which is hindering the automatic SLA-Driven governance. We claim the main goal of this thesis to be: the creation of an expressive, fully-fledged specification of SLAs for RESTful APIs endorsed with an open ecosystem of tools aimed at the SLA-Driven Governance of RESTful systems. The results of this endeavor are twofold: (I) Creation of a sufficiently expressive specification for the description of API pricings and the analysis of their validity. This comprises: (i) conducting an analysis of real-world APIs to evaluate the characteristics of the API pricings and limitations; (ii) identifying the relevance of SLAs in APIs in both academic and industrial scenarios; (iii) proposing a comprehensive model for describing API pricings; (iv) defining analysis operations for common questions regarding the validity in API pricings and limitations; (v) performing an evaluation of the model in real-world APIs. (II) Implementation of an ecosystem of tools to support the SLA-Driven governance of RESTful APIs. This includes: (i) developing a set of API governance tools; (ii) implementing a validity analysis operation; (iii) performing a validation of the tools and operations in realistic scenarios. In this thesis, we present the Governify4APIs ecosystem as the set comprised of (i) a model aimed at describing API pricings that is closely aligned with industry standards in APIs (OpenAPI Specification) and (ii) a set of companion tools for enacting the automatic governance using our specification, ranging from low-level validation tasks to SaaS solutions based on our model. Governify4APIs is, therefore, a fully-fledged specification, aligned with the mainstream standards and intended to enable an SLA-Driven Governance of RESTful Systems.El paradigma del software como servicio (SaaS) se ha afianzado en la industria como modelo de despliegue, aportando flexibilidad a los clientes y unos ingresos constantes a las organizaciones. El principal paradigma arquitectónico de los sistemas SaaS es la arquitectura orientada a servicios, ya que proporciona numerosas ventajas en términos de elasticidad, tolerancia a fallos y diseño flexible. RESTful, una capa de abstracción sobre el servidor creada mediante la definición de recursos y entidades a las que se puede acceder mediante una URI, es la opción preferida para la construcción de SaaS, ya que promueve el despliegue, el aislamiento y la integración de microservicios a través de APIs. Hoy en día, las APIs se consideran una nueva forma de producto empresarial y cada vez más organizaciones abren públicamente el acceso a sus APIs como forma de crear nuevas oportunidades de negocio. Del mismo modo, otras organizaciones también consumen una serie de APIs de terceros como parte de su negocio. A partir de ahora definimos el concepto de Sistema RESTful como un sistema de información que sigue el paradigma RESTful para conformar el modelo de integración tanto entre sus propios componentes como con otros sistemas de información. Además, entendiendo gobierno como la forma en que se dirige y controla un componente, en los sistemas RESTful, esos componentes serán las APIs RESTful y lo que pretendemos controlar o regular es su comportamiento (es decir, cómo se está consumiendo o proporcionando una API). Dado que las APIs están, cada vez más, siendo consideradas como productos comerciales, una actividad crucial es describir el conjunto de planes (es decir, el pricing) que describe la funcionalidad y el rendimiento que se ofrece a los clientes. Los proveedores de API suelen definir ciertas limitaciones en cada instancia de un plan (por ejemplo, quotas y rates); por ejemplo, un plan gratuito podría estar limitado a tener cien peticiones mensuales, y un plan profesional a tener quinientas peticiones mensuales. Sin embargo, aunque los proveedores de APIs utilizan el concepto de Acuerdo de Nivel de Servicio (SLA) para delimitar la funcionalidad y las garantías a las que se comprometen con sus clientes, no existe ningún modelo estándar usado por los proveedores para modelar el pricing de las API (incluyendo los planes y limitaciones). Aunque algunos proveedores modelan la información relativa a los pricings y las limitaciones de las APIs con un enfoque ad hoc, no existe un modelo ampliamente aceptado en el sector. Por lo tanto, responder a las preguntas relativas a las limitaciones de la APIs (por ejemplo, determinar si un determinado pricing es válido o no) sigue siendo un proceso manual o no interoperable, cosa que conlleva algunos inconvenientes (es tedioso, consume tiempo, es propenso a errores, etc.). Entendiendo el gobierno como la forma de dirigir y controlar un sistema, podemos traducir este concepto teniendo en cuenta el SLA, esto es, consideramos este elemento como aquel sobre el que se realiza la dirección, políticas y reglas para entregar y mantener el sistema RESTful. Añadir el concepto SLA a esa idea de gobierno de sistemas RESTful nos lleva a la hipótesis principal de esta tesis: no existe un modelo bien establecido para describir los SLAs (o pricing) en los sistemas RESTful, lo que está dificultando el gobierno automático. Es, por tanto, el objetivo principal de esta tesis la creación de una especificación expresiva y completa de SLAs para APIs RESTful, respaldada por un ecosistema abierto de herramientas orientadas al gobierno de sistemas RESTful dirigido por SLAs. Los resultados principales han sido: (I) Creación de una especificación suficientemente expresiva para la descripción de los pricings de la API y el análisis de su validez. Esto comprende: (i) realizar un análisis de APIs del mundo real para evaluar las características de los pricings y limitaciones de las APIs; (ii) identificar la relevancia de los SLAs en las APIs tanto en escenarios académicos como industriales; (iii) proponer un modelo completo para describir los pricings de las APIs; (iv) definir operaciones de análisis para preguntas comunes sobre la validez en los pricings y limitaciones de las APIs; (v) realizar una evaluación del modelo en APIs del mundo real. (II) Implementación de un ecosistema de herramientas para apoyar la gobernanza SLA-Driven de las APIs RESTful. Esto incluye: (i) desarrollar un conjunto de herramientas de gobierno de APIs; (ii) implementar una operación de análisis de validez; (iii) realizar una validación de las herramientas y operaciones en escenarios realistas. En esta tesis, presentamos el ecosistema Governify4APIs como el conjunto compuesto por (i) un modelo destinado a describir los pricings de las APIs y alineado estrechamente con los estándares de la industria (OpenAPI) y (ii) un conjunto de herramientas complementarias para el gobierno automático utilizando este modelo, que van desde tareas de validación hasta soluciones SaaS. Por lo tanto, Governify4APIs es una especificación acompañada de todo lo necesario, alineada con los estándares industriales y destinada a permitir un gobierno de sistemas RESTful dirigidos por SLAs

CloudSim Express: A Novel Framework for Rapid Low Code Simulation of Cloud Computing Environments

Cloud computing environment simulators enable cost-effective experimentation of novel infrastructure designs and management approaches by avoiding significant costs incurred from repetitive deployments in real Cloud platforms. However, widely used Cloud environment simulators compromise on usability due to complexities in design and configuration, along with the added overhead of programming language expertise. Existing approaches attempting to reduce this overhead, such as script-based simulators and Graphical User Interface (GUI) based simulators, often compromise on the extensibility of the simulator. Simulator extensibility allows for customization at a fine-grained level, thus reducing it significantly affects flexibility in creating simulations. To address these challenges, we propose an architectural framework to enable human-readable script-based simulations in existing Cloud environment simulators while minimizing the impact on simulator extensibility. We implement the proposed framework for the widely used Cloud environment simulator, the CloudSim toolkit, and compare it against state-of-the-art baselines using a practical use case. The resulting framework, called CloudSim Express, achieves extensible simulations while surpassing baselines with over a 71.43% reduction in code complexity and an 89.42% reduction in lines of code

DSL-driven Integration of HTTP Services in DIME

As the integration of web services into web applications becomes more and more common, it is necessary to find a solution for low-code or no-code environments. This thesis is the first attempt to allow for the easy integration of web services into the low-code immersive modeling environment (IME) DIME, by means of a domain-specific language (DSL), the HTTP-DSL. DIME users can specify HTTP requests to web services with few lines of code, and then integrate these requests into the modeling languages provided by DIME

Web Application Programming Interfaces (APIs): general-purpose standards, terms and European Commission initiatives

From their inception, digital technologies have had a huge impact on our everyday life. In both the private and the public sectors, they have contributed to, or at times driven, change in organisational structures, ways of working, and how products and services are shaped and shared. Governments and public administration units, driven by the digital evolution of information and communications technology (ICT), are evolving from traditional workflow-based public service provisions to digital equivalents (e-government), with more innovative forms of government and administration looking for the engagement of citizens and the private sector to co-create final services through user-centric approaches. Application Programming Interfaces (APIs), which are one of the most relevant ICT solutions, have contributed to this notable shift in the adoption of technology, especially when used over the web. They have affected the global economy of the private sector and are contributing to the digital transformation of governments. To explore this in more detail, the European Commission recently started the APIs4DGov study. One of the outputs of the study is an analysis of the API technological landscape, including its related standards and technical specifications for general purpose use. The goal of the analysis presented in this brief report is to support the definition of stable APIs for digital government services adopted by governments or single public administration units. Such adoption would avoid the need to develop ad hoc solutions that could have limited scalability or potential for reuse. Instead, the work suggests that we should consider a number of existing standards provided by standardisation bodies or, at least, technical specifications written by well-recognised consortia, vendors or users. The aim of this report is also to support API stakeholders in the identification and selection of such solutions. To do this, it first gives a series of definitions to help the reader understand some basic concepts, as well as related standards and technical specifications. Then, it presents the description and classification (by resource representation, security, usability, test, performance and licence) of the standards and technical specifications collected. A shortlist of these documents (based on their utilisation, maintenance and stability) is also proposed, together with a brief description of each of them. Finally, the report provides a useful glossary with definitions of the relevant terms we have collected so far within the APIs4DGov study.JRC.B.6-Digital Econom

Testiautomaatio mikropalvelujärjestelmälle

This thesis discusses the topic of automated testing as it relates to microservice systems. Microservice architecture is a highly scalable way of designing and implementing online applications. Since microservice applications are network-based applications by nature, testing them has to also happen in a network environment. Automating tests for this kind of environment involves generating artificial network traffic, often in the form of HTTP requests to a network API of some kind, like a REST API. These topics are discussed from a test design and implementation point of view, along with main features of the microservice architecture and automated testing in general. The main part of this thesis describes and documents the process of designing and implementing a test automation framework for Intel Insight, an automatic image storage and photogrammetry processing platform that is implemented as a microservice system. The framework design involves setting initial requirements for potential automation tools and finding and evaluating candidates for the task. In the end, the framework core is formed by automation tools Postman, Selenium, and SikuliX. The use of this combination for test automation purposes is examined by looking at how to the tools can be used to automate a core use case of the Intel Insight platform. The resulting framework was found to be well-suited and versatile enough for its intended purpose. The tools of the framework had a low barrier of entry to them and as such were easy to begin working with and to integrate automated test cases implemented with them to Continuous Integration systems Gitlab CI and Jenkins. All tools are reviewed in-depth, and positives and negatives of each individual automation tool that were encountered during test implementation are analyzed. The main negatives are brought up as possible ideas for future development of each tool, enabled by the fact that they are all open-source projects

Security and Usability in the HeadREST Language

Tese de mestrado, Engenharia Informática (Arquitectura, Sistemas e Redes de Computadores) Universidade de Lisboa, Faculdade de Ciências, 2020Actualmente, observa-se o crescimento contínuo de serviços web, sem sinais de abrandar. As trocas de informação com estes serviços seguem diferentes padrões. De entre os muitos padrões utilizados, destaca-se o REST (REpresentational State Transfer). O REST é um estilo arquitectural muito utilizado actualmente. Neste estilo arquitectural as operações e propriedades do protocolo HTTP, sobre o qual o World Wide Web funciona, são aproveitadas para realizar as interacções de clientes com serviços web. Em REST, o elemento basilar são os recursos, que correspondem a pedaços de informação que podem ser referenciados por um identificador. Cada recurso tem uma, ou várias, representações, que podem ter diferentes formatos, e que podem mudar na sequência de operações executadas sobre o mesmo. Um serviço web que adere ao estilo arquitectural REST é chamado de serviço REST. Para programar clientes de um serviço REST é fundamental que esteja disponível uma boa documentação da sua API, com especificações claras das suas operações e dos dados trocados nestas operações entre os clientes e o serviço. No desenvolvimento deste tipo de serviços são utilizadas linguagens de descrição de interfaces, tal como a OpenAPI Specification, o RAML ou a API Blueprint. Estas linguagens permitem especificar formalmente as operações suportadas por um serviço REST e oferecem a capacidade de documentar os dados que são trocados durante as interacções com o serviço. Apesar da sua popularidade, estas linguagens de especificação têm um poder expressivo limitado. Uma das limitações é que não terem capacidade para descrever com precisão o comportamento das diferentes operações. Numa tentativa de endereçar estas limitações, tem vindo a ser desenvolvida a linguagem HeadREST. A linguagem tem um sistema de tipos refinados que permite restringir os valores admissíveis de um tipo, e portanto descrever com mais rigor os tipos dos dados trocados num serviço REST. Para permitir especificar com precisão as operações de um serviço REST, a linguagem HeadREST dispõe de asserções. Estas asserções, semelhantes aos triplos de Hoare, são compostas por uma pré-condição, um URI template da operação e uma pós-condição. As asserções especificam que, quando a pré-condição é satisfeita, a execução da operação estabelece a pós-condição. Devido ao sistema de tipos refinados não é possível resolver através de regras sintácticas as relações de subtipagem. Para endereçar esta situação foi tomada a decisão de utilizar um procedimento semântico para tratar destas situações. A relação de subtipagem é transformada em fórmulas lógicas de primeira ordem, que são depois dadas a um SMT solver para as resolver. Apesar do seu grande poder expressivo, o HeadREST, como linguagem de especificação, está longe de ser perfeita. Um dos problemas mais importantes está relacionado com a sua usabilidade. Apesar da linguagem permitir descrever operações com grande rigor e detalhe, isso é feito à custa de asserções bastante complexas que são não só difíceis de escrever correctamente, como de compreender. Muitas das linguagens de especificação de serviços REST oferecem, mesmo que de forma limitada, uma forma de expressar o que o serviço exige em termos de autenticação e/ou autorização. Existem vários tipos de autenticação e autorização que podem ser usados para restringir acesso a recursos em serviços REST, por exemplo, API keys, Tokens, http authentication&HTTP digest, OAuth 2.0, OpenID Connect. Para além disto, cada serviço REST pode tomar abordagens diferentes em relação a políticas de autorização. Este trabalho endereçou estes dois problemas e pretendeu contribuir com soluções que os ajudassem a resolver. Para o problema de usabilidade, a solução concebida passou pela criação de extensões para a linguagem com ênfase em expressões derivadas. A linguagem foi estendida com: (i) iteradores quantificados que permitem expressar melhor propriedades sobre arrays, (ii) interpolação para permitir criar Strings a partir de URIs de uma forma mais simples e directa, (iii) um operador de extracção que permite aceder à representação de um recurso se esta for única e finalmente, (iv) funções que permitem abstrair expressões repetidas de uma forma mais flexível (apenas as funções não são derivadas). A abordagem para endereçar a especificação de políticas de segurança em APIs REST assentou na adição (i) de um novo tipo Principal, correspondente às entidades autenticadas e (ii) de uma função não-interpretada principalof capturando o Principal autenticado por um valor usado na autenticação. A linguagem foi estendida com a definição de funções não-interpretadas, para permitir que sejam feitas associações entre o tipo Principal e outros dados que possam vir de diferentes fontes (representações, templates de URIs, corpo dos pedidos, etc.), dando assim a possibilidade de especificar os diferentes tipos de políticas de segurança usadas em serviços REST. A avaliação das soluções propostas foi realizada de diferentes formas. Foi realizado um estudo com utilizadores envolvendo a resposta a um questionário com perguntas sobre a linguagem HeadREST antes e depois das extensões e foi feito um estudo quantitativo a comparar o impacto das extensões em termos de métricas de complexidade das especificações e no desempenho do validador. Para avaliar as extensões referentes à segurança foram realizados alguns casos de estudo, envolvendo a especificação parcial de alguns serviços REST do "mundo-real". Foi ainda explorado o impacto que as extensões introduzidas na linguagem têm nas ferramentas que actualmente fazem parte do ecossistema HeadREST: (i) a ferramenta HeadREST-RTester, que permite testar automaticamente a conformidade da implementação de um serviço REST contra uma especificação HeadREST da sua API, (ii) a ferramenta HeadREST-Codegen, que faz a geração de código, e (iii) a linguagem SafeRestScript, uma linguagem de script em que é realizada estaticamente a validação das chamadas a serviços REST cujas APIs tenham sido especificadas com HeadREST. A linguagem HeadREST possui um validador, um plug-in para o IDE Eclipse e uma versão headless para ser utilizada no terminal.The RESTful services are still today the most popular type of web services. Communication between these services and their clients happens through their RESTful APIs and, to correctly use the services, good documentation of their APIs is paramount. With the purpose of facilitating the specification of web APIs, different Interface Definition Languages (IDLs) have been developed. However, they tend to be quite limited and impose severe restrictions in what can be described. As a consequence, many important properties can only be written in natural language. HeadREST is a specification language of RESTful APIs that poses itself as a solution to the limitations faced by other IDLs. The language has an expressive type system via refinement types and supports the description of service endpoints through assertions that, among other things, allow to express relations between the information transmitted in a request and the response. HeadREST, like other IDLs, is however not without its limitations and issues. This thesis addresses the problems that currently affect the usability of HeadREST and also its lack of expressiveness for specifying security properties of RESTful APIs. The proposed solution encompasses (i) an extension of HeadREST with new specification primitives that can improve the degree of usability of the language and (ii) an ortogonal extension of HeadREST with specification primitives that support the description of authentication and authorisation policies for RESTful APIs. The evaluation of the proposed solution, performed through a user study, a quantitative analysis and the development of case studies, indicates that the primitives targeting the usability issues indeed improve usability of the language and that HeadREST become able to capture dynamic, state-based dependencies that exist in the access control policies that can be found in RESTful APIs