Investigating web APIs on the World Wide Web

Abstract—The world of services on the Web, thus far limited to “classical ” Web services based on WSDL and SOAP, has been increasingly marked by the domination of Web APIs, characterised by their relative simplicity and their natural suitability for the Web. Currently, the development of Web APIs is rather autonomous, guided by no established standards or rules, and Web API documentation is commonly not based on an interface description language such as WSDL, but is rather given directly in HTML as part of a webpage. As a result, the use of Web APIs requires extensive manual effort and the wealth of existing work on supporting common service tasks, including discovery, composition and invocation, can hardly be reused or adapted to APIs. Before we can achieve a higher level of automation and can make any significant improvement to current practices and technologies, we need to reach a deeper understanding of these. Therefore, in this paper we present a thorough analysis of the current landscape of Web API forms and descriptions, which has up-to-date remained unexplored. We base our findings on manually examining a body of publicly available APIs and, as a result, provide conclusions about common description forms, output types, usage of API parameters, invocation support, level of reusability, API granularity and authentication details. The collected data provides a solid basis for identifying deficiencies and realising how we can overcome existing limitations. More importantly, our analysis can be used as a basis for devising common standards and guidelines for Web API development. Keywords-Web APIs, RESTful services, Web services I

Pengines: Web Logic Programming Made Easy

When developing a (web) interface for a deductive database, functionality required by the client is provided by means of HTTP handlers that wrap the logical data access predicates. These handlers are responsible for converting between client and server data representations and typically include options for paginating results. Designing the web accessible API is difficult because it is hard to predict the exact requirements of clients. Pengines changes this picture. The client provides a Prolog program that selects the required data by accessing the logical API of the server. The pengine infrastructure provides general mechanisms for converting Prolog data and handling Prolog non-determinism. The Pengines library is small (2000 lines Prolog, 150 lines JavaScript). It greatly simplifies defining an AJAX based client for a Prolog program and provides non-deterministic RPC between Prolog processes as well as interaction with Prolog engines similar to Paul Tarau's engines. Pengines are available as a standard package for SWI-Prolog 7.Comment: To appear in Theory and Practice of Logic Programmin

Component-aware Orchestration of Cloud-based Enterprise Applications, from TOSCA to Docker and Kubernetes

Enterprise IT is currently facing the challenge of coordinating the management of complex, multi-component applications across heterogeneous cloud platforms. Containers and container orchestrators provide a valuable solution to deploy multi-component applications over cloud platforms, by coupling the lifecycle of each application component to that of its hosting container. We hereby propose a solution for going beyond such a coupling, based on the OASIS standard TOSCA and on Docker. We indeed propose a novel approach for deploying multi-component applications on top of existing container orchestrators, which allows to manage each component independently from the container used to run it. We also present prototype tools implementing our approach, and we show how we effectively exploited them to carry out a concrete case study

API diversity for microservices in the domain of connected vehicles

Web services in the domain of connected vehicles are subject to various requirements including high availability and large workloads. Microservices are an architectural style which can fulfill those requirements by fostering the independence and decoupling of software components as reusable services. To achieve this independence, microservices have to implement all aspects of providing the services themselves, including different API technologies for heterogeneous consumers and supporting features like authentication. In this work, we examine the use of a service proxy that externalizes these concerns into a sidecar that provides multiple APIs and common service functionality in a platform-independent manner. We look at how different kinds of API styles and technologies solve selected classes of problems and how we can translate between API technologies. We design and implement a framework for building gateways that enables the creation and composition of reusable components, in the fashion of Lego bricks, to maximize flexibility, while reducing the effort for building gateway components. We design and implement selected components of common and reusable API functionality enabling us to build a reference setup with a service proxy as a sidecar using our framework. Finally, we evaluate the proposed solution to identify benefits and drawbacks of the approach of using our framework as a service proxy. We conclude that the examined approach provides benefits for the development of many polyglot microservices, but splitting one service into two components adds additional complexity that has to be managed.Web Services für vernetzte Fahrzeuge unterliegen unterschiedlichen Anforderungen, unter anderem einer hohen Verfügbarkeit und einem großen Datendurchsatz. Microservices sind ein Architekturstil, der diesen Anforderungen gerecht werden kann, indem er die Unabhängigkeit und Entkopplung von Softwarekomponenten als wiederverwendbare Services fördert. Zum Erreichen der Unabhängigkeit implementieren Microservices alle Aspekte der Servicebereitstellung eigenständig. Dazu gehört verschiedene API Technologien für heterogene Clients bereitzustellen und unterstützende Funktionalität wie Authentifizierung zu implementieren. In dieser Arbeit wird die Verwendung einer Proxy Komponente vor einem Service untersucht, durch welche die Bereitstellung verschiedener API Technologien und allgemeiner unterstützender Funktionalität aus dem Service extrahiert wird. Die Lösungen verschiedener API Technologien und Stile für ausgewählte Klassen an Problemen werden verglichen und mögliche Umwandlungen der verschiedenen API Technologien werden untersucht. Es wird ein Framework konzeptioniert und implementiert, das die Erstellung von Gateways durch Kombination von wiederverwendbaren Komponenten, wie das Zusammensetzen von Legosteinen, ermöglicht. Dieses Framework sorgt für eine hohe Flexibilität, während es den Aufwand bei der Erstellung von Gateways gering hält. Es werden ausgewählte wiederverwendbare Komponenten entworfen, um eine Referenzimplementierung des Ansatzes umzusetzen, bei der allgemeine Funktionalität in einen parallel laufenden Proxy ausgelagert wird. Dieser Ansatz wird evaluiert, indem Vor- und Nachteile anhand eines mit dem Framework erstellten Proxys identifiziert werden. Das Fazit dieser Arbeit ist, dass dieser Ansatz bei Systemen mit vielen Microservices mit unterschiedlichen Programmiersprachen Vorteile bringt, aber die Trennung eines Services in zwei Komponenten eine nicht unerhebliche Komplexität einführt

A Framework for Automating the Invocation of Web APIs

Web APIs, characterized by their relative simplicity and their natural suitability for the Web, have become increasingly dominant in the world of services on the Web. Despite their popularity, Web APIs are so heterogeneous in terms of the underlying principles adopted and the means used for publishing them that discovering, understanding and notably invoking Web APIs is nowadays more an art than a science. In this paper, we present our work towards supporting the automated invocation of Web APIs. In particular, we describe a framework that provides a unique entry point for the invocation of most Web APIs that can be found on the Web, by exploiting non-intrusive semantic annotations of HTML pages describing Web APIs in order to capture both their semantics as well as information necessary to carry out their invocation

Automated OpenAjax Hub Widget Generation for Deep Web Surfacing

Antud magistritöö uurib, kuidas lihtsustada esitluskihil SOAP protokolli kasutavate veebiteenuste, mis on osa süvaveebist, kasutamist. Sellise teema valimist motiveerib asjaolu, et rakenduste kompositsiooniline raskuskese liigub üha enam esitluskihi suunas, kuid hetkel ei ole veebilehitsejale omaste tehnoloogiatega võimalik väliste domeenide teenuseid kasutada, nende väljundit kuvada ja teenuseid omavahel siduda. Et välja selgitada, kuidas antud probleemi lahendada, uuriti, mis on hetkel sellise lähenemise kasutusse võtmisel peamised pidurdavad tegurid. Selgus, et põhilisi raskusi tekitavad asjaolud, et veebilehitsejad ei võimalda teha päringuid rakenduse suhtes välistesse domeenidesse ja et JavaScriptis on SOAP päringute koostamise tugi võrdlemisi limiteeritud. Lisaks tõdeti, et teenustest saadava info visualiseerimine nõuab teenuse väljundi ja kuvamisloogika manuaalset kokku-traageldamist (hard-wiring ing k). Probleemi lahendamiseks otsustati kasutada nö veebividinapõhist lähenemist, kus iga teenuse operatsiooni jaoks genereeritakse nähtamatu JavaScripti vidin, millelt saadav info muudetakse nähtavaks mõne teise vidina poolt. Sellise lähenemise rakendamiseks loodi kaheosaline raamistik, mis koosneb kliendikihist ja serverikihist. Vidinate suhtlemise võimaldamiseks võeti kasutusele OpenAjax Hub raamistik, mis toimib vidinatevaheliste sõnumite vahendajana. Selleks, et vidinad ei oleks tihedalt kokku traageldatud, võeti appi Transformer Widget. Transformer Widget lisab OpenAjax Hub vidinatele võimaluse omavahel suhelda, kasutades semantilist integreerimist. Nähtamatute vidinate genereerimiseks loodi eraldi OpenAjax Hub vidin - Proxy Widget. See toimib teenuseid tarbivate vidinate ja tegeliku teenuse vahelise puhvrina ning lisaks hoolitseb selle eest, et vidin oleks korrektselt Transformer Widgetis registreeritud. Transformer Widgetis registreerimiseks pakub tuge ka serveripool. Serveris genereeritakse selle jaoks dokument, mis kirjeldab vidinate struktuuri ja semantikat ning lisaks ka skeem JSON vormingus andmete kirjeldamiseks. Serveripool kasutab selle jaoks teenuse semantiliselt annoteeritud WSDL keeles kirjeldust, kust saadakse kõik vajalik informatsioon. Proxy Widgeti puhverdamisloogika toimib nii, et esitluskihis võetakse sisendisse JSON vormingus andmed, mille abil luuakse JSON-RPC päring. See saadetakse edasi serveripoolele, mis omakorda transformeerib päringu SOAP päringuks ning saadab lõppteenusele. Lõppteenuselt saadud vastus teisendatakse tagasi JSON-RPC päringuks ning edastatakse Proxy Widgetile. Välja pakutud lahenduse toimimist testiti näidisrakendusega, kus esitluskihi tasemel võimaldati tarbida kolme Äriregistri teenust - firmade leidmine nime järgi, firma aastaaruannete leidmine ning aastaaruannete andmete leidmine. Näidisrakendus tõestas, et teenuste tarbimine ning andmete kuvamine osutus antud lahendusega oluliselt lihtsamaks. Lisaks oli see tõestuseks, et teenuste tarbimine oli võimalik vaid veebilehitsejale omaste tehnoloogiate kasutamisega.The Deep Web, as the name implies, is typically hidden from a common web user, because the information it contains, is not findable through standard search engines. However, this hidden information is often useful to the web user. The question is, what are the possibilities to surface those resources? An example of Deep Web resource would be a SOAP web service of Estonian Business Registry. If a developer wants to use this service in a web application, to query data about annual reports, he should create a service client on the server-side and then manually wire together the user interface and the web service. This requires quite a lot of work and knowledge of server-side programming. Following a current trend where Web application development is geared towards the browser-side implementations, what should a developer do in order to create a client-side mashup using Deep Web resources and web widgets to visualize the annual report data? Unfortunately, his possibilities narrow down quite heavily. The creation of SOAP requests on the client-side is not well supported and he should still put up a server-side proxy to request data outside his own domain. And of course, the wiring with visual widgets still requires much work. This thesis aims to provide a solution that helps a developer to create such client-side mashups. It will provide an infrastructure, that takes care of the cross-domain request problems by creating a common server-side proxy, that anyone could use. It will allow a developer to initiate SOAP requests from within a web browser, by using just JSON request data. Additionally, the solution allows a developer to integrate SOAP web services with visual widgets, by using semantic integration instead of hard-wiring

Data Ontology and an Information System Realization for Web-Based Management of Image Measurements

Image acquisition, processing, and quantification of objects (morphometry) require the integration of data inputs and outputs originating from heterogeneous sources. Management of the data exchange along this workflow in a systematic manner poses several challenges, notably the description of the heterogeneous meta-data and the interoperability between the software used. The use of integrated software solutions for morphometry and management of imaging data in combination with ontologies can reduce meta-data loss and greatly facilitate subsequent data analysis. This paper presents an integrated information system, called LabIS. The system has the objectives to automate (i) the process of storage, annotation, and querying of image measurements and (ii) to provide means for data sharing with third party applications consuming measurement data using open standard communication protocols. LabIS implements 3-tier architecture with a relational database back-end and an application logic middle tier realizing web-based user interface for reporting and annotation and a web-service communication layer. The image processing and morphometry functionality is backed by interoperability with ImageJ, a public domain image processing software, via integrated clients. Instrumental for the latter feat was the construction of a data ontology representing the common measurement data model. LabIS supports user profiling and can store arbitrary types of measurements, regions of interest, calibrations, and ImageJ settings. Interpretation of the stored measurements is facilitated by atlas mapping and ontology-based markup. The system can be used as an experimental workflow management tool allowing for description and reporting of the performed experiments. LabIS can be also used as a measurements repository that can be transparently accessed by computational environments, such as Matlab. Finally, the system can be used as a data sharing tool

Towards Automated Invocation of Web APIs

This paper, targeting the large variety of Web APIs, presents an approach towards automated invocation of Web APIs. This approach applies a data schema, to the SAWSDL lowering schema mapping, a grounding mechanism that connects the ontological representations of Web APIs with their execution messages. It is intuitive to existing standard efforts and effective in coping with the heterogeneities witnessed by a majority of Web APIs