Today, cloud computing is a well-established paradigm that
enjoys a moderate level of popularity among Science Gateways
and Virtual Research Environments. Historically, Science
Gateways have empowered researchers with traditional onpremise High-Performance Computing (HPC), but recent
years have seen many turning to the cloud to fill that space. One often cited cloud complaint is the distance that data must travel. Sensors and input devices generate data locally in the lab, but even for a basic calculation, all those bits must make the costly journey to a data centre hundreds of miles away. The proliferation of the Internet of Things, though, is quickly changing this. As microchips decrease in size, the compute power of those sensors and input devices is increasing, along with every other compute or networking device in the lab. These devices, at the edge of the network, can now be exploited for less intensive computations, such as pre-processing or analytics.
This creates an ideal opportunity for Science Gateways.
As well as connecting research communities to the cloud, they
can bring edges into the mix and get the advantages of both. In that, lies a considerable challenge though. Edge devices come in all shapes and sizes – or, from a computing point of view, all operating systems and system architectures. Many such devices are connected by Wi-Fi, so connection issues are common and unpredictable. Once deployed, communications between edge and cloud must be secured.
DevOps engineers are solving these issues with OCIcompliant
containers (Docker), configuration management (Ansible®), container orchestrators (Kubernetes), and edgeenabling
add-ons (k0s, k3s, KubeEdge or microk8s). But configuring a container orchestration cluster in the cloud takes expert knowledge and effort, and navigating the landscape of
edge add-ons takes even more. These are lengthy, manual
processes, not suited to a Science Gateway administrator.
This abstract proposes a generic solution to the problem of
edge connectivity by abstracting away this complexity and
automating the process. Users bring their containers and edge
devices, and Kubernetes, Ansible and KubeEdge or k3s do the
rest, with some help from MiCADO, a dynamic cloud
orchestrator. Figure 1 shows this process.
An Ansible Playbook creates a Kubernetes cluster in the
cloud, deploying an appropriate Kubernetes edge add-on in the
process. Then, a user provides a description of their application (container images and configuration) and edge nodes (connection details). MiCADO supports these descriptions in TOSCA, but higher-level tools can be used to generate TOSCA from descriptive metadata.
Actioning this description triggers Kubernetes and a second
Ansible Playbook, which create the connection to the edge and
schedule containers appropriately across edge and cloud. In
MiCADO, the TOSCASubmitter is responsible for these
triggers. A KubernetesAdaptor transpiles the relevant container descriptions to Kubernetes Manifests and executes them. An AnsibleAdaptor, newly created for connecting edges on the fly,automates configuration and execution of the Playbook.
This approach is being used for a Science Gateway in the
PITHIA-NRF Project, and for an Industry Gateway in the
DIGITbrain Project. It supports a simpler route to edge
computing that can be exploited by Science Gateways for
efficient computing along the cloud-to-edge continuum
