Thanks to DevOps, we now have a better way to provision and manage IT infrastructure. Instead of standard operating procedures and manual processes, we can do this by using source code. We are talking about Infrastructure as a Code (IaC), which helps users automate the process of infrastructure deployment in a repeatable and consistent manner.
The infrastructure automation and provisioning market is growing globally, especially due to the digitization of most industries, from logistics to healthcare. There are several tech giants in these markets, but the HPC coverage isn’t yet at large and it is a niche worth further exploration.
WHAT IS MOORING
MOORING is one of the five stacks of the SODALITE solution and covers deployment, orchestration, and provisioning. MOORING addresses various problems associated with the difficulty of manual deployments by providing the means to simplify and automate the provisioning of infrastructure. It provides a framework for the usage of IaC and ensures the smooth deployment, orchestration and provisioning of resources based on TOSCA and Ansible parameterized libraries.
And what makes it unique compared to existing tools? The model enables truly concurrent infrastructure provisioning, configuration and application deployment, supports full or partial redeployment and refactoring of TOSCA blueprints, while similar TOSCA orchestrators only support partial parallel execution of blueprints and partial redeployment only at a paid version level. MOORING offers through IaC the choice between various HPC environments and scalability of HPC applications.
Benefits:
- Use TOSCA in building runtime images (international OASIS TOSCA standard compliance)
- Heterogeneous concrete image building process – allows building images for different architectures that we can use for deployment (ARM/Intel/AMD CPU)
- Automated TOSCA compliant infrastructure discovery supporting heterogeneous platforms
- Lightweight application orchestration in HPC beyond Cloud
TECHNOLOGY & COMPONENTS
The MOORING stack relies on innovative technologies to enable deployment, orchestration, and provisioning. It supports the heterogeneity of the building process using IaC and TOSCA with the orchestrator to build heterogeneous runtime images and enables the execution of infrastructure provisioning and configuration in combination with application deployment and configuration on a spectrum of heterogeneous infrastructures ranging from private and public cloud to HPC environments, including full or partial redeployment and parallel deployment execution.
Let’s take a closer look into each of the innovations that relate to the MOORING stack:
Platform discovery service allows the user to automatically discover and create a model in TOSCA over a specific infrastructure (such as TORQUE/SLURM) discovering compute nodes, queues or partitions in a specific infrastructure to create the infrastructural TOSCA resource model, using Ansible Collections for HPC (define login node and gather info in JSON format). It also supports lightweight Openstack and Kubernetes discovery.
Image builder builds the runtime images, taking the form of a container with the application, its deployment plan and the runtime constraints. It supports building images of components for different architectures (AMD64, ARM and Intel).
Orchestrator manages the lifecycle of an application deployed in heterogeneous infrastructures. The xOpera, our lightweight, open source and state-aware orchestrator is able to connect to a specific infrastructure and runs playbooks to provision Openstack or AWS virtual machines, connect to HPC workload managers and deploys and configures applications on these heterogeneous infrastructures.
ALDE enables lifecycle management of cloud, edge and HPC applications. It provides the means to manage different workloads on heterogeneous infrastructures.
SODALITE functional API design provides clear separation between components so that they can be used as a complete distribution, as part of a cluster of connected components, or separately. All components provide solid documentation and examples of integration with.
The innovation of MOORING is also very much involved in each of the three SODALITE use cases. They all use discovery services to identify the types of nodes on which they are deployed and potential changes. Both Vehicle IoT and SNOW use cases use image builder - SNOW uses different types of VMs and clouds, as well as HPC to compute datasets used in the components to get the skyline on the images of snow-covered mountains, and Vehicle IoT uses the image building and deployment to the cloud through k8s and xOpera. The In-silico use case uses mostly HPC through xOpera to interact with MODAK.
Our next step is to focus on extending our initial Kubernetes and edge computing support. This will be particularly useful for our connected cars use case and the enhancement of their KnowGo solution.
And because “sharing is caring”, and we care for the Open Source community, we have been exposing these results and discussing our ideas throughout many presentations, especially related to our xOpera orchestrator – just recently we had a presentation at the EGI 2021 workshop. We also made tutorials for image builder as well as prepared review video of platform discovery.
In the next phase we plan to further strengthen our work towards exposing the SODALITE solution by participating at various conferences and collaborating with important initiatives from this field such as EGI, EOSC, GAIA-X, Heterogeneity Alliance and others.
