Context: The proliferation of cloud computing enabled companies to shift their approach
regarding infrastructure provisioning. The uprising of cloud provisioning enabled
by virtualisation technologies sprouted the rise of the Infrastructure as a Service
(IaaS) model. OutSystems takes advantage of the IaaS model to spin-up infrastructure
on-demand while abstracting the infrastructure management from the end-users.
Problem: OutSystems’ orchestrator system handles the automated orchestration of the
clients’ infrastructure, and it must be thoroughly tested. Problems arise because infrastructure
provisioning takes considerable amounts of time, which dramatically increases
the feedback loop for the developers. Currently, the duration of the orchestrator tests
hinder the ability to develop and deliver new features at a desirable pace.
Objectives: The goals of this work include designing an efficient testing strategy that
considers a microservices architecture with infrastructure provisioning capabilities while
integrating it in a Continuous Integration (CI)/Continuous Deployment (CD) pipeline.
Methods: The solution applies multiple testing techniques that target different portions
of the system and follow a pre-determined test distribution to guarantee a balanced
test suite. The strategy was tested against a set of prototypes to evaluate its adequacy and
efficiency. The strategy definition focuses on mapping the type of errors that each test
level should tackle and is, therefore, independent of the employed technologies.
Results: The devised strategy is integrated in a CI/CD pipeline and is capable of
comprehensively test the created prototypes while maintaining a short feedback loop. It
also provides support for testing against commonly found errors in distributed systems
in a deterministic way.
Conclusions: The work developed in this dissertation met the outlined objectives, as
the developed strategy proved its adequacy against the developed prototypes. Moreover,
this work provides a solid starting point for the migration of the orchestrator system to a
microservices architecture
