Integration tests automation of data processing pipeline in a Cloud hosted micro-service environment.

Abstract

Micro-services software architecture has become a standard adopted widely in the industry, as it enables high scalability, availability, and development speed. Modern software applications' quality assurance heavily relies on testing. Different testing methods exist, ranging from a function to a whole system. Integration testing stands midway, ensuring workflows operate properly. It guarantees a healthy interaction between components and enables identifying defects that are difficult to otherwise. Integration testing is vital for a distributed systems environment. Components are decoupled, inferring development at different paces. While functionally, a modification to the component may be working as intended, it could break the interaction with associated services. Moreover, services interact using protocols and tools provided by the platform underneath. Software architectures are increasingly leveraging Cloud hosted services, which are often accessible through an API. It is impossible to package mentioned services locally or within a CI/CD pipeline, hence integration tests either must mock the requests to the cloud backend service or skip tests involving them. This research focuses first on enabling integration tests within a Cloud hosted micro-service environment while evaluating multiple different solutions. In a second time, it automates those tests by integrating them into a CI/CD pipeline