CICD Pipeline and delivery of Apache Spark Applications on the cloud using AWS

Data Infrastructure & Security

One of the challenge our team is facing to deliver software to production is lack of consistency and excessive manual labor.
Building a continuous integration and continuous delivery pipeline helps delivering software to production environment with speed, safety and reliability.
The concept of automation and tooling is not new but finding the right tools that best fit our need becomes more difficult given a lot of tools is available.
The talk will take a look on how we use the AWS Resources to support the software lifecycle by automating a series of steps