CICD Pipeline and delivery of Apache Spark Applications on the cloud using AWS
Data Infrastructure & Security
Maria Fung
DataOps Engineer at CSU, Chancellors Office
Babu Repaka
DataOps Engineer at CSU, Chancellors Office
One of the challenge our team is facing to deliver software to production is lack of consistency and excessive manual labor.
Building a continuous integration and continuous delivery pipeline helps delivering software to production environment with speed, safety and reliability.
The concept of automation and tooling is not new but finding the right tools that best fit our need becomes more difficult given a lot of tools is available.
The talk will take a look on how we use the AWS Resources to support the software lifecycle by automating a series of steps