Spark challenges

Data Engineering

Investigating and rewriting Apache Spark jobs to avoid hitting spark limitations and improve data processing performance