Spark Streaming performance tuning

Spark Streaming performance tuning

The best performance for Spark Streaming applications in the cluster requires some tuning. T his chapter describes several parameters and configurations to improve the performance of Spark Streaming applications. There are two things you need to consider:

• Efficient use of cluster resources reduces batch data processing time

• Set the correct batch capacity (size) so that the data can be processed faster than the data is received

• Reduce the execution time of batch data

• Set the correct batch capacity
• Memory tuning