Understand Spark hierarchy in term of hardware and software design will help you better in develop an optimized Spark application.
Spark hierarchy
Spark fundamental
Spark is an unified engine designed for large scale distributed data processing and machine learning on compute clusters, whether running on-premise or cloud. It replaces Hadoop MapReduce with its in-memory storage for intermediate computations, making it much faster (100x) than Hadoop MapReduce.