High Performance Spark

High Performance Spark — Holden Karau, Adi Polak, and Rachel Warren

Spark SQL is not only simpler to understand and write, it also has performance optimizations that one would have to do by hand in RDDs. In some ways, Spark SQL is like the “autopilot” option of Spark; you should still keep an eye on it, but it does a pretty good job. — 2026-09-09

Lazy evaluation has important consequences for debugging since it means that a Spark program will fail only at the point of action. — 2026-09-09

Spark offers three options for memory management: in-memory as deserialized data, in-memory as serialized data, and on disk. — 2026-09-09