Practical notes on software engineering, AWS, distributed systems, performance, and cost optimization.
September 3, 2026
Why Your Spark Job Keeps OOMing Even After You Scale Amazon EMR
EMR sizes executors for the hardware, not your workload. Scaling out adds more of the same undersized executors, and mixed instance fleets can make it worse.
September 3, 2026
When Amazon EMR Gives YARN More Memory Than the Machine Has
On some instance types, EMR tells YARN it can allocate more RAM than Linux sees. Spark looks correctly sized. The node still runs out of memory.