EMR runnning slow on m5.xlarge AMI 6.5.0 Shredder 2.1.0

Hey @mgloel,

One more very useful source of information in newer AMIs is Spark History Server. You can check it in: Application user interfaces → Spark History UI.

In my experience it’s more reliable than Monitoring tab. But I think you’d have to interpret it on yourself as there’s quite a lot of details. What you’re after is big difference in step duration (between your old and new jobs) and/or failing executors.