Video upload date:  · Duration: PT1H46M27S  · Language: EN

A Spark job is taking longer than expected databricks video

data-engineer-professional video for a Spark job is taking longer than expected. Using the Spark UI, a data engineer notes that the Min, Median, and Max

This is a dedicated watch page for a single video.

Full Certification Question

A Spark job is taking longer than expected. Using the Spark UI, a data engineer notes that the Min, Median, and Max Durations for tasks in a particular stage show the minimum and median time to complete a task as roughly the same, but the max duration for a task to be roughly 100 times as long as the minimum. Which situation is causing increased duration of the overall job?