AWS Exams GCP Exams Azure Exams GitHub Exams Jira Exams ISC2 Exams

Video: A Structured Streaming job deployed to databricks video

Question 1 Be Honest
« Back   Next data-engineer-professional Certification Question »
Answer

Full Certification Question

A Structured Streaming job deployed to production has been resulting in higher than expected cloud storage costs. At present, during normal execution, each microbatch of data is processed in less than 3s; at least 12 times per minute, a microbatch is processed that contains 0 records. The streaming write was configured using the default trigger settings. The production job is currently scheduled alongside many other Databricks jobs in a workspace with instance pools provisioned to reduce start-up time for jobs with batch execution. Holding all other variables constant and assuming records need to be processed in less than 10 minutes, which adjustment will meet the requirement?