An upstream system sends more than 1000 databricks video
data-engineer-professional video for an upstream system sends more than 1000 files every hour with each file containing 5000-30000 records. The size of each
Answer
Full Certification Question
An upstream system sends more than 1000 files every hour with each file containing 5000-30000 records. The size of each file ranges from 580MB to 1.22 GB. The stream of data is ingested using a medallion architecture. The data engineer needs to limit the amount of data processed in each micro-batch. Which of the following options is valid for limiting the input rate while reading the stream of data?