data-analyst video for a data engineering team has created a Structured Streaming pipeline that processes data in micro-batches and populates gold-level tables.
A data engineering team has created a Structured Streaming pipeline that processes data in micro-batches and populates gold-level tables. The micro-batches are triggered every minute. A data analyst has created a dashboard based on this gold-level data. The project stakeholders want to see the results in the dashboard updated within one minute or less of new data becoming available within the gold-level tables. Which of the following cautions should the data analyst share prior to setting up the dashboard to complete this task?