data-engineer-pro video for scenario: You are designing a data pipeline that publishes application events to a Pub/Sub topic. Message ordering is not required,
Scenario: You are designing a data pipeline that publishes application events to a Pub/Sub topic. Message ordering is not required, but you need to aggregate events over hourly intervals before loading the results into BigQuery for analysis. The solution must handle large volumes of events and scale effectively. Question: Which technology should you use to process and load the data into BigQuery while ensuring scalability with high event volumes?