data-engineer-pro video for your scooter-sharing company gathers scooter information such as location, battery level, and speed, visualizing it in real time. To
Your scooter-sharing company gathers scooter information such as location, battery level, and speed, visualizing it in real time. To mitigate intermittent connectivity issues, each scooter sends repeated messages within a short timeframe. However, occasional data errors and duplicates have been observed. The messages are received via Pub/Sub and stored in BigQuery. How can you ensure the data is free of duplicates and reject erroneous data with empty fields?