Scenario: You need to rebuild your batch pipeline gcp video

October 01, 2025  ·  PT1H46M27S  ·  EN

data-engineer-pro video for scenario: You need to rebuild your batch pipeline for structured data on Google Cloud. You currently use PySpark for large-scale

Question 1 Be Honest

« Back   Next gcp data-engineer-pro Question »

Answer

Full Certification Question

Scenario: You need to rebuild your batch pipeline for structured data on Google Cloud. You currently use PySpark for large-scale data transformations, but the pipelines take over 12 hours to complete. You aim to speed up both development and pipeline execution using a serverless tool and SQL syntax. Your raw data is already stored in Cloud Storage. Question: How should you design your pipeline on Google Cloud to meet the requirements for faster development and processing?