data-engineer-pro video for you're implementing multiple batch jobs requiring scheduled execution. These jobs entail numerous interdependent steps that must
You're implementing multiple batch jobs requiring scheduled execution. These jobs entail numerous interdependent steps that must follow a specific order. Tasks within the jobs encompass executing shell scripts, running Hadoop jobs, and executing queries in BigQuery. Expected runtime ranges from several minutes to several hours, and in case of failure, tasks should be retried a predefined number of times. Which service is best suited to manage the execution of these jobs?