A data analyst has constructed an ML databricks video
ml-engineer-associate video for a data analyst has constructed an ML pipeline utilizing a fixed input dataset with Spark ML. However, the processing time of the
Answer
Full Certification Question
A data analyst has constructed an ML pipeline utilizing a fixed input dataset with Spark ML. However, the processing time of the pipeline is excessive. To improve efficiency, the analyst expanded the number of workers in the cluster. Interestingly, they observed a discrepancy in the row count of the training set post-cluster reconfiguration compared to its count prior to the adjustment. Which strategy ensures a consistent training and test set for each model iteration?