Use a labeled holdout dataset with ground truth and compute accuracy and F1-score is correct because it gives an objective and repeatable assessment of how the classifier performs on unseen claim photos and it reports both overall correctness and the balance between precision and recall.

Evaluating on a labeled holdout dataset measures generalization and prevents optimistic bias from training data. Accuracy summarizes overall agreement and F1-score combines precision and recall so it is especially useful when damage severity classes are imbalanced or when false positives and false negatives have different business costs.

Track training loss during initial epochs to predict how well the model will perform later is inadequate because training loss reflects fit to the training data and does not indicate real world generalization to new images.

Pick the model that ran the longest, assuming more training yields superior accuracy is flawed because wall clock training time is not a proxy for model quality and longer runs can simply reflect inefficient hyperparameters or overfitting.

Generate synthetic images to probe style shifts and compare the model's outputs can help with robustness testing and uncover distribution shifts but synthetic data should not replace a properly labeled holdout set for formal accuracy measurement.

Full AWS Practitioner Certification Question