問題文
A data scientist searches over 60 combinations of two hyperparameters. Each combination is scored on the validation split, and the best combination is then used. Which statement about this procedure is accurate?
選択肢
- The validation score of the winning combination is optimistic, so a separate untouched split is needed to estimate the performance that can be expected.
- Searching more combinations always improves the expected performance, because the search can only find better settings.
- The search removes the need for cross-validation, because scoring 60 combinations covers the variability that folds would show.
- The validation score of the winning combination is unbiased, because every one of the 60 was scored on rows it had never seen during the fitting step.