フリー問題

問題文

A team tunes 40 candidate configurations and picks the one with the best score on the same held-out set each time. The chosen model then performs worse in production than the held-out score suggested. Which statement is accurate?

選択肢

1. Repeated use of that set yields optimistic estimates, so the team should reserve a final holdout.
2. Repeated use of that set yields cautious estimates, so the team should reserve a final holdout.
3. Repeated use of that set yields optimistic estimates, so the team should reserve a larger holdout.
4. Repeated use of that set yields cautious estimates, so the team should reserve a larger holdout.