問題文
One deployment must serve three features from the same weights: a data extraction call that must be reproducible, a drafting call that should vary, and an audited summarization call whose output must be explainable later. Which design meets all three?
選択肢
- Pin one set of decoding settings for the whole deployment and choose the values that suit summarization; settings that differ between requests make the service's behavior impossible to reason about, and the extraction call can be made reproducible afterwards by comparing repeated calls instead.
- Use sampling for all three and raise the number of candidates for extraction, so that the most likely of the sampled candidates is the one returned.
- Run three separate deployments, one per feature, each with its own fixed settings, so that no request ever has to carry decoding settings of its own.
- Accept the decoding settings per request, default them to deterministic decoding, and record the settings actually used with each response.