問題文
Users of a chat feature complain that nothing appears on screen for several seconds when the answer is long, even though the total time is acceptable. The team wants to improve the perceived responsiveness without changing the model. What should they do?
選択肢
- Cache the whole response for identical requests, which helps only when the same question is asked twice in a row.
- Increase the maximum output length so that the response is generated in one larger step, which lets the server return everything at once rather than in pieces.
- Send the generated tokens to the client as they are produced instead of waiting for the full response.
- Lower the decoding randomness, and a more deterministic decode finishes each token faster and therefore shortens the delay before the first character appears.