問題文
A research team wants to fine-tune an existing language model on a domain corpus. The model's weights plus the training state do not fit in the memory of one accelerator, but do fit across the accelerators inside a single server. The team asks what the infrastructure requirement is. Which requirement follows?
選択肢
- More local storage on the server, because each accelerator writes its portion of the weights to disk between steps and reads it back for the next one.
- A server whose accelerators are connected by a high-bandwidth direct interconnect, so the split model can exchange data at low cost.
- A second server, because a model whose training state does not fit in the memory of one accelerator always has to be split across nodes rather than within a node.
- A separate management network, because splitting a model across accelerators is coordinated out of band.