問題文
A team must decide how much accelerator memory each node needs. Which statement correctly describes what that capacity limits?
選択肢
- It limits only the number of concurrent users the node can serve, and has no effect on training on the grounds that training state is kept in host memory, so a training run is sized on host memory instead.
- It limits the network bandwidth the node can use, because outbound messages are staged in accelerator memory and cannot exceed the free space there, so a node with more memory reaches a higher link rate.
- It limits how large a model partition plus its activations and optimizer state can be, and therefore how large a batch the node can process at once.
- It limits nothing in practice, as a node whose accelerator memory is exhausted transparently continues by using host memory instead, at the cost of a small and predictable amount of additional latency.