Core concept
PyTorch DCP (Distributed Checkpoint), FSDP + DeepSpeed support.
Advertisement
How it works
Format compatibility. Async writes to hide I/O time.
Advertisement
Trade-offs + gotchas
Any large training. Recovery from failures. Resharding for different world size.