I Am Researching Multi-GPU Training Speed Difference (Patreon)
Published:
2024-07-23 00:13:46
Edited:
2024-07-23 00:52:30
Imported:
2024-07
Content
Rented this machine on Massed Compute to test Stable Diffusion XL training speed difference with Kohya SS GUI on a 8x A100 (SXM4) machine and see the difference of multi-GPU training SXM vs PCIe linked GPUs
Hopefully will update this post as I get more info.
Update 1
Single GPU works with Batch size 7 but more than 1 GPU fails
Reported Kohya : https://github.com/kohya-ss/sd-scripts/issues/1434#issuecomment-2244049699
Single GPU training speed as below with batch size 7