Consider tips for selecting parts when designing a multi-GPU setup?
Consider tips for selecting parts when designing a multi-GPU setup?
I currently possess more than twenty RTX3070 GPUs at home because of cryptomining. I aim to utilize them as much as possible for testing Apache Spark and Kafka. Although, according to my understanding, you typically need a threadripper to handle many GPUs. However, I noticed that recent motherboards and threadripppers are quite costly, so I’m wondering if there are older models you’d suggest. My plan is to consider something like the M12SWA-TF and an AMD Ryzen™ Threadripper™ PRO 5955WX Processor. Server components might also be a viable option, but I’m very new to this area and don’t know what to choose. Also, I think 128GB of RAM would be ideal.
It seems the focus is on determining how to utilize this multi-GPU configuration. Beyond rendering, there are possibilities for deep learning and AI, though these often require more than just multiple GPUs. The specific needs depend on the task, and not all applications demand a large number of PCIe lanes—similar to what's used in crypto mining. You could test your current setup if it still works.
Threadripper and similar enthusiast boards handle PCIe lane splitting more efficiently than consumer models. Threadripper functions as a 'server' chip for consumers, making it a potentially cost-effective option. The most recent affordable Threadripper chips are around $1400, which isn't very budget-friendly and hasn't been widely released yet.
Intel doesn't offer a direct match, but older 10th Gen X class chips with LGA2066 connectors could provide a more economical solution.
I am organizing my plan for deep learning and focusing on distributing models with Apache Spark and Kafka, aiming to maximize GPU usage. This is why I want to use as many GPUs as possible. It would be helpful to learn how to work with Apache Spark efficiently. Recently, I discovered that EPYC 7642 and EPYC 7402 are promising choices. Their prices are 1100 and 1700 dollars, including RAMs and motherboards. What are your opinions on this? I’m still uncertain about their compatibility.
For reference: Getting-Started
Spark RAPIDS plugin - boost Apache Spark using GPUs
nvidia.github.io
and it appears Kafka mainly depends on storage throughput?
The issue with several 3070 cards is that the connection speed between them isn’t very high. This would likely be the main constraint here. Unlike professional quadro RTX cards which utilize the newest NVlinks for up to 2tb/s, the 3070 offers only about a quarter of that capacity. Therefore, deploying multiple GPUs might not scale linearly. Investigate further and decide on your own.
And as Eximo mentioned, your mining setup can serve as a solid starting point. Try it out before investing heavily in expensive components.