What are the recommendations for a motherboard or case that can support four 4090 GPUs?
What are the recommendations for a motherboard or case that can support four 4090 GPUs?
I’m looking to set up a system starting with two 4090 GPUs, with the option to add two more later if required. This setup is intended for running large language models and fine-tuning smaller ones. I’ve encountered very few motherboards with more than three PCI slots, and among them, the third slot is usually a x1 port. These boards have closely spaced slots, making it tricky to place the 4090s elsewhere. I have no prior experience mounting cards in alternative cases and don’t know what features to prioritize. Any guidance on choosing a case, selecting a motherboard, or adapting card placement would be invaluable. A quieter cooling solution would also be appreciated, though most advice suggests air cooling works best for GPUs. Thank you for your help!
My suggestion is to utilize Newegg filters to locate the mobo matching that specification. A very fast search right now suggests this need might lean towards AMD, though I didn’t go into it thoroughly.
I also concur that there could be challenges with mounting in a case. Depending on the situation, you might find a better fit in a mining rig-style case or bench tailored to this particular requirement.
Thaks, I'll review the mining cases carefully. Most of them appear open to public scrutiny, which might make them quite noisy. The newegg filters are useful, but once I secure top four slots, I'll be using a seriously powerful CPU—though for my needs, LLMs take priority over GPU performance. A good CPU paired with 1.5x more memory than main memory seems to be the main goal.
I realize I'm a bit late joining this chat, but I'm new to the forum and just discovered this.
For someone familiar with LLMs who is currently assembling a dual GPU setup tailored for this task, I’d like to emphasize that the idea CPU doesn’t matter for LLMs is a myth.
Firstly, a powerful CPU with many cores lets you run bigger models via CPU inference for parts that can’t fit into VRAM. For instance, with 256GB DDR5 and a TR 7000X CPU, you could execute an unquantized 70B model using GPU for certain layers while relying on CPU/RAM for the remainder—though it would be quite slow (a few tokens per second).
Secondly, using 4x4090 cards will hit PCIe speed limits of consumer CPUs, and you’ll see why high-end options like the Threadripper Pro 7000x series are better suited for handling traffic between GPUs, especially since NVLink isn’t available with RTX4090.
Lastly, if you plan to fine-tune or train your own LLM models instead of just running inference, the pre-processing phase heavily depends on CPU and RAM performance. Handling terabytes of text data demands a fast CPU.
It is a misconception that CPU has no impact on LLMs.
Once you move tasks to the CPU, performance drops significantly.
Most of my work involves inference on models optimized elsewhere, which means less reliance on CPU power.
This supports the idea that CPU isn't critical; however, a solid CPU is still useful for pre/post processing, even with many cores and threads, particularly when running full inference.
You won't locate a typical case that supports four RTX 4090s unless it's water cooled. It's unclear how much PCIe bandwidth is necessary, but if it is, you can't rely on a desktop CPU for this setup. The available lanes are very restricted, and connecting them directly to the CPU isn't possible. If bandwidth isn't a concern, consider a board with extra slots that offer four lanes, though these tend to be quite costly.
Yes, it is clear that models that fit entirely into VRAM will be significantly quicker than those requiring CPU inference. And if your main task is just inference, then concentrating on ample VRAM seems to be the key priority, especially for models up to 70B and smaller. But if you aim to run very large models—such as Mixtral 8x22B, Miqu 120B, Command R+, etc.—without needing hundreds of GB of VRAM—then a strong CPU and fast RAM (preferably 8 channels) will enable these massive 100B+ models to operate at speeds of 3-10 tokens per second.
To address your query directly, consider the WRX90E-SAGE motherboard; it offers sufficient PCI slots for four GPUs, though TheStryker noted that four GPUs would need water cooling. It also provides eight DDR5 RAM slots, doubling the memory bandwidth compared to quad-channel boards.
I recently assembled a 2xRTX 4090 system on the WRX90E-SAGE, both of which are air cooled. If I plan to upgrade to four RTX 4090s later (once prices drop after the 5090 release), I’ll convert them to water-cooled units so all four can fit in the case. Choosing the WRX90E-SAGE would mean buying a Threadripper Pro CPU as well.