128 GB DDR4 compared to 96 GB DDR5 from Intel or AMD
128 GB DDR4 compared to 96 GB DDR5 from Intel or AMD
I’m evaluating three memory configurations for your deep learning/AI setup. You need more than 80 GB of RAM and want stable performance. The options are:
1. Two 48 GB DDR5 6000 MHz sticks (C30)
2. Four 32 GB DDR5 4800 MHz sticks (C30)
3. Four 32 GB DDR4 3600 MHz sticks (C16)
For bandwidth, the effective rate depends on the memory type and configuration. DDR4 typically offers lower bandwidth than DDR5, but with quad-channel configurations you can achieve higher aggregate throughput. The 4x32GB DDR5 sticks should provide strong bandwidth, possibly exceeding 5200–5600 Mbps depending on the profile. The 2x48GB DDR5 setup is likely to be faster than the 4x32GB option.
Regarding stability, all three meet your capacity needs, but DDR5 offers better reliability and future-proofing. Intel generally has an edge over AMD in high-end platforms, especially with newer AM5 chipsets, though performance gains are narrower.
If you’re considering upgrading to higher speeds like 7800X3D or 5800X3D, the bandwidth benefits would be significant, but the cost difference may not justify it unless you need extreme throughput. For now, the 5800X3D with new memory seems like the most balanced choice.
For ML applications, memory bandwidth isn't a big factor; you'll mainly hit limits through PCIe. This discussion could help: https://forum.level1techs.com/t/ddr5-4-d...not/197153 For consumer setups, there are no dual-channel options—only two DIMMs per channel. Bandwidth remains consistent regardless of stick count. Intel still doesn't offer AVX512 support, which is important for performance. X3D models won't help much either; focus on more processing cores instead. Your current build (5950x, 128GB RAM, two 3090s) works fine for you. Depending on your tasks—whether mostly training or heavy preprocessing—the CPU might not be critical. A 5700x would give similar gains to a 5800x3D. The 3D cache isn't useful in this scenario. Expect peak speeds around 5200–5600MHz.
Because of consumer platform restrictions: dual channel only applies. This means the performance gains from using four sticks won’t be realized. Overclocking isn’t achieved through XMP/EXPO settings, even if they activate automatically. If quad-channel support existed, you could multiply speed by the number of channels for a better estimate. For example, two 6000 MT/s channels at 64 bits would total 96 GB/s. Currently, 2 x 6000 MT/s equals 12,000 MT/s, which is less than 4 x 4800 MT/s (19,200 MT/s). However, with only two channels, 2 x 6000 MT/s × 64 bits = 768,000 Mbit/s ≈ 96 GB/s.
Eigenvektor confirmed, isn't DDR5 really quicker overall (unless you pay a lot) for handling whatever tasks they manage? (thanks to those MHz numbers)
It operates in dual channel mode but also supports single rank and dual rank configurations. The memory controller manages each rank separately, allowing commands to be sent between ranks during data transfers. With 32 GB or higher modules it’s likely dual rank sticks are used, maximizing performance. I wouldn’t choose 48 GB RAM sticks just because they’re large; they use 3 GB chips and some software might not work well with them. For high bandwidth needs, consider EPYC systems. This seller on eBay has a solid reputation on servethehome forums, offering many boards, CPUs, and complete setups, and they’re responsive to questions there. For instance, a $370 board pairs with the AMD Epyc 7281+ASRock EPYCD8, featuring 7 PCIe dual M2 SSDs. It supports up to 8 channels of DDR4 ECC, offering affordable performance. A CPU around 5600MHz gives strong base speed, but adding more RAM sticks can boost capacity further. Options range from CPUs and boards to mixed configurations, such as the same motherboard with a 7551P CPU paired with a 3200MHz RAM module. Prices vary—$150 for two 32 GB DDR4 modules or $100 for two 32 GB 2666 modules, depending on the model.
Thanks for your updates. Intel seems unlikely now—high power use plus the Q500L isn’t ideal. I’m handling two GPUs in a subpar setup. Right now I’m using Windows with WSL, but since gaming isn’t my focus, I’ll switch to Linux with the latest release. My needs include SD, SDXL inference & training, custom Graph Neural Network development (PyTorch, NetworkX, Scikit-learn), testing LLMs (large models), and local inference. Memory bandwidth is key for me, especially for LLM tasks. I’ve noticed that running these workloads on the GPU helps, but I won’t be able to load full models due to limited RAM. Training GNNs and SDXL are mostly GPU-bound, though CPU usage stays around 20-25% during hypernetwork or Lora training. My main constraint is memory bandwidth, not just compute power. I plan to upgrade to a 4090 when prices drop so I can build a more robust platform long-term. I’ll check the discussions about adding four RAM sticks and look into Threadripper/EPYC options.
Absolutely, for full CPU inference memory usage is a key constraint. Still, some models around 65-70B can fit into 24GB VRAM with reduced precision—typically under 4 bits—and that could be feasible. Expect performance in the range of 1–2 tokens per second on a CPU. If you're serious about this path, using an EPYC with multiple channels is ideal, even for older configurations. For other workloads, the GPU will play a much bigger role.