A large L4 DRAM cache could boost CPU performance by reducing access delays and increasing data availability.
A large L4 DRAM cache could boost CPU performance by reducing access delays and increasing data availability.
Envision CPUs featuring 4-8GB of extremely rapid DDR5 memory at speeds between 10,000-12,000 MT/s, similar to AMD and Ryzen designs. The Ryzen architecture uses distinct chiplets with an older, budget-friendly IO die, while the cutting-edge ultra-fast logic CCD is built on the latest node. This imagined L4 cache DRAM chiplet could potentially enhance performance. A mere 4GB of DRAM in this context would be significant, not by memory capacity but by cache value. Adding such a small amount shouldn't be costly. It might marginally boost low-frequency and slightly lower latency performance. Edited January 17, 2025 by CatTNT
L4 would assist, though DRAM seems unusual for this approach. DDR5 also tends to be slow. Cache performance hinges on latency—DRAM is notably slower than L3, only acceptable if it's below L3. HBM offers even less benefit due to higher latency. eDRAM exists and was integrated into Haswell as a L4 cache.
Placing 4GB of DRAM isn't done lightly; it would require a huge cost rise that might be better allocated elsewhere. The SC example uses just 960MB of eDRAM, with each PU supporting a 12-core CPU, and the CPU already occupies 700mm²—where will the eDRAM fit?
An L4 could provide major advantages for a mobile CPU by serving as both an iGPU cache to boost graphics performance and holding compact LLMs to enhance NPU efficiency. The drawback is that poor cache hit or miss rates might lead to higher latency compared to simply moving from L3 to DIMM initially.
I made a mistake in my description. I actually wanted to include a 4-8GB ultra-fast DRAM chiplet or tile, not just on the die itself. I understand how precious die space is. Right now AMD uses one IO die and one CCD for that setup, which would require one more die, one CCD, and a DRAM cache chip. This was my original idea, and I’ll revise it later. Alternatively, they might adopt a similar approach like X3D, placing some DRAM directly beneath or above the logic die. Essentially, I’m talking about positioning a significantly large amount of fast memory near the CPU, offering much lower cost than traditional L1/L2/L3 caches.
This issue has been on my mind for a while, but the advantages might not justify the expense. I'm hoping AMD would adopt some form of L4 processing integrated into the IOD on 2 CCD-based CPUs. At present, the two CCDs seem to be in separate sockets, making it difficult to exchange large amounts of data efficiently. IFOP's bandwidth is constrained, and RAM is similarly restricted. Implementing L4 would eliminate the need to transfer data to RAM and provide a unified data source for packages. Still, IFOP must be upgraded to a much faster solution to truly function effectively and make two CCD CPUs act as a single, cohesive unit.
I personally own a Broadwell-C i7 5775C with 128MB L4 cache. The challenge isn't just about latency; bandwidth is equally important. It depends on the specific demands of the software you're using. The L4 cache in that model was advertised at 50GB/s. Back then, DDR3-1600 RAM officially supported up to 25GB/s in dual-channel mode. I didn’t focus much on latency, but I noticed performance differences. When DDR3 speeds surpassed that, latency for L4 dropped to around 40ns versus roughly 60+ ns for unoverclocked RAM at the time. By the end of DDR3’s era, speeds exceeded 50% more than those figures.
In my environment, bandwidth was consistently a bottleneck, especially with only RAM running under Windows. Comparisons with other systems using Haswell and Skylake showed similar limitations. L4, however, transformed the system into a nearly limitless bandwidth scenario, scaling smoothly across multiple cores. Haswell and Skylake could handle unlimited performance up to two cores, but only slightly beyond that, and four cores barely improved speed compared to three. I could install just one stick of RAM in Broadwell and achieve better results than with Haswell/Skylake!
I considered this too. The cache is quite limited, yet boosting its size would make a big difference. Adding a quicker intermediate cache could be more effective than relying solely on RAM.