256GB ECC memory paired with storage and a RAID card is required (Intel compatible).
256GB ECC memory paired with storage and a RAID card is required (Intel compatible).
Hey guys! Memory I am shopping for server memory and I would like some advice. The server we are building needs large amounts of ddr4 memory, the mobo we have has 8 slots, Supermicro MBD-X12SPI-TF-O. Here are my constraints Size: 256GB (Our applications run memory heavy with lighter cpu usage) Price Points: $2k ($3k is probably our max, not sure at what point performance gained for every extra dollar starts to approach 0) Configuration: 8x32 or 4x64 ECC DDR4 Our application benefits from lower latencies more often than higher frequencies. However speeds lower than 2666 is not optimal either, however we acquired this info on various 12x Nanya dimms with 16GB per stick and 2Rx8 So not sure how this will work out. We didnt pick this manufacturer by the way, we were at the mercy of our server provider for that. However this new server will be custom built by us and probably sent to a equinix datacenter, so we have full flexibility on what to buy. And I was searching exclusively for samsung b die dimms so far, as I have had a good experience with them in the past and been told that b die's are the best. However I am open to other manufacturers if they provide better performance and reliability. This server is the core of our infrastructure and any potential for errors needs to be minimized. ======================================================================================================================================== Others specs so far: Xeon 5318N X12SPI-TF-O Mellanox ConnectX-6 Dx 3x980 PRO 1 TB (Not bought for this build, from inventory) 2x Kingston KC300 2 TB Highpoint 4 M.2 NVME RAID Controller (Not bought for this build, can replace) Everything above is already purchased. Below is what I mgiht get in addition to this ======================================================================================================================================== Storage Now for storage! Reliability is obviously important, however since most of our applications are pretty light we will probably not need more than 3 TB of space. The server will be writing logs for a ton of applications very quickly. We have two OEM (PM981 1TB) 980 pros running in software raid 0 (Server hasent rebooted in 700 days its a disaster waiting to happen). We would need something equivelent to that or better. I was thinking 4x 980 PRO's with a raid card but I see sk hynix and others have new nvme ssds that have passed the 980 pro. And since we were restricted by host to this choice, I dont know how well the 980 pro performs for sustained writes. SSD Constraints (Very rough): Min 3 TB Total RAID Redundency required (I was thinking raid 10?) Read faster than 2x980 pro's in raid 0 Write faster than 2x 980 pro's in raid 0 Low latency $2000 budget including the raid card (Does not include anything already purchased) I have 3x 980 Pro 1TB unopened from a build I was going to do a couple months ago. I was going to use them but like I said, the market seems to have better options now. I was previously set on a p5800x from intel since thats whats in my pc at home and it is a level above everything else, I realized I would not have redundency anymore unless I got 2 of them, which is a bank breaker. I am also open to having two sets of drives, OS+Regular App Data Drives Data wont be written too often besides for updating/deploying new apps. However must be faster than the 2x980 pro in raid 0 High Read speeds required Low latency required High Random R+W speeds Redundency needed Log Drives High Sustained Write Speeds (Ram cache will fill up) High Read speeds in bursts (Reading huge log file) ======================================================================================================================================== Raid Card Lastly, I need a raid card recommendation for this system as well! I am not sure how the highpoint raid card performs, I got it for my personal pc and never used it. If I decide to go for RAID 10 with 4x 1TB 980 Pros and RAID 0 with 2x KC300 what hardware raid controller would work best? I will only have 1 16x slot open as the mellanox nic and nvidia gpu will take up the other 2. Would I need 2 raid cards (or would it work just be worse with 1?) for the aformentioned setup? I definetely cant use the Hightop card if I go for 2 sets of drives as it only has 4 slots Misc: Chassis and cooling recommendations are welcome! Thanks in advance guys!
regarding the RAM, you've already addressed your own concerns: - 2666MHz memory (based on my recent search for DDR4 ECC... don't waste time chasing higher speeds if you're not requiring them.) - if you require extra space, opt for 4x64, otherwise stick with 8x32. - for the rest of the setup, aim to keep as close as possible to the QVL list. concerning storage, do you prioritize reliability—like being unable to simply replace drives—or fault tolerance across the array? these are quite different approaches. as for the RAID card: do you truly need a hardware solution? if you plan to use M.2 drives, I’m leaning toward a bifurcation card that directly integrates four slots into the PCIe lanes, letting software handle the rest. this gives you greater flexibility and control. regarding writes, do you have specific bandwidth requirements and data volumes in mind? with those numbers, it becomes straightforward to calculate how long each drive will last under your conditions.
For storage one other option is 2.5in ssds. How about something like a Micron 7400 MAX. Should be able to get 2 of those for sub 2k easily, and then you can have it hot swap too. These drives generlly do very well with sustained performance. re you running vms? Id normally go with pair of slow raid 1 drives for boot and os, then those 2.5in ssds for. dat, and log(maybe get 4 of em). How big do you need your log drive? Id really suggest getting the supermicro case built for this board, or getting the dell/hpe/lenovo eqv if you can. There much nicer hardware to work with. THere really isn't hardware raid card for nvme drives that I've seen. They all seem to be pcie switches using software raid. So Id just get the switches and use standard software raid like zfs or storage spaces.