100gbe NAS-4 inexpensive option
100gbe NAS-4 inexpensive option
I'm considering two Melissa boxes and planning to use NAS4cheap. I suspect the PCI lane could limit performance on my i5-8400, so I'm looking into server-grade gear. The E5 line is the most affordable with high lane count. Would a 2650L (yes, quiet one) on an X99 board with four NVMe drives work with a Melissa ConnectX-4 100GbE? By "handle" you mean it shouldn't be limited by PCIe or CPU speed. I'm sure it can, but I thought 1920x might be better but it uses more power and isn't as cheap.
100gbps equals roughly 12.5 GB/s... this translates to 16 PCIe 3.0 lanes. The challenge lies in how the NVMe drives will be connected—most will come from the chipset, linked via a PCIe 4.0 interface (like DMI or Intel’s own variant). You’ll likely have one NVMe drive connected at PCIe 3.0 x4 (providing about 4 GB/s), while the remaining drives will be bottlenecked by that same connection. Certain motherboards, such as the Gigabyte B550 Aorus Master, can split those 16 lanes: eight go to the video card slot and the other eight are divided into two M.2 slots. This setup lets you use a budget video card in the lower lane section—plugging in a 10$ card that uses 2-4 lanes from the chipset would give you around 7.5 GB/s (8 lanes × 0.9 GB/s). You’ll need to verify the specific board layout to confirm where the lanes originate. Boards supporting 1920x resolution will have 48 lanes, and some offer bifurcation options for adapters that fit into PCIe slots. However, it’s unclear if a CPU can handle such high throughput regardless of configuration.
the latest linux whonnock I discovered focuses more on system architecture than consumer parts. I plan to run it on this desktop setup—a z790 master with all m.2 slots upgraded to gen4+, one gen5 dedicated to the CPU, and another for PCIe channels. The performance drops noticeably when switching from CPU to PCIe to PCIe again: sustained speeds fall from 3.4 GB/s to just 2 GB/s. CPU utilization seems to become a bigger concern now. It appears the PCIe path to PCIe is either limited by the gen4 M.2 slot or by the round-trip through DMI and PCIe. I hope the task manager includes a bus graph for better insight. Overall, this quick trial suggests consumer components won’t suffice; an older 2650Lv4 might perform better. From the x99 diagram, the maximum theoretical speed is around 16 GT/s. It also looks like two 16 GT/s PCIe lanes connect directly to the CPU without any bottleneck. If that’s accurate, the connection between the motherboard and PCIe should be straightforward—possibly using one port for MX and another for NVMe. I’m not sure if this BIOS supports a dual-bus configuration. At these speeds, the real limiting factor shifts up the chain. I’m curious whether servers use a transfer mode that skips the CPU or if the CPU acts as a fast switch. The 2560L offers a bus rate of roughly 2x9.6 GT/s, which equals about 7.5 GB/s in both directions simultaneously. The 1920x and 8400 deliver double that, while the 13600K reaches four times that (8x the 2560L). Yet the 1920x can still work in a file server, but not the 13600K—perhaps “bus rate” isn’t the best metric. It might depend on how many PCIe lanes are used and the actual path taken.