F5F Stay Refreshed Hardware Desktop Exploring SR-IOV capabilities on a Z390 processor

Exploring SR-IOV capabilities on a Z390 processor

Exploring SR-IOV capabilities on a Z390 processor

S
samrcar
Member
Find
137
03-19-2016, 04:58 PM
#1
I upgraded my Z370-based setup to an i9-9900K, letting my i7 8700K rest on the sidelines. That led me to design a fresh home server to supersede my i7-3930K. I spotted an Asus Prime Z390-P on discount from NewEgg. The product description read something unusual—“ASUS Prime Z390-P...... Intel Motherboard for Cryptocurrency Mining (BTC) with Above 4G Decoding...”—but the value was appealing, and it got me going. RAM was another need, so I found a DDR4-3200 kit on Amazon priced at $89. I hoped it wasn’t hype and picked it up, realizing it was significantly cheaper than usual (a hint). It came with 2x16GB, which felt like a bargain compared to the market rate. After installing it, I tested it thoroughly on Memtest86; all 32GB remained intact. Curious, I bought another unit, but it now costs $180. It seems I was fortunate—the seller had been listing 32GB units for half the price. (This happened just a few days after my initial order and after a couple of memory tests.)

Setting up the BIOS revealed SR-IOV as an option I wasn’t familiar with on consumer boards. For this project, I wanted to run multiple virtual machines on the server. I discovered that two BIOS slots offered IO-SRV settings—one global, one per detected device. In my case, a quad NIC appeared as four separate devices in the Advanced menu, each with SR-IOV enabled. I adjusted a few kernel parameters and configuration files. Suddenly, I had 28 virtual NICs ready to feed VMs directly, eliminating CPU overhead. This was a big win for performance, especially for network-intensive tasks.

Using virt-manager was a breeze; just swap the existing NIC with a PCI device (lspci->04:00.0) and enable SR-IOV. I also tweaked a few kernel settings and added mod-probe files. The result? A substantial CPU efficiency boost, perfect for handling heavy network workloads or benchmarking with tools like iperf. It’s a solid step forward in my server setup.
Reply
Reply
S
samrcar
03-19-2016, 04:58 PM #1

I upgraded my Z370-based setup to an i9-9900K, letting my i7 8700K rest on the sidelines. That led me to design a fresh home server to supersede my i7-3930K. I spotted an Asus Prime Z390-P on discount from NewEgg. The product description read something unusual—“ASUS Prime Z390-P...... Intel Motherboard for Cryptocurrency Mining (BTC) with Above 4G Decoding...”—but the value was appealing, and it got me going. RAM was another need, so I found a DDR4-3200 kit on Amazon priced at $89. I hoped it wasn’t hype and picked it up, realizing it was significantly cheaper than usual (a hint). It came with 2x16GB, which felt like a bargain compared to the market rate. After installing it, I tested it thoroughly on Memtest86; all 32GB remained intact. Curious, I bought another unit, but it now costs $180. It seems I was fortunate—the seller had been listing 32GB units for half the price. (This happened just a few days after my initial order and after a couple of memory tests.)

Setting up the BIOS revealed SR-IOV as an option I wasn’t familiar with on consumer boards. For this project, I wanted to run multiple virtual machines on the server. I discovered that two BIOS slots offered IO-SRV settings—one global, one per detected device. In my case, a quad NIC appeared as four separate devices in the Advanced menu, each with SR-IOV enabled. I adjusted a few kernel parameters and configuration files. Suddenly, I had 28 virtual NICs ready to feed VMs directly, eliminating CPU overhead. This was a big win for performance, especially for network-intensive tasks.

Using virt-manager was a breeze; just swap the existing NIC with a PCI device (lspci->04:00.0) and enable SR-IOV. I also tweaked a few kernel settings and added mod-probe files. The result? A substantial CPU efficiency boost, perfect for handling heavy network workloads or benchmarking with tools like iperf. It’s a solid step forward in my server setup.

Reply
Reply
T
TomTMR_NL
Junior Member
Find
42
03-19-2016, 04:58 PM
#2
Can I help you dismantle IOMMU groups? Otherwise, it seems you might need to go through eight distinct lines per VM.
Reply
Reply
T
TomTMR_NL
03-19-2016, 04:58 PM #2

Can I help you dismantle IOMMU groups? Otherwise, it seems you might need to go through eight distinct lines per VM.

Reply
Reply
O
ors0n
Member
Find
73
03-19-2016, 04:58 PM
#3
Breaking groups is feasible but hasn't been necessary. Every PCI device displays one member only. I confirmed this by removing a single VM, then adding two VMs across multiple networks using varied configurations. Identical Ports, distinct Ports—everything functions smoothly.
Reply
Reply
O
ors0n
03-19-2016, 04:58 PM #3

Breaking groups is feasible but hasn't been necessary. Every PCI device displays one member only. I confirmed this by removing a single VM, then adding two VMs across multiple networks using varied configurations. Identical Ports, distinct Ports—everything functions smoothly.

Reply
Reply