F5F Stay Refreshed Power Users Networks Have a 10GbE issue? Let me assist you.

Have a 10GbE issue? Let me assist you.

Have a 10GbE issue? Let me assist you.

Pages (2): 1 2 Next
J
JoloYolo
Member
Find
205
05-03-2018, 07:12 PM
#1
Client setup: Asus Rampage IV Extreme. i-7900K, 16GB RAM, 1080ti, 960evo SSD. Windows 10 Server: Zenith Extreme II, Treadripper 2950X, 64GB ECC RAM, 780ti, 2x raidz2 of 12x 8TB drives, 970evo 1TB SSD. Ubuntu 20 Each machine has its own 1GbE connection to the internet and P2P over 10GbE directly between them. iperf shows stable connections from PC1 to the server at port 5201.
Reply
Reply
J
JoloYolo
05-03-2018, 07:12 PM #1

Client setup: Asus Rampage IV Extreme. i-7900K, 16GB RAM, 1080ti, 960evo SSD. Windows 10 Server: Zenith Extreme II, Treadripper 2950X, 64GB ECC RAM, 780ti, 2x raidz2 of 12x 8TB drives, 970evo 1TB SSD. Ubuntu 20 Each machine has its own 1GbE connection to the internet and P2P over 10GbE directly between them. iperf shows stable connections from PC1 to the server at port 5201.

Reply
Reply
J
JR_GAMER07
Posting Freak
Find
915
05-11-2018, 04:36 PM
#2
This leads me to believe the Ubuntu box isn't utilizing jumbo packets. Verify that the MTU matches on both sides.
Reply
Reply
J
JR_GAMER07
05-11-2018, 04:36 PM #2

This leads me to believe the Ubuntu box isn't utilizing jumbo packets. Verify that the MTU matches on both sides.

Reply
Reply
M
MikeDragon159
Senior Member
Find
661
05-11-2018, 09:53 PM
#3
Establishing link to host 10.2.2.1 on port 5201 [4] local 10.2.2.3 on port 57686 [4] connected to 10.2.2.1 via port 5201 [ID]. Interval Transfer Bandwidth [4]: 0.00-1.01 sec 239 MBytes 1.99 Gbits/sec [4] 1.01-2.01 sec 0.00 Bytes 0.00 bits/sec [4] 2.01-3.00 sec 0.00 Bytes 0.00 bits/sec [4] 3.00-4.01 sec 0.00 Bytes 0.00 bits/sec [4] 4.01-5.01 sec 0.00 Bytes 0.00 bits/sec [4] 5.01-6.01 sec 0.00 Bytes 0.00 bits/sec [4] 6.01-7.01 sec 0.00 Bytes 0.00 bits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 bits/sec [4] 8.01-9.01 sec 0.00 Bytes 0.00 bits/sec now 9.01-10.01 sec 0.00 Bytes - - - - - - - - - - - - - - - - - - - - - - [ID] Interval Transfer Bandwidth [4]: 0.00-10.01 sec 239 MBytes 200 Mbits/sec sender [4]: 0.00-10.01 sec 239 MBytes receiver iperf Done. Changed MTU to 9000 (continues resetting on reboot) – performance degraded
Reply
Reply
M
MikeDragon159
05-11-2018, 09:53 PM #3

Establishing link to host 10.2.2.1 on port 5201 [4] local 10.2.2.3 on port 57686 [4] connected to 10.2.2.1 via port 5201 [ID]. Interval Transfer Bandwidth [4]: 0.00-1.01 sec 239 MBytes 1.99 Gbits/sec [4] 1.01-2.01 sec 0.00 Bytes 0.00 bits/sec [4] 2.01-3.00 sec 0.00 Bytes 0.00 bits/sec [4] 3.00-4.01 sec 0.00 Bytes 0.00 bits/sec [4] 4.01-5.01 sec 0.00 Bytes 0.00 bits/sec [4] 5.01-6.01 sec 0.00 Bytes 0.00 bits/sec [4] 6.01-7.01 sec 0.00 Bytes 0.00 bits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 bits/sec [4] 8.01-9.01 sec 0.00 Bytes 0.00 bits/sec now 9.01-10.01 sec 0.00 Bytes - - - - - - - - - - - - - - - - - - - - - - [ID] Interval Transfer Bandwidth [4]: 0.00-10.01 sec 239 MBytes 200 Mbits/sec sender [4]: 0.00-10.01 sec 239 MBytes receiver iperf Done. Changed MTU to 9000 (continues resetting on reboot) – performance degraded

Reply
Reply
M
muppet3197
Junior Member
Find
4
05-13-2018, 08:11 AM
#4
If you set MTU to 1500 on both, you should be able to reach 9+Gibs with that configuration.
Reply
Reply
M
muppet3197
05-13-2018, 08:11 AM #4

If you set MTU to 1500 on both, you should be able to reach 9+Gibs with that configuration.

Reply
Reply
G
Gilberto_
Member
Find
51
05-13-2018, 04:13 PM
#5
Establishing link to server 10.2.2.1, port 5201 [4] local 10.2.2.3 port 50555 linked to 10.2.2.1 port 5201 [ID] Time Range Transfer Speed [4] 0.00-1.00 sec 162 MBytes 1.36 Gbits/sec [4] 1.00-2.00 sec 157 MBytes 1.32 Gbits/sec [4] 2.00-3.00 sec 218 MBytes 1.82 Gbits/sec [4] 3.00-4.01 sec 141 MBytes 1.17 Gbits/sec [4] 4.01-5.00 sec 59.8 MBytes 508 Mbits/sec [4] 5.00-6.00 sec 240 MBytes 2.01 Gbits/sec [4] 6.00-7.01 sec 78.9 MBytes 653 Mbits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 bits/sec [4] 8.01-9.00 sec 165 MBytes 1.39 Gbits/sec [4] 9.00-10.00 sec 206 MBytes 1.73 Gbits/sec - ] Interval Transfer Bandwidth [4] 0.00-10.00 sec 1.39 GBytes 1.20 Gbits/sec sender [4] 0.00-10.00 sec 1.39 GBytes 1.20 Gbits/sec receiver Jumbo Packet off on Windows and Ubuntu set to 1500. Still not great. Linking to host 10.2.2.1, port 5201 [4] local 10.2.2.3 port 57686 connected to 10.2.2.1 port 5201 [ID] Time Range Transfer Speed [4] 0.00-1.01 sec 239 MBytes 1.99 Gbits/sec [4] 1.01-2.01 sec 0.00 Bytes 0.00 bits/sec [4] 2.01-3.00 sec 0.00 Bytes 0.00 bits/sec [4] 3.00-4.01 sec 0.00 Bytes 0.00 bits/sec [4] 4.01-5.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 5.01-6.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 6.01-7.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 8.01-9.01 sec 0.00 Bytes 0.00 Bytes - ] Interval Transfer Bandwidth [4] 0.00-10.01 sec 239 MBytes 200 Mbits/sec sender [4] 0.00-10.01 sec 239 MBytes 200 Mbits/sec receiver iperf Done. Changed MTU to 9000 (continues resetting on restart) – it’s getting worse
Reply
Reply
G
Gilberto_
05-13-2018, 04:13 PM #5

Establishing link to server 10.2.2.1, port 5201 [4] local 10.2.2.3 port 50555 linked to 10.2.2.1 port 5201 [ID] Time Range Transfer Speed [4] 0.00-1.00 sec 162 MBytes 1.36 Gbits/sec [4] 1.00-2.00 sec 157 MBytes 1.32 Gbits/sec [4] 2.00-3.00 sec 218 MBytes 1.82 Gbits/sec [4] 3.00-4.01 sec 141 MBytes 1.17 Gbits/sec [4] 4.01-5.00 sec 59.8 MBytes 508 Mbits/sec [4] 5.00-6.00 sec 240 MBytes 2.01 Gbits/sec [4] 6.00-7.01 sec 78.9 MBytes 653 Mbits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 bits/sec [4] 8.01-9.00 sec 165 MBytes 1.39 Gbits/sec [4] 9.00-10.00 sec 206 MBytes 1.73 Gbits/sec - ] Interval Transfer Bandwidth [4] 0.00-10.00 sec 1.39 GBytes 1.20 Gbits/sec sender [4] 0.00-10.00 sec 1.39 GBytes 1.20 Gbits/sec receiver Jumbo Packet off on Windows and Ubuntu set to 1500. Still not great. Linking to host 10.2.2.1, port 5201 [4] local 10.2.2.3 port 57686 connected to 10.2.2.1 port 5201 [ID] Time Range Transfer Speed [4] 0.00-1.01 sec 239 MBytes 1.99 Gbits/sec [4] 1.01-2.01 sec 0.00 Bytes 0.00 bits/sec [4] 2.01-3.00 sec 0.00 Bytes 0.00 bits/sec [4] 3.00-4.01 sec 0.00 Bytes 0.00 bits/sec [4] 4.01-5.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 5.01-6.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 6.01-7.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 7.01-8.01 sec 0.00 Bytes 0.00 Bytes 0.00 bits/sec [4] 8.01-9.01 sec 0.00 Bytes 0.00 Bytes - ] Interval Transfer Bandwidth [4] 0.00-10.01 sec 239 MBytes 200 Mbits/sec sender [4] 0.00-10.01 sec 239 MBytes 200 Mbits/sec receiver iperf Done. Changed MTU to 9000 (continues resetting on restart) – it’s getting worse

Reply
Reply
P
Penguin_Shades
Member
Find
58
05-15-2018, 01:25 AM
#6
The size of the data being transmitted appears to be the issue I don’t understand.
Reply
Reply
P
Penguin_Shades
05-15-2018, 01:25 AM #6

The size of the data being transmitted appears to be the issue I don’t understand.

Reply
Reply
Y
Yodeuu
Member
Find
163
05-16-2018, 06:19 PM
#7
For the initial test, how many parallel streams are you employing? Which chips are you employing? Are they overheating? And what would happen if you switched the chips between systems?
Reply
Reply
Y
Yodeuu
05-16-2018, 06:19 PM #7

For the initial test, how many parallel streams are you employing? Which chips are you employing? Are they overheating? And what would happen if you switched the chips between systems?

Reply
Reply
N
nuke_master11
Junior Member
Find
36
05-17-2018, 01:06 AM
#8
The NICs are integrated into the motherboards. They don't run hot. Sorry for the confusion about parallel streams.
Reply
Reply
N
nuke_master11
05-17-2018, 01:06 AM #8

The NICs are integrated into the motherboards. They don't run hot. Sorry for the confusion about parallel streams.

Reply
Reply
T
TMHThomas
Junior Member
Find
20
05-17-2018, 01:14 AM
#9
The -P flag in iperf enables multiple streams simultaneously; however, your CPU might become constrained, limiting the performance of a single TCP connection.
Reply
Reply
T
TMHThomas
05-17-2018, 01:14 AM #9

The -P flag in iperf enables multiple streams simultaneously; however, your CPU might become constrained, limiting the performance of a single TCP connection.

Reply
Reply
L
lizzard89
Senior Member
Find
707
05-17-2018, 01:22 AM
#10
Connecting to host 10.2.2.1, port 5201 [ 4] local 10.2.2.3 port 60531 connected to 10.2.2.1 port 5201 [ 6] local 10.2.2.3 port 60532 connected to 10.2.2.1 port 5201 [ 8] local 10.2.2.3 port 60533 connected to 10.2.2.1 port 5201 [ 10] local 10.2.2.3 port 60534 connected to 10.2.2.1 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 67.0 MBytes 562 Mbits/sec [ 6] 0.00-1.00 sec 148 MBytes 1.24 Gbits/sec [ 8] 0.00-1.00 sec 149 MBytes 1.25 Gbits/sec [ 10] 0.00-1.00 sec 94.2 MBytes 790 Mbits/sec [SUM] 0.00-1.00 sec 458 MBytes 3.84 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 104 MBytes 868 Mbits/sec [ 6] 1.00-2.00 sec 135 MBytes 1.13 Gbits/sec [ 8] 1.00-2.00 sec 72.6 MBytes 609 Mbits/sec [ 10] 1.00-2.00 sec 60.9 MBytes 511 Mbits/sec [SUM] 1.00-2.00 sec 372 MBytes 3.12 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 156 MBytes 1.30 Gbits/sec [ 6] 2.00-3.00 sec 97.4 MBytes 817 Mbits/sec [ 8] 2.00-3.00 sec 17.2 MBytes 145 Mbits/sec [ 10] 2.00-3.00 sec 74.9 MBytes 628 Mbits/sec [SUM] 2.00-3.00 sec 345 MBytes 2.89 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 78.1 MBytes 655 Mbits/sec [ 6] 3.00-4.00 sec 114 MBytes 959 Mbits/sec [ 8] 3.00-4.00 sec 157 MBytes 1.32 Gbits/sec [ 10] 3.00-4.00 sec 138 MBytes 1.16 Gbits/sec [SUM] 3.00-4.00 sec 487 MBytes 4.09 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec [ 6] 4.00-5.00 sec 52.9 MBytes 444 Mbits/sec [ 8] 4.00-5.00 sec 49.1 MBytes 412 Mbits/sec [ 10] 4.00-5.00 sec 80.6 MBytes 676 Mbits/sec [SUM] 4.00-5.00 sec 184 MBytes 1.55 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec [ 6] 5.00-6.00 sec 71.4 MBytes 599 Mbits/sec [ 8] 5.00-6.00 sec 110 MBytes 925 Mbits/sec [ 10] 5.00-6.00 sec 83.1 MBytes 697 Mbits/sec [SUM] 5.00-6.00 sec 265 MBytes 2.22 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.01 sec 164 MBytes 1.37 Gbits/sec [ 6] 6.00-7.01 sec 25.4 MBytes 212 Mbits/sec [ 8] 6.00-7.01 sec 160 MBytes 1.33 Gbits/sec [ 10] 6.00-7.01 sec 8.12 MBytes 67.8 Mbits/sec [SUM] 6.00-7.01 sec 357 MBytes 2.98 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 7.01-8.00 sec 12.8 MBytes 108 Mbits/sec [ 6] 7.01-8.00 sec 83.8 MBytes 707 Mbits/sec [ 8] 7.01-8.00 sec 168 MBytes 1.42 Gbits/sec [ 10] 7.01-8.00 sec 90.0 MBytes 760 Mbits/sec [SUM] 7.01-8.00 sec 355 MBytes 2.99 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 8.00-9.00 sec 18.0 MBytes 151 Mbits/sec [ 6] 8.00-9.00 sec 52.1 MBytes 437 Mbits/sec [ 8] 8.00-9.00 sec 84.4 MBytes 708 Mbits/sec [ 10] 8.00-9.00 sec 26.5 MBytes 222 Mbits/sec [SUM] 8.00-9.00 sec 181 MBytes 1.52 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 9.00-10.00 sec 52.1 MBytes 437 Mbits/sec [ 6] 9.00-10.00 sec 101 MBytes 845 Mbits/sec [ 8] 9.00-10.00 sec 157 MBytes 1.32 Gbits/sec [ 10] 9.00-10.00 sec 78.0 MBytes 654 Mbits/sec [SUM] 9.00-10.00 sec 388 MBytes 3.25 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 653 MBytes 548 Mbits/sec sender [ 4] 0.00-10.00 sec 653 MBytes 547 Mbits/sec receiver [ 6] 0.00-10.00 sec 880 MBytes 739 Mbits/sec sender [ 6] 0.00-10.00 sec 880 MBytes 738 Mbits/sec receiver [ 8] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec sender [ 8] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec receiver [ 10] 0.00-10.00 sec 734 MBytes 616 Mbits/sec sender [ 10] 0.00-10.00 sec 734 MBytes 616 Mbits/sec receiver [SUM] 0.00-10.00 sec 3.31 GBytes 2.85 Gbits/sec sender [SUM] 0.00-10.00 sec 3.31 GBytes 2.85 Gbits/sec receiver Tried 4 streams better but what is stopping multiple streams on setup?
Reply
Reply
L
lizzard89
05-17-2018, 01:22 AM #10

Connecting to host 10.2.2.1, port 5201 [ 4] local 10.2.2.3 port 60531 connected to 10.2.2.1 port 5201 [ 6] local 10.2.2.3 port 60532 connected to 10.2.2.1 port 5201 [ 8] local 10.2.2.3 port 60533 connected to 10.2.2.1 port 5201 [ 10] local 10.2.2.3 port 60534 connected to 10.2.2.1 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 67.0 MBytes 562 Mbits/sec [ 6] 0.00-1.00 sec 148 MBytes 1.24 Gbits/sec [ 8] 0.00-1.00 sec 149 MBytes 1.25 Gbits/sec [ 10] 0.00-1.00 sec 94.2 MBytes 790 Mbits/sec [SUM] 0.00-1.00 sec 458 MBytes 3.84 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 1.00-2.00 sec 104 MBytes 868 Mbits/sec [ 6] 1.00-2.00 sec 135 MBytes 1.13 Gbits/sec [ 8] 1.00-2.00 sec 72.6 MBytes 609 Mbits/sec [ 10] 1.00-2.00 sec 60.9 MBytes 511 Mbits/sec [SUM] 1.00-2.00 sec 372 MBytes 3.12 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 2.00-3.00 sec 156 MBytes 1.30 Gbits/sec [ 6] 2.00-3.00 sec 97.4 MBytes 817 Mbits/sec [ 8] 2.00-3.00 sec 17.2 MBytes 145 Mbits/sec [ 10] 2.00-3.00 sec 74.9 MBytes 628 Mbits/sec [SUM] 2.00-3.00 sec 345 MBytes 2.89 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 3.00-4.00 sec 78.1 MBytes 655 Mbits/sec [ 6] 3.00-4.00 sec 114 MBytes 959 Mbits/sec [ 8] 3.00-4.00 sec 157 MBytes 1.32 Gbits/sec [ 10] 3.00-4.00 sec 138 MBytes 1.16 Gbits/sec [SUM] 3.00-4.00 sec 487 MBytes 4.09 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 4.00-5.00 sec 1.88 MBytes 15.7 Mbits/sec [ 6] 4.00-5.00 sec 52.9 MBytes 444 Mbits/sec [ 8] 4.00-5.00 sec 49.1 MBytes 412 Mbits/sec [ 10] 4.00-5.00 sec 80.6 MBytes 676 Mbits/sec [SUM] 4.00-5.00 sec 184 MBytes 1.55 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec [ 6] 5.00-6.00 sec 71.4 MBytes 599 Mbits/sec [ 8] 5.00-6.00 sec 110 MBytes 925 Mbits/sec [ 10] 5.00-6.00 sec 83.1 MBytes 697 Mbits/sec [SUM] 5.00-6.00 sec 265 MBytes 2.22 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 6.00-7.01 sec 164 MBytes 1.37 Gbits/sec [ 6] 6.00-7.01 sec 25.4 MBytes 212 Mbits/sec [ 8] 6.00-7.01 sec 160 MBytes 1.33 Gbits/sec [ 10] 6.00-7.01 sec 8.12 MBytes 67.8 Mbits/sec [SUM] 6.00-7.01 sec 357 MBytes 2.98 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 7.01-8.00 sec 12.8 MBytes 108 Mbits/sec [ 6] 7.01-8.00 sec 83.8 MBytes 707 Mbits/sec [ 8] 7.01-8.00 sec 168 MBytes 1.42 Gbits/sec [ 10] 7.01-8.00 sec 90.0 MBytes 760 Mbits/sec [SUM] 7.01-8.00 sec 355 MBytes 2.99 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 8.00-9.00 sec 18.0 MBytes 151 Mbits/sec [ 6] 8.00-9.00 sec 52.1 MBytes 437 Mbits/sec [ 8] 8.00-9.00 sec 84.4 MBytes 708 Mbits/sec [ 10] 8.00-9.00 sec 26.5 MBytes 222 Mbits/sec [SUM] 8.00-9.00 sec 181 MBytes 1.52 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 4] 9.00-10.00 sec 52.1 MBytes 437 Mbits/sec [ 6] 9.00-10.00 sec 101 MBytes 845 Mbits/sec [ 8] 9.00-10.00 sec 157 MBytes 1.32 Gbits/sec [ 10] 9.00-10.00 sec 78.0 MBytes 654 Mbits/sec [SUM] 9.00-10.00 sec 388 MBytes 3.25 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 653 MBytes 548 Mbits/sec sender [ 4] 0.00-10.00 sec 653 MBytes 547 Mbits/sec receiver [ 6] 0.00-10.00 sec 880 MBytes 739 Mbits/sec sender [ 6] 0.00-10.00 sec 880 MBytes 738 Mbits/sec receiver [ 8] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec sender [ 8] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec receiver [ 10] 0.00-10.00 sec 734 MBytes 616 Mbits/sec sender [ 10] 0.00-10.00 sec 734 MBytes 616 Mbits/sec receiver [SUM] 0.00-10.00 sec 3.31 GBytes 2.85 Gbits/sec sender [SUM] 0.00-10.00 sec 3.31 GBytes 2.85 Gbits/sec receiver Tried 4 streams better but what is stopping multiple streams on setup?

Reply
Reply
Pages (2): 1 2 Next