Problem with achieving 10Gbps speed.
Problem with achieving 10Gbps speed.
Hello everyone, I followed the 40Gbit networking guide and felt satisfied with the results. It’s been functioning well for a while now, but I decided to switch things up by trying Ubuntu. I installed the newest Mellanox OFED drivers (4.4-2.0.7) and everything worked smoothly. The other machine has an IP address of 172.16.44.2, a netmask of 255.255.255.0, and a gateway of 172.16.44.1. I adjusted the network settings on both devices—changing the addresses to 172.16.44.1, updating the netmask to 255.255.255.0, and switching the gateway to 172.16.44.2. After a short while, it connected successfully and I could ping the IP without issues. However, when I tried to access it via a browser or add it to a map, it kept refusing to connect. What’s happening? I expected it to be straightforward like on Windows. What further steps should I take to resolve this?
Tell me about your connection setup. Are you using a direct link between the two devices or does a switch play a role? Also, let me know the type of cables and network cards you're employing.
You're trying to connect to a Samba network you set up on Ubuntu. If the target machine can't reach any services, that's why browser or drive access fails. Check if both devices can ping each other—successful round-trips mean no firewall blocking.
Before I moved to Windows, I managed to get this working with FreeNAS. If I recall right, the main change was switching from Windows to Ubuntu. Now I can ping from Ubuntu without packet loss. It seems the issue isn't related to a firewall since UFW isn't active. When trying to reach webgui via the IP, it keeps failing or disconnecting unexpectedly.
So...I'm not really sure what your ultimate goal or objective is other than trying to get the two systems to talk to each other. If you're using Ubuntu, try this: (DISCLAIMER, I'm using Mellanox ConnectX-4 4x EDR Infiniband cards (100 Gbps), so there might be some differences.) 1) It would be more beneficial to google how to enable the root user in Ubuntu because by default, root is disabled. You can't su to root in Ubuntu (by default) but it would be EXTERMELY beneficial to do so so that you can just fire off the commands in rapid succession rather than having to do "sudo sh -c 'command'" and HOPE that that'll work (which sometimes it does, and sometimes it doesn't.) Once you've enabled the root user (and give it a password) in Ubuntu, su to root. 2) Since I don't really know your setup, here's what I would do (in order): (run these commands and look for the following) # mst start # mlxfwmanager (to get the PCI device name because you might need it later) # hca_self_test.ofed You should look to see if there are any fails in the output on both systems. You should check to see that the ports are up and that a link has been established. If there isn't, then we'll come back to this a little later. # ibv_devinfo (Again, there's a real possibility that this command may not work for you if you don't have an Infiniband device, so it might take a little bit of research/googling to find out what would be the synonymous command.) # ibdev2netdev This should show if you have an IP assigned to your Mellanox device/port. It should show that port as being "up" if it is working. # ip link show ib0 # mlxconfig -d <<PCI device name>> --query (e.g. # mlxconfig -d /dev/mst/mt4115_pciconf0 --query) # mlxfwmanager -d <<PCI device name>> --query So this is just some basic diagnostics on or about the card/ports themselves. Any where there's an anomaly here - and you'll have to start digging deeper. Much deeper. The other thing that I would suggest is that I would set the IP address for the gateway as something other than the IP address of the other machine. If I understood you correctly, your network config probably looks something like this: machine1: /etc/sysconfig/network/ifcfg-eth0 BOOTPROTO='static' BROADCAST='' ETHTOOL_OPTIONS='' IPADDR='172.16.44.1/24' /etc/sysconfig/network/ifroute-eth0 default 172.16.44.2 machine2: /etc/sysconfig/network/ifcfg-eth0 BOOTPROTO='static' BROADCAST='' ETHTOOL_OPTIONS='' IPADDR='172.16.44.2/24 /etc/sysconfig/network/ifroute-eth0 default 172.16.44.1 right? Something like that? I would suggest changing the gateway to something like 172.16.44.100 on both machines rather than pointing the gateways towards each other. That's probably a part of your problem. and then you can try and ping each other to see if it will work. (This is how I have my machines set up. I have four compute nodes, all with Mellanox ConnectX-4 dual port VPI 4x EDR IB 100 Gbps cards and I have set them all up with a common gateway IPv4 address but each card/port has their own address and this is working for me.) Also, in regards to opening a browser and being able to connect to the machine - there is no indication in your original question that you have some kind of webserver running (unless you're talking about FreeNAS, which I don't have any experience with it). But try this first and see. I'm somewhat surprised that this would have worked in Windows as well because with it having different gateways, it shouldn't have been able to see each other since it might think that they might not belong on the same network. Not really sure about that one. But you can try this and see what happens.