Inserting a PCIe card causes ethernet to stop working

amihart

New Member
Joined
Jul 13, 2021
Messages
2
Reaction score
0
Credits
31
I am having a strange issue, wonder if anyone has any ideas of what I could do to troubleshoot. I have a computer setup as a server and it is currently running Debian and connected to the ethernet and has a single PCIe slot. It has been working great until recently, I bought an NVMe PCIe card to let me use those SSDs.

Weird thing is, if I power off the machine, plug in the card, then boot the machine, I can no longer connect to the internet. It is not a WiFi problem as I have a direct connection to the internet over ethernet. If I power off the machine, remove the card, and boot it again, internet connection works just great. I am 100000000% certain that inserting the card is what is causing the internet to go down, I have tried dozens of times booting with the card in and without it in and the connection is always lost when it is not in and there when it is in. The router is fine as well as I have other devices connected to the router and they all have internet connectivity.

I don't know too much about this kinda stuff, my first thought was to run "ip a".

Here is "ip a" with the card removed.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp2s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 20:6a:8a:6b:1a:26 brd ff:ff:ff:ff:ff:ff inet 192.168.3.3/24 brd 192.168.3.255 scope global dynamic enp2s0f0 valid_lft 85451sec preferred_lft 85451sec inet6 fe80::226a:8aff:fe6b:1a26/64 scope link valid_lft forever preferred_lft forever

Here is "ip a" with the card inserted initially.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp3s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 20:6a:8a:6b:1a:26 brd ff:ff:ff:ff:ff:ff

I say "initially" because I did try to fix it myself and made changed. I noticed that for some reason enp2s0f0 is renamed to enp3s0f0, so I opened up /network/interfaces and copied the interface setup of enp2s0f0 to enp3s0f0.

# The loopback network interface auto lo iface lo inet loopback # The primary network interface allow-hotplug enp2s0f0 iface enp2s0f0 inet dhcp # I made this one allow-hotplug enp3s0f0 iface enp3s0f0 inet dhcp

I then tried rebooting and now my "ip a" output looks like this with the card in.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 20:6a:8a:6b:1a:26 brd ff:ff:ff:ff:ff:ff inet 169.254.4.230/16 brd 169.254.255.255 scope link enp3s0f0:avahi valid_lft forever preferred_lft forever inet6 fe80::226a:8aff:fe6b:1a26/64 scope link valid_lft forever preferred_lft forever

It looks... normal to me now? But still doesn't work.

Any ideas?
 


Might want to check your BIOS settings for the PCIe slot and make sure they are compatible
 
Might want to check your BIOS settings for the PCIe slot and make sure they are compatible
I'm not sure why there would be a compatibility issue for the slot since it works perfectly? The issue is not the NVMe card in the PCIe slot, it works perfectly and I can use it no problem. Both the PCIe slot and the ethernet work perfectly individually. Pretty sure this is a Linux issue and not an issue with the hardware or the BIOS. As I showed with "ip a" Linux is rearranging something as a response to an additional card being added.
 
inet 169.254.4.230/16 brd 169.254.255.255 scope link enp3s0f0:avahi
Try disabling avahi and then reboot.
Code:
systemctl disable avahi-daemon.service
If that doesn't work configure a static ip instead of a dynamic one and see if you are able to get a network connection then, if so you will then only need to figure out what is causing dhcp to not work properly.
 
I mostly run Linux on Enterprise hardware these days, (I don't run it on the desktop anymore) but I will offer some places to look.

  • First, I would look at the output of lspci to ensure you do not see any issues.
  • You can monitor your interrupts too by watching "watch -n1 cat /proc/interrupts" Look at your network adapters and see if you have interrupts happening while it's not working. The new pci card you installed too.
  • Check your boot messages and tail system messages
  • Backup your grub config and then update the grub cmdline to say: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pci=noaer" This enabled advanced error reporting.
  • You can also disable MSI in case the hardware vendor goofed up the implementation with GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pci=nomsi".
Obviously, after you make the changes to grub configs, you need to activate them with update-grub.

There are other pci troubleshooting things out there also.

Also, I would not dismiss hardware conflicting issues so quickly. (both pci and bios related)

Good luck,
Dave
 

Members online


Top