Pesky Udev Rules

Have you ever replaced a network interface (NIC) or performed a chassis swap? If you are an administrator or IT professional the chances are good that at some point or another you have had to perform one of these.

Wait! "What's this!?" You say to yourself as you think your just about to get out of the woods. No networking? On a remote system this ultimately means one of two things. If you have IPMI your day can probably be saved assuming you do not in fact have a hardware or cabling problem. If however you are in a situation where IPMI,ILO, IKVM or DRAC is not available you're going to have to put in yet another datacenter service ticket and wait it out.

The IPMI Route

If you have IPMI on your system go ahead and fire up your favorite browser. Now head on over to the address connected to dedicated management interface. Login and select Console Redirection or any of the other available options that may provide you with Console Access. Login as your normal user or super-user on the system and take a look at the output of ifconfig -a or ip a. The problem should become immediately obvious, eth0 seems to be missing and your network doesn't provide DHCP services so the system does not correctly setup its routes nor set an IP to the interface let alone bring the interface up successfully.

[root@server ~]# ifconfig -a
eth1      Link encap:Ethernet  HWaddr 5b:72:32:4f:92:c8
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          LOOPBACK  MTU:16436  Metric:1
          RX packets:37208558 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37208558 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3554697746 (3.3 GiB)  TX bytes:3554697746 (3.3 GiB)

The fix

The fix to this problem is fairly simple, in most cases Udev will regenerate the rules on its own during boot unless the file is write protected for some reason. That might sound a little crazy but there are instances in which write protecting the rules might be favorable. In some cases Udev will simply, "get it wrong," if there are old configurations still in place for the 70-persistent-net rules.

1. Bring down networking
2. Remove and backup /etc/udev/rules.d/70-persistent-net.rules

Then run one of the following commands which should create a new rule set.

1. udevadm trigger
2. udevadm trigger --action=add
3. Check the new rules to verify that it has indeed changed.
4. Bring networking back up and you should be back in business.

[root@server ~]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 5b:72:32:4f:92:c8
          inet addr:192.168.88.247  Bcast:192.168.88.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:61872910 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32448302 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:81114825423 (75.5 GiB)  TX bytes:2958545446 (2.7 GiB)
          Interrupt:55 Base address:0xe000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          LOOPBACK  MTU:16436  Metric:1
          RX packets:37208558 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37208558 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3554697746 (3.3 GiB)  TX bytes:3554697746 (3.3 GiB)