XenServer 5.6: Network problems with r8169 kernel module


Update 2010-10-21: Even with the new kernel module (r8168) the Hetzner monitoring reported short offline times now and then (about one ping failure every day). Compared to the old module (r8169) the problems occurred much less often. To examine the problem, I set up a cronjob to report the network status on the host system with a small script:

#!/bin/bash
echo -n $(date +%Y%m%d-%H%M)
echo -n "   `/sbin/ethtool eth0 | grep "Link detected"`"
if ! ping -c 1 -q google.de >/dev/null
then
        echo "   google ping: FAIL"
else
        echo "   google ping: OK"
fi

Surprisingly, this script did not report any network errors and since then also the Hetzner monitoring stopped reporting errors. In result my solution till this day is to use the module r8168 and to run my “network keep alive” script periodically (every minute).

Introduction

Recently, I noticed some network problems (package losts) with an almost fresh XenServer 5.6 installation on a Hetzner EQ root server running a (SSH sessions disconnects, host and client system are not pingable for some minutes). After some research I found some resources that point out problems with the r8169 kernel module:

Summarized the resources suggested to replace the driver r8169 with the r8168 driver to eliminate the network problems. Of course, the driver has to be replaced only in the host.

lspci and lsmod show that the root server has a Realtek network card and that the problematic r8169 driver is used:

# lspci
[...]
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
[...]

# lsmod | grep r81
r8169                  29108  0
mii                     5040  1 r8169

Replace r8169 module with r8168

As suggested, I replaced the r8169 driver with a new r8168 one. Unfortunately, there is no pre-compiled module for the XenServer 5.6. Here is a step-by-step introduction how to manually build the r8169 module and replace the r8168 module in the host system:

  1. Download XenServer’s DDK (Driver Development Kit) virtual machine from
    https://www.citrix.com/English/ss/downloads/details.asp?downloadId=2301285&productId=683148 (Additional Components -> XenServer Support Components)
  2. Copy the downloaded ISO to XenServer.
  3. Mount the ISO:
    # mount -o loop /PATH/TO/ISO/XenServer-5.6.0-DDK.iso /mnt/PATH
  4. Import the DDK virtual machine:
    # xe vm-import filename=/mnt/PATH/ddk/ova.xml
  5. Wait some seconds/minutes for the import job to be finished.
  6. In XenCenter add a network device to the newly created DDK virtual machine, start the machine and enter a new root password.
  7. Configure the network in the DDK VM; for a Hetzner EQ server this the typical settings are:
    • /etc/sysconfig/network-scripts/ifcfg-eth0
      DEVICE=eth0
      BOOTPROTO=static
      IPADDR=IP-ADRESS-OF-HOST-SYSTEM
      NETMASK=255.255.255.224
      ONBOOT=yes
      TYPE=ethernet
    • /etc/sysconfig/network-scripts/route-eth0
      IP-ADRESS-OF-HOST-SYSTEM dev eth0 scope link
      default via IP-ADRESS-OF-HOST-SYSTEM
    • /etc/resolv.conf
      nameserver 213.133.99.99
      nameserver 213.133.100.100
  8. Download the newest r8168 sources from ftp://WebUser:2mG8dBW@202.134.71.22/cn/nic/. I used the file r8168-8.018.00.tar.bz2 (49 KB 01.04.2010 09:59:00) (Download link: ftp://WebUser:2mG8dBW@202.134.71.22/cn/nic/r8168-8.018.00.tar.bz2)
  9. Unpack the sources and build the module (this creates the file r8168-8.018.00/src/r8168.ko):
    # tar vjxf r8168-8.018.00.tar.bz2
    # cd r8168-8.018.00/
    # make all
  10. Copy the created r8168.ko to /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8168.ko in the host system and set permissions to 0744.
  11. Rename the old r8169 module to prevent loading:
    # mv /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8169.ko /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8169.backup.ko
    
  12. Edit /etc/modprobe.conf:
    alias eth0 r8168
    alias r8169 off
    
  13. Unload the old module, load the new one and reboot the system (use one line for all commands, because rmmod will disconnect the server from the network):
    # rmmod r8169 && depmod -a && modprobe r8168 && reboot
    

After the system is again up and running again, lsmod should no longer list the r8169 module:

#  lsmod | grep r81
r8168                  95780  0

Additional information


9 responses to “XenServer 5.6: Network problems with r8169 kernel module”

  1. Hi Jan,
    thanks for the post,
    but after the procedure above with .019 release of the driver I retrieve

    r8169 29108 0
    mii 5040 1 r8169
    r8168 95780 0

    for lsmod | grep r81
    how can I make sure that only the r8168 driver is loaded
    I am on XenServer 5.5

    Thanks,
    RTG

  2. Hi,

    My host is unable to connect to the Internet so I have tried to compile it through rescue system using chroot but Xen Server has no make. I have also tried your compiled .ko module but insmod says that it has a bad format.

    My server is EQ6 and I just don’t have any idea how to fix it 🙁 Pleas help.

    Thanks 🙂

  3. Hi Igor, you need download the preconfigured XenServer’s DDK (Driver Development Kit) virtual machine from Citrix to compile the driver. This VM contains all required development tools (like make) to build drivers (see Step 1 under the section “Replace r8169 module with r8168”). Hope this helps …

  4. I’m getting the no symbol version for module_layout error in dmesg, everytime I try the modprobe I get the Invalid module format FATAL, so I can’t run the module so far.

    I compiled it under the VM in a XenServer 6.0 machine (I need it for a Xen Server 6.1 machine), but I don’t think that was a problem.

    I also tried your module, same luck

    I’m gonna googe a bit more, and then I’m gonna install a 5.6.0 version and let you know.

    If anything comes to your mind, please, tell me.

    Thanks.

  5. Dear all,

    try to add option under your realtek alias into modprobe.conf.

    [root@xen ~]# Vi /Etc/modprobe.conf
    options r8169 use_dac=1

    write & save (:wq)

    Regards
    smtx

  6. You completed some good points there. I did a search on the theme and found the majority of persons will have the same opinion with your blog.

  7. Thanks for finally writing about >XenServer 5.6: Network problems with
    r8169 kernel module | Jan Jonas’ blog <Loved it!

Leave a Reply

Your email address will not be published. Required fields are marked *