Update 2010-10-21: Even with the new kernel module (r8168) the Hetzner monitoring reported short offline times now and then (about one ping failure every day). Compared to the old module (r8169) the problems occurred much less often. To examine the problem, I set up a cronjob to report the network status on the host system with a small script:
#!/bin/bash
echo -n $(date +%Y%m%d-%H%M)
echo -n " `/sbin/ethtool eth0 | grep "Link detected"`"
if ! ping -c 1 -q google.de >/dev/null
then
echo " google ping: FAIL"
else
echo " google ping: OK"
fi
Surprisingly, this script did not report any network errors and since then also the Hetzner monitoring stopped reporting errors. In result my solution till this day is to use the module r8168 and to run my “network keep alive” script periodically (every minute).
Introduction
Recently, I noticed some network problems (package losts) with an almost fresh XenServer 5.6 installation on a Hetzner EQ root server running a (SSH sessions disconnects, host and client system are not pingable for some minutes). After some research I found some resources that point out problems with the r8169 kernel module:
- http://wiki.hetzner.de/index.php/Installation_des_r8168-Treibers (German)
- http://forum.hetzner.de/wbb2/thread.php?postid=138116#post138116 (German)
Summarized the resources suggested to replace the driver r8169 with the r8168 driver to eliminate the network problems. Of course, the driver has to be replaced only in the host.
lspci and lsmod show that the root server has a Realtek network card and that the problematic r8169 driver is used:
# lspci
[...]
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
[...]
# lsmod | grep r81
r8169 29108 0
mii 5040 1 r8169
Replace r8169 module with r8168
As suggested, I replaced the r8169 driver with a new r8168 one. Unfortunately, there is no pre-compiled module for the XenServer 5.6. Here is a step-by-step introduction how to manually build the r8169 module and replace the r8168 module in the host system:
- Download XenServer’s DDK (Driver Development Kit) virtual machine from
https://www.citrix.com/English/ss/downloads/details.asp?downloadId=2301285&productId=683148 (Additional Components -> XenServer Support Components) - Copy the downloaded ISO to XenServer.
- Mount the ISO:
# mount -o loop /PATH/TO/ISO/XenServer-5.6.0-DDK.iso /mnt/PATH
- Import the DDK virtual machine:
# xe vm-import filename=/mnt/PATH/ddk/ova.xml
- Wait some seconds/minutes for the import job to be finished.
- In XenCenter add a network device to the newly created DDK virtual machine, start the machine and enter a new root password.
- Configure the network in the DDK VM; for a Hetzner EQ server this the typical settings are:
- /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0 BOOTPROTO=static IPADDR=IP-ADRESS-OF-HOST-SYSTEM NETMASK=255.255.255.224 ONBOOT=yes TYPE=ethernet
- /etc/sysconfig/network-scripts/route-eth0
IP-ADRESS-OF-HOST-SYSTEM dev eth0 scope link default via IP-ADRESS-OF-HOST-SYSTEM
- /etc/resolv.conf
nameserver 213.133.99.99 nameserver 213.133.100.100
- /etc/sysconfig/network-scripts/ifcfg-eth0
- Download the newest r8168 sources from ftp://WebUser:2mG8dBW@202.134.71.22/cn/nic/. I used the file r8168-8.018.00.tar.bz2 (49 KB 01.04.2010 09:59:00) (Download link: ftp://WebUser:2mG8dBW@202.134.71.22/cn/nic/r8168-8.018.00.tar.bz2)
- Unpack the sources and build the module (this creates the file r8168-8.018.00/src/r8168.ko):
# tar vjxf r8168-8.018.00.tar.bz2 # cd r8168-8.018.00/ # make all
- Copy the created r8168.ko to /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8168.ko in the host system and set permissions to 0744.
- Rename the old r8169 module to prevent loading:
# mv /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8169.ko /lib/modules/2.6.27.42-0.1.1.xs5.6.0.44.111158xen/kernel/drivers/net/r8169.backup.ko
- Edit /etc/modprobe.conf:
alias eth0 r8168 alias r8169 off
- Unload the old module, load the new one and reboot the system (use one line for all commands, because rmmod will disconnect the server from the network):
# rmmod r8169 && depmod -a && modprobe r8168 && reboot
After the system is again up and running again, lsmod should no longer list the r8169 module:
# lsmod | grep r81
r8168 95780 0
Additional information
- I found some useful information in the howto “Compiling the Realtek RTL8111 driver on XenServer by Martin Guppy”.
- Here is a link to my post “XenServer 5.6: Netzwerkprobleme / Realtek 8169” (German) in the “Hetzner Online Diskussionsforum”. The user “drieschel” has attached a install script that can be used alternatively for step 10. to 13.
- The r8168.ko file I’ve built can be downloaded here: http://blog.janjonas.net/wp-content/uploads/2010/08/r8168-8.018.00.zip
9 responses to “XenServer 5.6: Network problems with r8169 kernel module”
Hi Jan,
thanks for the post,
but after the procedure above with .019 release of the driver I retrieve
r8169 29108 0
mii 5040 1 r8169
r8168 95780 0
for lsmod | grep r81
how can I make sure that only the r8168 driver is loaded
I am on XenServer 5.5
Thanks,
RTG
Hi, did you follow step 11 (Rename the old r8169 module to prevent loading)?
Hi,
My host is unable to connect to the Internet so I have tried to compile it through rescue system using chroot but Xen Server has no make. I have also tried your compiled .ko module but insmod says that it has a bad format.
My server is EQ6 and I just don’t have any idea how to fix it 🙁 Pleas help.
Thanks 🙂
Hi Igor, you need download the preconfigured XenServer’s DDK (Driver Development Kit) virtual machine from Citrix to compile the driver. This VM contains all required development tools (like make) to build drivers (see Step 1 under the section “Replace r8169 module with r8168”). Hope this helps …
I’m getting the no symbol version for module_layout error in dmesg, everytime I try the modprobe I get the Invalid module format FATAL, so I can’t run the module so far.
I compiled it under the VM in a XenServer 6.0 machine (I need it for a Xen Server 6.1 machine), but I don’t think that was a problem.
I also tried your module, same luck
I’m gonna googe a bit more, and then I’m gonna install a 5.6.0 version and let you know.
If anything comes to your mind, please, tell me.
Thanks.
Dear all,
try to add option under your realtek alias into modprobe.conf.
[root@xen ~]# Vi /Etc/modprobe.conf
options r8169 use_dac=1
write & save (:wq)
Regards
smtx
You completed some good points there. I did a search on the theme and found the majority of persons will have the same opinion with your blog.
Toսche. Solid arguments. Keeр up the good spirit.
Thanks for finally writing about >XenServer 5.6: Network problems with
r8169 kernel module | Jan Jonas’ blog <Loved it!