Troubleshooting KVM Virtualization Problem With Log Files

There are various log files stored on the host system to assist with debugging KVM related problems. In this tutorial, I will cover log file locations and tools used to track down KVM problems.

=> $HOME/.virtinst/virt-install.log - virt-install tool log file.
=> $HOME/.virt-manager/virt-manager.log - virt-manager tool log file.
=> /var/log/libvirt/qemu/ - Log files for each running virtual machine. If centos is virtual machine name, than log file is /var/log/libvirt/qemu/centos.log.

You can use the grep and other Linux tools to view this files:
# tail -f /var/log/libvirt/qemu/freebsd.log
# grep something $HOME/.virtinst/virt-install.log

Connecting To Console

Use the virsh command to connect to guest serial console as follows :
# virsh list
# virsh console freebsd
This is useful to troubleshoot problem such as networking and much more from host itself. However, you need to configure guest operating system for a serial console. See how to setup a serial console for FreeBSD and CentOS virtual machines.

KVM Configuration Files

You can edit configuration file to define hardware properties for VMs or other setup. They are located in /etc/libvirt/qemu/ directory. Here is sample FreeBSD guest configuration file.
less /etc/libvirt/qemu/freebsd.xml
OR
virsh dumpxml freebsd
Sample outputs:

<domain type='kvm'>
  <name>freebsd</name>
  <uuid>6b7f44df-b67a-b1e1-0f9a-40c9ad760b0a</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel5.4.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
<pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' cache='none'/>
      <source file='/nfs/freebsd73.img'/>
      <target dev='hda' bus='ide'/>
    </disk>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <interface type='bridge'>
      <mac address='54:53:01:12:4c:0a'/>
      <source bridge='br0'/>
      <model type='ne2k_pci'/>
    </interface>
    <interface type='bridge'>
      <mac address='54:53:02:1e:9c:0b'/>
      <source bridge='br1'/>
      <model type='ne2k_pci'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
<input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
  </devices>
</domain>
 

When you create guests with the virt-manager or virt-install, the guests configuration files are created automatically in the /etc/libvirt/qemu/ directory. You can use this file error checking. You can edit this file using a text editor or virsh command itself:
virsh edit freebsd
virsh edit centos

KVM tools

Type the following command to install kvm-tools package which contains some diagnostics and debugging tools for KVM, such as kvmtrace and kvm_stat, enter:
# yum -y install kvm_tools
You need to mount debugfs, enter:
# mount -t debugfs debugfs /sys/kernel/debug
# kvm_stat
Sample outputs:

Fig.01: Getting stats about KVM with kvm_stat

You can use normal tools to troubleshoot and get other information using the following tools:

• ps, pstree, and top
• vmstat, iostat, and lsof
• tcpdump, brctl, ip, and ifconfig