Virtualization @ Ubuntu
f you I have just started to explore virtualization for an internal project. The objective is to provide an “unbounded” number of ad-hoc servers for testing and customer show-case. So, I’m going to assemble a physical server, with a 4-core 64bit CPU, 8GB RAM and 1TB of mirrored disk space (RAID1). In the mean time I’m investigating how to set up and use KVM running on Ubuntu, on one unused server at home.
Virtualization
Virtualization is the art of emulating computer with a computer. It has been around for a very long time, becuase it was one of the corner stones of main-frames. However, it is only recently it has been easly available for “mere mortals”. The term is sometimes blurred by the two different concepts of application virtualization and system virtualization respectively. The former means virtual machines like the JVM for Java, CLR for .NET, JAM for Erlang, Parrot for Perl6 and more.
However, it is the latter (system) we normally refer to as virtualization. That means running one or more emulated computers on top of a real computer. User-space emulators has been around for several year, like VMware Workstation, VirtualBox and QEMU, they do not require hardware support nor changes of the OS.
When hardware manufactors noticed the increased interest for virtualization, the started to support it natively. Intel-VT and AMD-V are two such technologies. As soon as the these CPUs hit the market new tools came out relying on the built-in support. KVM is one such open source tool, which is directly supported in Ubuntu.
So the rest of this (quite long post) will discuss how to
- Install KVM on Ubuntu (Server)
- Create virtual machines and install an OS on them
- Configure bridged networking, so one can reach a VM
Before you proceed reading, just a small disclaimer: The text is the result of my own recent investigation of the topic and serves as a presentation of my findings. Googling around gives many different view points of how to setup KVM and most of them are not complete from a newbies point of view. Therefore, I’m trying to fill that missing spot.
Prerequisites
The first step is to understand and check the requirements: Does my CPU support virtualization?
If you are considering buying a CPU for assembling you own server, look for support of Intel-VT or AMD-V, depending on your preference of CPU manufactor.
If you already have a system you can check for harware support with this command:
jens@goofy:~$ egrep '(vmx|svm)' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr lahf_lm flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr lahf_lm
If you don’t see any output, it means your CPU do no support hardware virtualization. You can still use user-space virtualization tools, but not KVM. The flag ‘vmx’ is for an Intel CPU and ’svm’ for AMD.
You can use virtualization on both 32-bit and 64-bit systems. However, for the former you are limited to 2GB address space, therefore it is common to go for a 64-bit system. For a 64-bit system, you need both a 64-bit CPU and an 64-bit OS installed. (N.B. all commands below are for Ubuntu Linux)
Use this command to check if your CPU is 64-bit
jens@goofy:~$ grep ' lm ' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr lahf_lm flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr lahf_lm
If nothing is printed it means the ‘lm’ long mode flag is not present. To see if your OS is 64-bit you can use the following command
jens@goofy:~$ uname -m x86_64
If you see anything else, like 386, i486, i586 or i686, it means your are running a 32-bit OS.
Installation
The first step is to install the basic package of software.
sudo apt-get install kvm qemu uml-utilities bridge-utils
This will install
- kvm - provides kernel based virtualization
- qemu - provides virtualization tools (kvm and qemu goes together)
- uml-utilities - user mode Linux, contains a set of tools and we are going to use tunctl
- bridge-utils - another set of tools for creating network bridges
Next you need to add yourself to the kvm group and add the kvm kernel module (kvm-intel for Intel and kvm-amd for AMD)
sudo adduser `id -un` kvm sudo modprobe kvm-intel
Finally, logout and login again, so the group change will take place. Remark; you can postpone the logout till after the network configuration, because you will need to reboot anyway. However, if you want to create your first VM without the bridge, do the logout/login now.
Virtual Machines
Compared to using products like VMware and VirtualBox, it is relievingly easy to create a VM in KVM. First you create the virtual disk, then you launch KVM with an OS ISO to run the installer. Therefore, grab an ISO file (or CD) for an OS. I have chosen to go for Ubuntu Server 8.04 64-bit (the link goes to a Swedish mirror, you might want to use another mirror). You can find all Ubuntu variants here. Let’s assume you now have the file ubuntu-8.04.1-server-amd64.iso, in a sub-directory called iso/.
Create a virtual disk
We are using the qemu tool to create the virtual disk.
jens@goofy:~/vm$ qemu-img create ubuntu-server-8_04-disk.img -f qcow2 6G Formatting 'ubuntu-server-8_04-disk.img', fmt=qcow2, size=6291456 kB jens@goofy:~/vm$ ls -lFh total 40K drwxr-xr-x 2 jens jens 4.0K 2008-09-23 13:06 iso/ -rw-r--r-- 1 jens jens 36K 2008-09-23 13:06 ubuntu-server-8_04-disk.img
This will create a file named ubuntu-server-8_04-disk.img in the current directory, of maximum size 6GB (expands on demand) and with the QEMU file format ‘qcow2′. You can for example create a virtual disk that is compatible with VMware. See the manual for more information ‘man qemu-img’. That’s it. Now time to install the OS.
Launch the installer
When you launch KVM it will pop-up a console, which of course requires you to be running on a desktop system. If you are running on a server, without a desktop, you can instead launch KVM with a VNC server providing the console. That means you can connect remotely to using a VNC viewer, such as TightVNC for Windows or Vinagre for a GNOME desktop.
Desktop console
kvm -m 512 -k sv -boot d -cdrom iso/ubuntu-8.04.1-server-amd64.iso -hda ubuntu-server-8_04-disk.img
This will boot a VM from a CDROM (’-boot d’), where the CD is pointing to an ISO file (-cdrom iso/…). The VM will be running with 512 MB of RAM (’-m 512′) and use a Swedish keyboard layout (’-k sv’). If you are not Swedish you probably want to change ’sv’ to something else or skip this option. Finally, it will use the virtual disk we created before as the IDE hard disk 0 (’-hda ubuntu-…’). You can find more info with the command ‘kvm –help’.
VNC remote console
If you want to work remotely, just append ‘-vnc :1′ to the command. This will launch KVM with a VNC server running on port 5901 (-vnc :n => 5900 + n).
kvm -m 512 -k sv -boot d -cdrom iso/ubuntu-8.04.1-server-amd64.iso -hda ubuntu-server-8_04-disk.img -vnc :1
Installing OS
If you went for the desktop console you will see a console pops up and the machine booting, finally the Ubuntu server installer screen. I will proceed with the VNC console. My host machine (goofy) has (an internal) IP of 192.168.189 and I will connect using VNC. (Of course, you should substitute your own IP)
When I’ve logged on to the console, the VM has already booted and the first screen of the installer is shown (click on the image for a full-scale picture). Choose your language and press ENTER, it will then show you the main menu.
I will not digress into installing Ubuntu Server, just point out a few topics. The installer will ask you for the system language to use, your location, machine name (e.g. vm1) etc. When it comes to disk partitioning, let the installer do the full job.
Installing the OS will take serveral minutes. When it prompts you for setting up a user name and password, write it down as well, becuase you will need it shortly. Towards the end it will ask you to install various packages. Just choose SSH to start with, you can install the server tools later using apt-get.
At last, the installer will ask you to reboot. The reboot will fail, becuase you need to unmount the CD and change the KVM launch parameters. For now, just terminate KVM (CTRL-C in the command window you launched it from) and return to the command prompt, You will of course loose your VNC connection when the VM goes down.
After the installation we can see the disk has grown a bit.
jens@goofy:~/vm$ ls -lFh total 704M drwxr-xr-x 2 jens jens 4.0K 2008-09-23 13:06 iso/ -rw-r--r-- 1 jens jens 703M 2008-09-23 13:55 ubuntu-server-8_04-disk.img
Running VM
Now is the time to launch the new VM. Basically, you will use the same command as above, except without the boot and cdrom flags. Here I’m launching the VM with a VNC server console.
kvm -m 512 -k sv -hda ubuntu-server-8_04-disk.img -vnc :1
I’m now logged on to the VM as you can see. The next step is to check the network connectivity. Do we have an IP number? As you can see below, we have one (10.0.2.15).
Next, let’s see if we can reach out from the VM.
wget -o /dev/null -O - http://www.ubuntu.com/ | head
This will print the first lines of the welcome HTML page of the Ubuntu web. Choose you favorite site. You you can now reach out, but not reach in, i.e., without a bridge you cannot logon using SSH, for example.
jens@goofy:~$ ssh jens@10.0.2.15
^C
jens@goofy:~$ ping 10.0.2.15
PING 10.0.2.15 (10.0.2.15) 56(84) bytes of data.
^C
--- 10.0.2.15 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6008ms
jens@goofy:~$ traceroute6 10.0.2.15
traceroute: unknown host 10.0.2.15
jens@goofy:~$ tracepath6 10.0.2.15
getaddrinfo: Resolver Error 0 (no error)
jens@goofy:~$ tracepath 10.0.2.15
1: goofy.local (192.168.0.189) 0.112ms pmtu 1500
1: kerberos (192.168.0.1) 0.282ms
1: kerberos (192.168.0.1) 0.228ms
2: kerberos (192.168.0.1) 0.249ms reached
Resume: pmtu 1500 hops 2 back 64
jens@goofy:~$
Shutdown the VM and procced to the next section. Enter this command inside the VM’s console
sudo poweroff
Network Configuration
After you have created a VM and installed an OS, it has access to the outer world using NAT, but is hidden inside a private VLAN. You have only access to it using its console, i.e., you cannot login using SSH nor set up Apache and view it over HTTP. Therefore, you almost always want to create a network bridge, which means the VM will operate as if it was a ‘normal’ machine on your network. Setting up a bridge and launch KVM requires several steps: some system re-configuration, creating some scripts and a reboot. The steps below are taken from the Ubuntu community KVM doc.
In short, these are the steps:
- Add a bridge interface to /etc/network/interfaces
- Add permissions to a TUN device in /etc/udev/rules.d/40-permissions.rules
- Sudo:fy an kvm interface startup hook in /etc/kvm/kvm-ifup
- Add sudo rights for the kvm group
- Create a kvm launcher script
- Reboot
I suggest you perform all changes directely on the host machine’s console, not via SSH, because some of the changes will temporary cutoff your network connectivity.
You can use your favorite editor to edit the files, just remember to launch the editor using sudo. Ubuntu comes preinstalled with both VIM and NANO. Personally I prefer Emacs (and hate VI), but you have to install it first. Nano is a good very small editor and intuitive to use in a console window.
/etc/network/interfaces
The first file to edit requires you to temporary cutoff your network. Shut it down with
sudo invoke-rc.d networking stop
The open /etc/network/interfaces in nano
sudo nano /etc/network/interfaces
Change the content to something similar to
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
The important points are the physical device eth0 is set to manual mode, before it was probably set to dhcp. A new device (br0) is defined, which bridges to eth0 and gets its IP from DHCP. You can also use a static IP, for details, see the Ubuntu docs mentioned above. Save and exit with ^O ENTER ^X. Finally, restart the network again.
sudo invoke-rc.d networking start
/etc/udev/rules.d/40-permissions.rules
Next, edit the file /etc/udev/rules.d/40-permissions.rules and append the following line to the end of the file
KERNEL=="tun", GROUP="kvm", MODE="0660"
/etc/kvm/kvm-ifup
Next, edit /etc/kvm/kvm-ifup and add ”’sudo”’ (tripple single quote around sudo) in front of ifconfig and brctl. Here is how it should look.
#!/bin/sh
switch=$(ip route ls | awk '/^default / { for(i=0;i<NF;i++) { if ($(i) == "dev") print $(i+1) }}')
'''sudo''' /sbin/ifconfig $1 0.0.0.0 up
'''sudo''' /usr/sbin/brctl addif ${switch} $1
exit 0
/etc/sudoers
Next, add sudo rights for the kvm group. In this case, you must use the VI tool called ‘visudo’, which updates the file /etc/sudoers. The tool visudo checks the syntax and guards you from accidental misstakes.
sudo visudo
Append the following line to the end of the file
%kvm ALL=(ALL) NOPASSWD: /sbin/ifconfig, /usr/sbin/brctl, /usr/sbin/tunctl
This will permit members of the kvm group to use ifconfig, brctl and tunctl. If you never have used VI before, please be very, very careful. VI operates in modes, that means insert, append and navigation mode. If you screw up, leave without saving with <ESC>: q!
Use the arrows keys to scroll to the very last character of the file, type <ESC>a to go into append mode, press <ENTER> for a new line and paste the line above. Then press <ESC>: wq to write and quit. Verify by looking at the file.
sudo cat /etc/sudoers
kvm-run
At last, you will now create your own launcher script. You can put it in the same directory as your VMs or in ~/bin (if you have one) or somewhere else. Just remember where. Create the file (kvm-run) and fill it with the text below
#!/bin/bash USERID=`whoami` MAC=$(echo DE:AD:BE:EF:`echo $RANDOM | cut -c -2`:`echo $RANDOM | cut -c -2`) NIC_MODEL=rtl8139 NIC_IF=`sudo tunctl -b -u $USERID` kvm -net nic,vlan=0,macaddr=$MAC,model=$NIC_MODEL -net tap,vlan=0,ifname=$NIC_IF $@ sudo tunctl -d $NIC_IF &> /dev/null
The script will be used as your alternate kvm launcher. It first gets your user name, then generates a hardware id (MAC) for the network interface (NIC), assigns a model name to the nic and creates a tunnel device owned by you and with brief output. Finally, it launches kvm with the network settings and the command line parameters appended at the end. You need to make the launcher script executable
chmod +x kvm-run
Reboot
Now is the time to reboot your host machine.
sudo reboot
Running with bridged network
Back again, it’s time to take your VM out for a ride. Start it using the launcher script and add parametes for memory, disk, keyboard and vnc.
./kvm-run -m 512 -k sv -hda ubuntu-server-8_04-disk.img -vnc :1
Attach to the VNC console and check the IP. This time we have an IP at our local net (192.168.0.186).
Now, let’s check if we can logon using SSH (PuTTY).
We are in, great! From now on, you barely need the VNC console, use PuTTY or your favorite SSH tool.
Before I end this (very long) post, let’s install Apache and verify it works as expected. Within your VM (now use your ssh access), type the command below and confirm with ‘Y’
sudo apt-get install apache2
After the installation you should have two servers running, the SSH server from before and the new Apache web server.
jens@vm2:~$ netstat -l -t inet Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 *:www *:* LISTEN tcp6 0 0 [::]:ssh [::]:* LISTEN
Check with your browser, from somewhere in your local net and you will see the default Ubuntu Apache start page.
Final words
This concludes my tutorial of how to setup KVM on Ubuntu and make the VMs visible on your local net. There are many more topics to describe, but that will be in some other post. For example (no promise):
- High-level tools for creating and managing VMs
- Convenient installation of Ubuntu direct from the network, using a shrink-wrapped Ubuntu OS (JeOS)
- Installation of some other Linux OS
- Installation of some Windows flavor OS
Good luck and please comment.
Comments
3 Responses to “Virtualization @ Ubuntu”
Leave a Reply
-
Jens Riboe
Hi,
this looks like a very nice step by step howto. I also like kvm and I use it for several things. I am coming from Germany and I have recognized one problem with the -k and -vnc option. I have selected a German keyboard layout, but when using the vnc connection, some keys are missing, i.e. the pipe, the keypad, etc.
I saw that you use sv as layout. Did you recognize such problems, too?
And one suggestion: Have a look for vde2. Cool stuff.
I have written a kvm-manage script that can start and stop (and some more) kvm guests controlled. That means. One-by-one with time-parameters. It is desigend for the init.d. If you want to, have a look here:
German blog entry
http://www.roessner-net.com/?p=219
and the script (commented in english):
http://www.roessner-net.com/kvm-manage
So long
Christian
>>I saw that you use sv as layout. Did you recognize such problems, too?
Yes, I noticed that some keys were missing even with a Swedish layout.
Your kvm-manage scripts seem to be really cool. Good work.
I haven’t heard about VDE2 before, it seems to be very useful for cloud like virtualization deployments. Thanks for the tip, I will investigate.
Merry Christmas,
/jens
[...] Virtualization @ Ubuntu : RiboComments Very good guide to set up KVM virtualization. Allows for running the installation process instead of using a prebuilt template either via the kvm command directly or as a VNC service to be attached to remotely. (tags: kvm tutorial ubuntu virtualization howto Install) [...]