18

Couldn't the guests somehow inherit the system time of the host ?

It seems kind of pointless to run the same daemon to get the same results on the same machine multiple times but I didn't find anything related to time when reading KVM or Xen articles. My understanding is that the guest gets the host time on boot but that it might then drift apart. Is that correct ?

zimbatm
  • 443
  • 1
  • 4
  • 7
  • 3
    http://www.linux-kvm.org/page/KVMClock – Hauke Laging Mar 16 '13 at 03:45
  • 1
    I presume you mean an NTP client? – Michael Hampton Mar 16 '13 at 11:49
  • 1
    @MichaelHampton: Yes, the distinction between a NTP client and server never was really clear to me since the ntp package always provides both. – zimbatm Mar 21 '13 at 13:55
  • 2
    Related to this question: on Debian and derivatives, the `openntp` package is much more lite-weight than the `ntp` package. It doesn't bind by default and so just acts as a client which is exactly what we need here. – zimbatm May 04 '13 at 18:16

4 Answers4

22

In a perfect world, your VM guests would keep perfect time, or at least as perfect as the host provides. Unfortunately we don't live in a perfect world.

Based on my experience with virtually every hypervisor known to man, I always run an NTP client in virtual machines, without exception. My usual setup is ntpd with the -g option, or ntpdate starting right before it for old systems, to step the clock (which may be far out of sync at system boot).

KVM has nearly the perfect setup, with its paravirtualized realtime clock; guests with the appropriate driver (all recent Linux, at least) will keep time as well as the host. But still things go wrong here: For instance, the host may not be running NTP, the host may have an incorrect time zone set, the host's clock may just be plain wrong, etc.

VMware and Hyper-V fall in the middle. Each has a tool meant to be run on the guest which syncs the clock with the host periodically, but again, this is vulnerable to any existing problems with the host clock.

Guests on my test Hyper-V server also exhibited some strange behavior: even with integration services, the guest clock would drift faster than 500 ppm, preventing ntpd from working (it considers the clock insane if it drifts faster than this). I had to switch these guests to chrony, which allows this value to be adjusted.

Xen is the worst in this respect; it has absolutely no synchronization and running NTP in the guests is pretty much required. (I am told that very recent versions of Xen have some sort of synchronization but haven't personally worked with it yet.)

Things just get worse if the host hypervisor isn't under your control, such as a public cloud. You are at the mercy of the provider with respect to the host clock, and if they aren't diligent in keeping it synchronized, you lose.

With all that, running NTP clients in your virtual machines is pretty much required if you need even a semi-accurate clock. NB: If you run Windows virtual machines, get a third party NTP client that adjusts the clock continuously; the poor excuse for a client that comes with Windows only adjusts the clock once a week, which is utterly ridiculous.

Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • Is running `ntpdate` in a daily cron job enough or does it have to be the ntp daemon? – AngerClown Mar 17 '13 at 00:26
  • 4
    You really need something to sync the clock continuously, since it could go far enough out of sync in a day to cause you issues. And some programs don't like having the clock stepped. – Michael Hampton Mar 17 '13 at 15:05
  • The issue with the ntp daemon is that it's primarily intended to provide time to others and thus will stop working if time drifts too much apart. I really wish there was something more like ntpdate that would run continuously and not bind any ports. – zimbatm Mar 21 '13 at 14:09
  • 1
    @JonasPfenniger If ntpd isn't working out for you, try using chrony instead (as suggested above). It can be tuned quite a bit more than ntpd to accommodate the weird scenarios we see with virtual machines. – Michael Hampton May 18 '13 at 02:13
  • 3
    While I agree with 95% of what you said, I just wanted to point out that a Windows client using NTP isn't limited to one clock change per week. The following reg key allows you to define the update interval effectively: `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\UpdateInterval` The value is defined in seconds and is usually set to 360000 (100 hours), but you can set it to 900 to sync every 15 minutes if your applications are particularly time sensitive. Remember to restart the W32Time service afterwards. – kayjay Jul 13 '15 at 16:00
13

This is correct. It should be noted that time not only "can" drift away, but will drift away due to the fact that intervals between timer interrupts (which timekeeping on OS is often based on) are stretched and compressed as the hypervisor would see fit.

A commonly taken workaround across most virtualization platforms (Hyper-V integration services, VMWare tools) is running a daemon on the guest which periodically synchronizes clocks with the VM host. As noted by Hauke in a comment to your question, KVM additionally provides a paravirtualized clock which would need the respective driver loaded on the guest OS to work.

Further reading:

Timekeeping in VMWare Virtual Machines (vmware.com)
KVM guests clock synchronisation (s19n.net)

the-wabbit
  • 40,319
  • 13
  • 105
  • 169
  • Thanks a lot. For those on EC2, Xen also seems to provide a clock source: `cat /sys/devices/system/clocksource/clocksource0/current_clocksource` => xen – zimbatm Mar 21 '13 at 14:05
  • Still chronyd is recommended even if kvm-clock is used with the host as a server and the VM as a client. – akostadinov May 14 '15 at 09:52
3

I would recommend to use NTP because it is well known and has been around a long time. Adjusting the clock is not trivial. NTP has solved this issue.

The official line of VMware is to use one mechanism, with NTP preferred because it is more fine-grained and take smaller steps to adjust the time. The internal VMware solution takes bigger steps. When you run both they could fight each other. With the internal VMware solution taking a big step and then NTP adjusting it and taking it back a bit.

In practice however we run both at the same time and I haven't seen a problem yet.

$ ntpq   
ntpq> peers
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
something.org  172.2.1.5          2 u   57   64  377    1.597   -2.409   5.952


$  vmware-toolbox-cmd  timesync status
Enabled


$ vmware-toolbox-cmd help timesync
timesync: functions for controlling time synchronization on the guest OS
Usage: vmware-toolbox-cmd timesync <subcommand>

Subcommands:
  enable: enable time synchronization
  disable: disable time synchronization
  status: print the time synchronization status
Anon
  • 3
  • 1
jris198944
  • 87
  • 2
0

I wonder why this aspect wasn't mentioned in the answers so far:

VMs give you virtual machines, not being related to any other machine (that's the theory), so you could even have the freedom to set any VM to some arbitrary time.

That could be the reason why some environments do not sync the VMs time periodically to the host. (Some systems that do, actually do a poor job with time jumping back and forth noticeably)

Another point is boot time of a VM: You do not really expect the date to be the start of the epoch, like Jauary 1st 1970. Instead you expect the boot time to "continue" where you left the VM (possibly requiring a virtual RTC chip). So if you set your clock ahead of time by (let's say) two hours, and you reboot, you do not really expect boot time to jump back two hours, do you? Again not all environments do a good job there, however.

So in summary:

If you want all VMs to have the correct time, you can either synchronize each using NTP (and disable any other mechanism). The advantage is that NTP even allows to monitor the time being sync'ed correctly.

As an alternative optionally sync the VM host's clock, then sync the VMs from the host's clock using some mechanism the VM environment provides.

Final remark:

With massively over-committing CPUs, any VM will have a hard time to keep the correct time. Imagine your host has four CPUs, and you configured four VMs with two CPUs each. Then every VM runs two processes that read the CPU's cycle counter (TSC) at full speed. Which values should they see, and how could the VM host help doing so?

U. Windl
  • 286
  • 2
  • 13