/sbin/hwclock --utc --systohc not working on AWS EC2 under Ubuntu

Question

I'm in the process of moving some servers to AWS. One problem I'm encountering is that my script to synchronize the time using ntpdate and then write that time to the hwclock is failing. When the server reboots, I'd like the time to be close to accurate. However, I can't seem to adjust the hardware clock.

My script reports that the system time and the hardware clock differ by about half a second:

systime: 2018-01-03 05:51:58.529746-0500
hwtime: 2018-01-03 05:51:59.023462996-05:00
diff (sec): 0.493716955184937

I'd expect that running sudo /sbin/hwclock --utc --systohc would correct the problem. It certainly does on my traditional non-virtualized servers. However, when I run that command, I still see the same half second difference between the system time and the hardware clock.

My instance was created using ubuntu/images/hvm-ssd/ubuntu-artful-17.10-amd64-server-20171208 (ami-d53b3eb5). I'm getting the system time using date --iso-8601=ns and the hardware time using sudo /sbin/hwclock --show. I'm comparing them after parsing them with Perl's Date::Parse.str2time.

Is setting the hardware clock not supported in virtualized environments such as AWS? If that is the case, is there any workaround to ensure that the time remains synchronized when AWS servers reboot?

they recommend chrony and erasing ntp - 'Amazon Time Sync Service' — Sum1sAdmin, Jan 03 '18 at 11:38
I see that they recommend chrony and have their own servers for you to use with it: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.htm However, that article doesn't address the hardware clock at all. Any synchronization done by ntp or by chrony will be lost at reboot unless I can change the hardware clock. — Stephen Ostermiller, Jan 03 '18 at 12:10
well it doesn't have hardware does it - the clock is on the hypervisor which has a 'weird' system bus inside the VM. /sys/devices/system/clocksource/clocksource0/current_clockso‌urce — Sum1sAdmin, Jan 03 '18 at 12:18
I would expect that the hypervisor would have its own hardware clock and emulate a hardware clock for each instance by storing an offset for each one. Given that setting the hardware clock doesn't seem to work, I suspect that may not be the case. Is the only workaround to synchronize the time via the network after each boot? — Stephen Ostermiller, Jan 03 '18 at 12:49
I'm not sure - I've never really had a dependency for the hardware clock, always use network. but you could look at the cloud-init scripts (run one, run every boot etc.) here's ntp but you could also include your perl script to run afterwards https://github.com/number5/cloud-init/blob/master/doc/examples/cloud-config-ntp.txt — Sum1sAdmin, Jan 03 '18 at 13:20

score 0 · Answer 1 · answered Jan 07 '18 at 10:37

As Sum1sAdmin says in the comments, the virtual machines don't have a hardware clock. You can tell what the clock source is by checking /sys/devices/system/clocksource/clocksource0/current_clocksource.

I've altered my script to check this file and only synchronize the hardware clock from system time on machines that are not XEN virtual machines.

my $clocksource=`cat /sys/devices/system/clocksource/clocksource0/current_clocksource`;
chomp $clocksource;
if ($clocksource !~ /^xen$/){  # Can't adjust hardware clock on virtual machines
    my $hwdate=`/sbin/hwclock --show`;
    chomp $hwdate;
    $hwdate=~s/T/ /g;
    $hwdate=~s/,/./g;
    my $hwtime=str2time($hwdate);

    my $diff=abs($systime-$hwtime);

    if ($diff > $report_change_threshold_seconds){
        print "diff (sec): $diff, hwtime: $sysdate, systime: $hwdate\n";
        my $msg="the hardware clock to the current system time";
        my $cmd="/sbin/hwclock --utc --systohc";
        if ($test){
            print "Set $msg using:\n";
            print "  sudo $cmd\n";
        } else {
            print "Setting $msg\n";
            print `$cmd`;
        }
    }
}

/sbin/hwclock --utc --systohc not working on AWS EC2 under Ubuntu

1 Answers1