42

I'm working on getting some servers running in the EC2 environment and I'm noticing some errors with ntpd trying to sync (using CentOS).

I was reading on this site and the impression I get is that I don't need to run ntpd since EC2 is Xen and the host takes care of the time for the virtual servers.
http://support.ntp.org/bin/view/Support/KnownOsIssues

Is this accurate or do I need to figure out how to get around the error I'm having?
cap_set_proc() failed to drop root privileges
It looks like it involves building a new kernel and other stuff I'd rather not do if I don't have to.

ahanson
  • 1,674
  • 2
  • 15
  • 21

4 Answers4

74

Yes, you need to run ntpd.

My clock was 18.5 seconds off on an EC2 micro instance (running Ubuntu UEC Maverick) with 5 days uptime.

After shutting down and starting again, it was back to normal, so there seems to be some kind of drift.

This is despite /sys/devices/system/clocksource/clocksource0/current_clocksource saying xen, by the way. I'm not sure why it's not working.

Installing the ntp package has solved the problem for me. The clock stays accurate, and there's nothing suspicious in the syslog that might indicate a conflict with Xen's clock synchronization. (It uses ntp.ubuntu.com as its server. I'm not sure if there's an NTP server in the AWS network that I could use instead, but the Ubuntu server will do nicely for now.)

Update: I've recently observed that on my (newer?) instances the clock stays accurate automatically, without ntp running. Judging by the comments, this doesn't seem to be the case for everyone though, so it's probably still best to use ntp just in case.

Jo Liss
  • 2,818
  • 1
  • 22
  • 23
  • 8
    I just noticed a several minute drift in one of my servers in which `/sys/devices/system/clocksource/clocksource0/current_clocksource` is set to `xen` as well – jberryman Jul 07 '11 at 15:39
  • 3
    independent_wallclock sysctl was not accepted upstream for the pvops kernels. Newer kernels (perhaps 2.6.24 and above) behave as independent wallclock == 1. So the only solution is to use NTP. – akostadinov Jan 23 '13 at 19:32
  • 3
    This post has a comment from (supposedly) an AWS staff member saying that EC2 does not sync clocks, so you should run NTP. https://forums.aws.amazon.com/thread.jspa?messageID=364749 – Dan Pritts Feb 15 '13 at 19:56
  • 2
    ...also, the ntp.org pool has some amazon specific servers: http://forums.aws.amazon.com/message.jspa?messageID=312682 – Dan Pritts Feb 15 at 22:21 – Dan Pritts Mar 20 '13 at 21:55
  • 1
    just had a > 5 min time drift on ec2 micro instance with /sys/devices/system/clocksource/clocksource0/current_clocksource == xen – Moritz Oct 12 '15 at 12:38
  • Looks like my latest Ubuntu are synchronized without effort, but my older instances, using different Debian distribs are not. I added some daily crons using ntpdate, and hope it will do. (ntdp doesn't seem available on my older systems, even after installing ntp / ntpdate. Did I miss something ?) If anyone is interrested by my noobie crons, here's what I set : `00 5 * * * /usr/sbin/ntpdate 0.amazon.pool.ntp.org > /tmp/logResyncAZ0.log 2>&1` Note that since most my servers are now on AWS, I use an Amazon ntp server . You can also use 1.amazon.[...], or 2/3. ...or any other ntp server. – Balmipour Jun 01 '16 at 13:38
  • chrony replaced ntpd in RHEL/CENTOS 7, so in this case ntp would still be synchronising the box apparently without ntpd running. – Bryan Solan May 17 '19 at 15:28
9

We run Linux ec2 instances from small to extra large and I just did a check to make sure they were in sync.

All servers were running the ntp daemon connecting to amazon specific pools.

server 0.amazon.pool.ntp.org iburst
server 1.amazon.pool.ntp.org iburst
server 2.amazon.pool.ntp.org iburst
server 3.amazon.pool.ntp.org iburst

This may not have been the case before bug I just wanted to make sure whoever stumbles across this in the future has up to date information.

Zach
  • 91
  • 1
  • 2
2

You shouldn't need to. The clock of your EC2 instance will be synchronized to the Xen Dom0. You can verify this by checking that the contents of /proc/sys/xen/independent_wallclock are 0.

Kamil Kisiel
  • 11,946
  • 7
  • 46
  • 68
  • 2
    I could not find anything like this. Is this information still applicable? – Kariem Mar 28 '11 at 11:15
  • 8
    @Kariem: Newer kernels have `/sys/devices/system/clocksource/clocksource0/current_clocksource`, see the comment below. – intgr Apr 13 '11 at 10:38
  • 4
    For me /sys/devices/system/clocksource/clocksource0/current_clocksource contains "xen" but still time on the machine is off by a couple of minutes. – akostadinov Jan 22 '13 at 17:32
  • 1
    The time is often wrong on an AWS service. It's not even clear if their servers sync at all. – Ken Sharp Jan 27 '16 at 22:10
  • 1
    I'm also getting bitten by this on Ubuntu EC2 instances. For future reference I would recommend setting up ntp for non Amazon Linux amis. – Daryl Teo Feb 15 '16 at 22:45
  • This answer is incorrect. – jonatan May 05 '17 at 10:53
2

You have to configure ntpd on all instances. If you do not your servers' time will start drifting.

You have to allow UDP 123 to the public internet, as described in the user guide under Configuring Network Time Protocol.

If your instances are inside a VPC, you may want to dedicate an instance to be your NTP server, only expose that instance to the internet and let the others sync against it.

jonatan
  • 465
  • 4
  • 10