Timezone inaccuracy

Question

we've been having an issue getting time synced across our Litespeed web cluster, that being that the system time (we're using CentOS) appears to be thinking that the current time for EST (Being the "America/New_York") time zone is 3 hours ahead of current time.

In Florida right now the time is 10:53PM The server reports this: TZ='America/New_York' will be used. Local time is now: Tue Jun 4 01:51:43 EDT 2013. Universal Time is now: Tue Jun 4 05:51:43 UTC 2013.

What would cause something like this? The same is applying in our PHP.ini configuration for the cluster.

This is the response from a perfectly fine web-server which is configured exactly the same: TZ='America/New_York' will be used. Local time is now: Mon Jun 3 22:53:15 EDT 2013.

--Update 1-- ntp peers on a WORKING server:

krillin.ecansol 184.164.149.162  2 u   52   64    1   35.631  110337. 7937.50
 ntp.sunflower.c 184.164.149.162 16 u    -   64    0    0.000    0.000 15937.5
 triangle.kansas 184.164.149.162  2 u   50   64    1   55.125  110331. 7937.50

ntp peers on a BROKEN server:

 fairy.mattnordh 164.244.221.197  2 u   59   64    7   49.397  -108052   1.171
 ec2-50-16-231-1 209.51.161.238   2 u   60   64    7   59.565  -108052   0.016
 colossus915.ser 200.23.51.102    2 u   59   64    7   46.693  -108052   0.051

Answer 1

Unless you've specifically configured your OS to expect otherwise, your server's system clock must reflect UTC, not local timezone. Timezone offsets are applied after-the-fact, as multiple timezones can be used at the same time on a single server. Your timezone file tells the server what time to return given the assumption that the system clock is set to UTC.

NTPd will refuse to synchronize if the system clock is too far off the target clock. You have to manually set it to something within a minute or so of the target time (UTC); either that or use ntpdate to fetch the time from an NTP server and immediately set the clock accordingly. Then you can run ntpd to fine-tune your clock to get the millisecond precision you need.

After a warm-up time, ntpq -pn should show one remote server marked with a * on the left to indicate that it has been been declared the primary synchronization peer, with others hopefully marked with a + to indicate that they are qualified back-up candidates for peering.

If not, then your clock is not being synchronized.

Note that in the list of peers, the offset should be less than 1ms [absolute value] in normal running condition, perhaps a few hundred ms as the clock gets fine-tuned. Your list shows an offset so far out of range that synchronization using clock slewing is impossible. Instead, you'll need to "jump" your clock to the correct time, something ntpd will by policy never do on its own. That's what the ntpdate tool is for.

Answer 2

Are you running ntpd with the -g option? It should be specified in /etc/sysconfig/ntpd. Without -g, ntpd will refuse to set the clock if it is off by more than 1000 seconds.