2

Is there an option to show, that metadata.google.internal is used as NTP when i connect to a pod or to the kubernetes VM? I tried the following: ntpq -p, which results in command not found, but was listed on https://cloud.google.com/compute/docs/instances/managing-instances. This file does not exist: vim /etc/ntp.conf. I need that as a proof, showing which server is used.

mc_plectrum
  • 121
  • 2
  • 1
    Are you using `Container-Optimized OS` in your GKE cluster? – Dawid Kruk Mar 09 '20 at 09:35
  • One workaround would be to go for pod with package manager, install nslookup on it and check `$ nslookup metadata.google.internal`. It should give the IP address of `169.254.169.254`. After that you can check `systemctl status systemd-timesyncd.service` and specifically the part of `Synchronized to time server`. – Dawid Kruk Mar 09 '20 at 15:15

1 Answers1

4

When you create a GKE cluster you have an option to specify which operating system will be the base for your Kubernetes installation. You have the options to choose from:

  • Container-Optimized OS (cos)
  • Container-Optimized OS (cos) with containerd
  • Ubuntu
  • Ubuntu with containerd

You can choose it by following a path in Google Cloud Platform:

Kubernetes Engine -> Create a Kubernetes cluster -> Node-pools -> default-pool -> Nodes -> Image type.

What you choose will affect what tools you will be able to use internally on the nodes.

Container-Optimized OS

A word about Container-Optimized OS:

Container-Optimized OS is an operating system image for your Compute Engine VMs that is optimized for running Docker containers. With Container-Optimized OS, you can bring up your Docker containers on Google Cloud Platform quickly, efficiently, and securely. Container-Optimized OS is maintained by Google and is based on the open source Chromium OS project.

-- Cloud.google.com: Container-optimized OS

Please take a look on official documentation talking about features, benefits and limitations of Container-Optimized OS: Cloud.google.com: Container-optimized OS features,benefits and limitations

Container-Optimized OS does not include a package manager; as such, you'll be unable to install software packages directly on an instance. However, you can use CoreOS toolbox to install and run debugging and admin tools in an isolated container.

As said in above documentation there is no package manager available but you can use CoreOS toolbox to debug and install required software.

You can check which server is used by your Container-Optimized OS node with systemd as follows:

$ systemctl status systemd-timesyncd

It should give you an output similar to this:

● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/usr/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-03-03 21:17:35 UTC; 6 days ago
     Docs: man:systemd-timesyncd.service(8)
 Main PID: 219 (systemd-timesyn)
   Status: "Synchronized to time server 169.254.169.254:123 (169.254.169.254)."
    Tasks: 2 (limit: 4915)
   Memory: 832.0K
      CPU: 591ms
   CGroup: /system.slice/systemd-timesyncd.service
           └─219 /usr/lib/systemd/systemd-timesyncd

Please take a specific look on

   Status: "Synchronized to time server 169.254.169.254:123 (169.254.169.254)."

Above message indicates which server is used by node to synchronize time.

You can get more information about IP address of 169.254.169.254 by following: Serverfault.com: What is this IP address 169.254.169.254

You can check the DNS name of 169.254.169.254 with mentioned CoreOS Toolbox on your node and nslookup:

  • $ /usr/bin/toolbox
  • $ apt-get install -y dnsutils
  • $ nslookup 169.254.169.254

It should give you an output:

Server:         169.254.169.254
Address:        169.254.169.254#53

Non-authoritative answer:
254.169.254.169.in-addr.arpa    name = metadata.google.internal.

Authoritative answers can be found from:

Above output indicates that the address of 169.254.169.254 is the mentioned metadata.google.internal

Ubuntu

With GKE cluster created with Ubuntu image you check NTP information with:

  • systemd
  • dhcp lease
  • ntptrace
  • nslookup

Systemd

You can check which NTP server is used by your Ubuntu node the same as with Container-Optimized OS as follows:

$ systemctl status systemd-timesyncd

DHCP lease

The DHCP lease contains information about NTP server. You can get information about DHCP lease as follows:

USER@NODE:~$ cat /var/lib/dhcp/dhclient.leases

Output of above command should look like below: 

lease {
  interface "eth0";
  fixed-address 10.156.0.X;
  option subnet-mask 255.255.255.255;
  option routers 10.156.0.1;
  option dhcp-lease-time 86400;
  option dhcp-message-type 5;
  option domain-name-servers 169.254.169.254;
  option dhcp-server-identifier 169.254.169.254;
  option interface-mtu 1460;
  option domain-search "X.internal.", "Y.internal";
  option ntp-servers 169.254.169.254;
  option rfc3442-classless-static-routes 32,10,156,0,1,0,0,0,0,0,10,156,0,1;
  option host-name "X.internal";
  option domain-name "X.internal";
  renew 2 2020/03/10 09:12:37;
  rebind 2 2020/03/10 18:40:09;
  expire 2 2020/03/10 21:40:09;
}

Please take a specific look on: 

  option ntp-servers 169.254.169.254;

This line indicates the NTP server used.

Ntptrace

Ntptrace should show 169.254.169.254 as follows:

localhost: stratum 3, offset 0.000058, synch distance 0.026242
169.254.169.254: timed out, nothing received
***Request timed out

Nslookup

You can check the ntp-server with nslookup in the same manner as in CoreOS Toolbox if it matches the metadata.google.internal.

Please let me know if you have any questions in this topic.

Dawid Kruk
  • 588
  • 2
  • 8