Case

For distribute system, sometimes the datetime on every server in the cluster, or a group of servers need to be synced.

Time on a Ubuntu Server

Timekeepers in the System

Timekeeper, who hold its own time.

  1. System time: managed by the Linux kernel
  2. RTC: real-time clock, hardware clock on your motherboard CMOS
    • Should always in the UTC timezone. But there’s nothing in the hardware clock itself says which timezone is used. Can be adjusted by,
      • BIOS setting
      • hwclock(8)
    • set this clock only to a whole second
    • hwclock --systohc is called on server shutdown and also every 11 mins (if 11 mode enabled), which means setting just hwclock make no sense (ref. man hwclock)

Adjustment

The Hardware Clock is usually not very accurate. However, much of its inaccuracy is completely predictable - it gains or loses the same amount of time every day. This is called systematic drift.

Stock services in Ubuntu 16.04

systemd-timesyncd.service

systemd-timesyncd.service come with stock Ubuntu 16.04.

It will conflict with any other time sync daemons, ref:

So stop it by sudo timedatectl set-ntp 0 before installing other service, although the stock systemd-timesyncd comes with the built-in confliction setting,

# /lib/systemd/system/systemd-timesyncd.service.d/disable-with-time-daemon.conf
[Unit]
# don't run timesyncd if we have another NTP daemon installed
ConditionFileIsExecutable=!/usr/sbin/ntpd
ConditionFileIsExecutable=!/usr/sbin/openntpd
ConditionFileIsExecutable=!/usr/sbin/chronyd
ConditionFileIsExecutable=!/usr/sbin/VBoxService

So, if we have another timesync service installed, systemd-timesyncd will not be operating even that service is enabled. For ex., With Chrony installed,

$ timedatectl
...
Network time on: yes
$ systemctl status systemd-timesyncd
  ...
  Active: inactive (dead)
  Condition: start condition failed at Thu 2020-01-09 14:07:32 MST; 2h 6min ago

So it's not operating. Relax.

systemd-timedated.service

systemd-timedated.service come with the stock Ubuntu 16.04.

Useful with,

time-sync.target

//TODO

Sync Protocols

NTP

Accuracy

Implementations

  1. Chrony
  2. Ntpd
  3. timed

PTP

Sync Service: Chrony

https://chrony.tuxfamily.org/manual.html

Normally chronyd will cause the system to gradually correct any time offset, by slowing down or speeding up the clock as required.

Such config comes with chrony 2.1.1-1ubuntu0.1 at /etc/chrony/chrony.conf

pool 2.debian.pool.ntp.org offline iburst
keyfile /etc/chrony/chrony.keys
commandkey 1
driftfile /var/lib/chrony/chrony.drift
log tracking measurements statistics
logdir /var/log/chrony
maxupdateskew 100.0
dumponexit
dumpdir /var/lib/chrony

# This directive forces `chronyd' to send a message to syslog if it
# makes a system clock adjustment larger than a threshold value in seconds.
logchange 0.5
hwclockfile /etc/adjtime
# enables kernel synchronisation (every 11 minutes)
rtcsync

The Simplest Sync Model

POOL.NTP.ORG -ntp-> Master
                     |-ntp-> Client
                     |-ntp-> Client
                     |-ntp-> Client
                     ...
# stop the stock NTP service
timedatectl set-ntp 0

On Master,

sed -i '/^pool/s/$/ minpoll 1 maxpoll 2/' /etc/chrony/chrony.conf
cat <<EOT >> /etc/chrony/chrony.conf
allow CLIENT_IP
# allow makestep any time by the 2nd -1
makestep 1 -1
local stratum 8
EOT

On Clients,

sed -i '/^pool/s/^/# /' /etc/chrony/chrony.conf
cat <<EOT >> /etc/chrony/chrony.conf
# presend to prevent the delay on ARP request
server MASTER_IP prefer offline iburst presend 9 minpoll 1 maxpoll 2
makestep 1 -1
local stratum 10
EOT

systemctl restart chrony.service to reload the service.

Useful Tools

Ref.

  1. https://en.wikipedia.org/wiki/Orders_of_magnitude_(time)
    • ns 1e-9
    • us (micro-) 1e-6
    • ms (milli-) 1e-3
  2. man 8 hwclock
  3. https://www.linux.com/tutorials/keep-accurate-time-linux-ntp/
  4. https://chrony.tuxfamily.org/manual.html
  5. https://www.digitalocean.com/community/tutorials/how-to-set-up-time-synchronization-on-ubuntu-16-04
  6. https://chrony.tuxfamily.org/doc/2.1/manual.html
  7. https://www.meinbergglobal.com/english/info/ntp-packet.htm
  8. https://wiki.archlinux.org/index.php/Chrony#Usage
  9. https://chrony.tuxfamily.org/faq.html
  10. https://docs.fedoraproject.org/en-US/Fedora/18/html/System_Administrators_Guide/sect-Checking_if_chrony_is_synchronized.html