In this article I'll be exploring two distinct issues I faced with ntp sync wherein the servers were not able to properly sync their time with ntp servers.
We observed a high offset when we checked the ntpq -p output.
I did a couple of stop-start operations followed by troubleshooting procedure outlined in RedHat KB articles 35640 and 64868 whose logs I'm sharing below:
[ssuri@usporiainfrar00:~]' $ sudo ntpq
ntpq> peers
remote refid st t when poll reach delay offset jitter
==============================================================================
+ntp.nova.org .GPS. 1 u 43 64 377 0.189 309.823 175.931
*mtntime.emrets. .GPS. 1 u 44 64 377 13.401 309.833 174.006
xusporz-infrac15 10.16.64.15 3 u 43 64 377 0.208 286.078 169.984
xgblonz-infrac07 10.16.64.15 3 u 39 64 377 91.365 287.473 167.180
ntpq> as
ind assid status conf reach auth condition last_event cnt
===========================================================
1 57822 941a yes yes none candidate sys_peer 1
2 57823 961a yes yes none sys.peer sys_peer 1
3 57824 9124 yes yes none falsetick reachable 2
4 57825 9124 yes yes none falsetick reachable 2
ntpq> rv 57822
associd=57822 status=941a conf, reach, sel_candidate, 1 event, sys_peer,
srcadr=ntp.nova.org, srcport=123, dstadr=10.16.216.184, dstport=123,
leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.290,
refid=GPS, reftime=dd6328c0.a7684541 Wed, Sep 13 2017 3:47:12.653,
rec=dd6328c3.fc501541 Wed, Sep 13 2017 3:47:15.985, reach=377,
unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=29, flash=00 ok,
keyid=0, offset=504.129, delay=0.171, dispersion=1.525, jitter=183.751,
xleave=0.028,
filtdelay= 0.19 0.17 0.19 0.22 0.21 0.19 0.20 0.21,
filtoffset= 553.93 504.13 454.51 405.18 356.99 309.82 250.70 191.10,
filtdisp= 0.00 0.96 1.94 2.93 3.92 4.91 5.88 6.87
ntpq> q
[ssuri@usporiainfrar00:~]' $ sudo ntpdate -u ntp.nova.org
13 Sep 03:51:50 ntpdate[87098]: step time server 10.16.64.124 offset 0.757074 sec
[ssuri@usporiainfrar00:~]' $ sudo ntpdate -d ntp.nova.org
13 Sep 03:51:59 ntpdate[87462]: ntpdate 4.2.6p5@1.2349-o' Tue May 3 15:12:51 UTC 2016 (1)
Looking for host ntp.nova.org and service ntp
host found : ntp.nova.org
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
server 10.16.64.124, port 123
stratum 1, precision -20, leap 00, trust 000
refid [GPS], delay 0.02580, dispersion 0.00000
transmitted 4, in filter 4
reference time: dd6329d6.e0896c6a Wed, Sep 13 2017 3:51:50.877
originate timestamp: dd6329df.4e7f4739 Wed, Sep 13 2017 3:51:59.306
transmit timestamp: dd6329df.4d0d781a Wed, Sep 13 2017 3:51:59.300
filter delay: 0.02585 0.02580 0.02580 0.02585
0.00000 0.00000 0.00000 0.00000
filter offset: 0.005469 0.005474 0.005467 0.005453
0.000000 0.000000 0.000000 0.000000
delay 0.02580, dispersion 0.00000
offset 0.005474
13 Sep 03:51:59 ntpdate[87462]: adjust time server 10.16.64.124 offset 0.005474 sec
We also did some comparisons with the system hardware clock and system time set by ntp by running the following command snippet:
s(){ printf "\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n";}; s; printf '`date`:\t '; date; printf '`sudo hwclock`: ';sudo hwclock; s; printf '`sudo ntpq -pn`:\n\n'; sudo ntpq -pn; s; printf '`sudo ntpq -c as`:\n'; sudo ntpq -c as; for id in $(sudo ntpq -c as | awk '/^ ./{print $2}'); do s; printf "\`sudo ntpq -c \"rv $id\"\`:\n\n"; sudo ntpq -c "rv $id"; done
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `date`: Wed Sep 13 05:23:47 UTC 2017 `sudo hwclock`: Wed 13 Sep 2017 05:23:48 AM UTC -0.343942 seconds ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -pn`: remote refid st t when poll reach delay offset jitter ============================================================================== *10.16.64.124 .GPS. 1 u 14 64 377 0.160 428.657 142.242 +10.20.64.124 .GPS. 1 u 40 64 377 13.456 416.054 136.215 +10.16.64.11 10.16.64.15 3 u 30 64 377 0.214 411.056 141.797 +10.24.64.11 10.16.64.15 3 u 28 64 377 91.317 419.916 141.874 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c as`: ind assid status conf reach auth condition last_event cnt =========================================================== 1 33492 961a yes yes none sys.peer sys_peer 1 2 33493 943a yes yes none candidate sys_peer 3 3 33494 9414 yes yes none candidate reachable 1 4 33495 9414 yes yes none candidate reachable 1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33492"`: associd=33492 status=961a conf, reach, sel_sys.peer, 1 event, sys_peer, srcadr=ntp.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.305, refid=GPS, reftime=dd633f51.48ffa52c Wed, Sep 13 2017 5:23:29.285, rec=dd633f55.f6c4f89f Wed, Sep 13 2017 5:23:33.963, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=50, flash=00 ok, keyid=0, offset=428.657, delay=0.160, dispersion=0.980, jitter=142.242, xleave=0.033, filtdelay= 0.16 0.20 0.18 0.15 0.25 0.23 0.19 0.20, filtoffset= 428.66 397.34 365.10 333.82 302.01 269.84 237.60 205.38, filtdisp= 0.00 1.01 2.04 3.05 4.07 5.10 6.14 7.17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33493"`: associd=33493 status=943a conf, reach, sel_candidate, 3 events, sys_peer, srcadr=mtntime.emrets.net, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.458, refid=GPS, reftime=dd633f2b.eeaa5a6e Wed, Sep 13 2017 5:22:51.932, rec=dd633f3b.031f6da1 Wed, Sep 13 2017 5:23:07.012, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=21, flash=00 ok, keyid=0, offset=416.054, delay=13.456, dispersion=0.947, jitter=136.215, xleave=0.026, filtdelay= 13.46 13.42 13.43 13.45 13.42 13.50 13.50 13.47, filtoffset= 416.05 385.25 355.33 324.97 294.15 264.31 232.99 202.63, filtdisp= 0.00 0.99 1.95 2.93 3.92 4.88 5.88 6.86 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33494"`: associd=33494 status=9414 conf, reach, sel_candidate, 1 event, reachable, srcadr=usstlz-pinfdc15.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=3, precision=-6, rootdelay=62.500, rootdisp=121.170, refid=10.16.64.15, reftime=dd633ccb.357101ca Wed, Sep 13 2017 5:12:43.208, rec=dd633f45.f8454467 Wed, Sep 13 2017 5:23:17.969, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=32, flash=00 ok, keyid=0, offset=411.056, delay=0.214, dispersion=16.513, jitter=141.797, xleave=0.018, filtdelay= 0.21 0.27 0.27 0.22 0.23 0.26 0.27 0.26, filtoffset= 411.06 377.74 348.50 307.74 276.47 246.26 226.04 197.64, filtdisp= 15.63 16.60 17.61 18.57 19.57 20.58 21.57 22.54 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33495"`: associd=33495 status=9414 conf, reach, sel_candidate, 1 event, reachable, srcadr=gblonz-pinfdc07.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=3, precision=-6, rootdelay=109.375, rootdisp=122.223, refid=10.16.64.15, reftime=dd633c4d.a4ae07e5 Wed, Sep 13 2017 5:10:37.643, rec=dd633f47.0cce5653 Wed, Sep 13 2017 5:23:19.050, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=33, flash=00 ok, keyid=0, offset=419.916, delay=91.317, dispersion=16.524, jitter=141.874, xleave=0.010, filtdelay= 91.32 91.61 91.40 91.13 91.36 92.86 91.49 91.38, filtoffset= 419.92 388.51 354.44 316.21 298.64 270.20 226.80 195.30, filtdisp= 15.63 16.62 17.62 18.61 19.60 20.58 21.58 22.57
We disabled slew mode for ntp as shown below by updating the OPTIONS parameter in etc/sysconfig/ntpd file:
# grep -i ntpd ps ntp 1082 0.0 0.0 26520 1980 ? Ss 03:09 0:00 ntpd -x -u ntp:ntp -p /var/run/ntpd.pid -g -4
# grep -v "^#" /etc/sysconfig/ntpd
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g -4"
After that we restarted the service:
# ps -ef | grep -E ^ntp
ntp 59539 1 0 04:48 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g -4
After this we did a stop and start of the service a few times and a manual update with the ntp server to finally fix the high offset problem.
sudo ntpdate -u ntp.nova.org
13 Sep 02:20:49 ntpdate[19667]: step time server 10.16.64.124 offset -42.674761 sec
Just a note that apart from the troubleshooting we did here to fix the high offset, this problem could also be caused by network latency so that is worth checking out.
In the second scenario we were again facing high offsets on one of our linux servers but in this case the time displayed by the hardware clock was accurate.
We did a few stop, start and restart operations along with a manual update with the ntp server but with no success.
[ssuri@usporinfradr00:~] $ sudo nptdate -u ntpls.nova.org
7 Oct 09:13:38 ntpdate[14262]: step time server 10.20.64.124 offset -173.793480 sec
[ssuri@usporinfradr00:~] $ sudo ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*ntp.nova.org .GPS. 1 u 4971 64 1 0.087 -173811 0.000
10.16.64.124 .INIT. 16 u - 64 0 0.000 0.000 0.000
usporz-infrac21 10.16.64.15 3 u 4971 64 1 0.183 -173819 0.000
gblonz-infrac07 10.16.64.15 3 u 4971 64 1 101.189 -173806 0.000
We observed a high offset when we checked the ntpq -p output.
I did a couple of stop-start operations followed by troubleshooting procedure outlined in RedHat KB articles 35640 and 64868 whose logs I'm sharing below:
[ssuri@usporiainfrar00:~]' $ sudo ntpq
ntpq> peers
remote refid st t when poll reach delay offset jitter
==============================================================================
+ntp.nova.org .GPS. 1 u 43 64 377 0.189 309.823 175.931
*mtntime.emrets. .GPS. 1 u 44 64 377 13.401 309.833 174.006
xusporz-infrac15 10.16.64.15 3 u 43 64 377 0.208 286.078 169.984
xgblonz-infrac07 10.16.64.15 3 u 39 64 377 91.365 287.473 167.180
ntpq> as
ind assid status conf reach auth condition last_event cnt
===========================================================
1 57822 941a yes yes none candidate sys_peer 1
2 57823 961a yes yes none sys.peer sys_peer 1
3 57824 9124 yes yes none falsetick reachable 2
4 57825 9124 yes yes none falsetick reachable 2
ntpq> rv 57822
associd=57822 status=941a conf, reach, sel_candidate, 1 event, sys_peer,
srcadr=ntp.nova.org, srcport=123, dstadr=10.16.216.184, dstport=123,
leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.290,
refid=GPS, reftime=dd6328c0.a7684541 Wed, Sep 13 2017 3:47:12.653,
rec=dd6328c3.fc501541 Wed, Sep 13 2017 3:47:15.985, reach=377,
unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=29, flash=00 ok,
keyid=0, offset=504.129, delay=0.171, dispersion=1.525, jitter=183.751,
xleave=0.028,
filtdelay= 0.19 0.17 0.19 0.22 0.21 0.19 0.20 0.21,
filtoffset= 553.93 504.13 454.51 405.18 356.99 309.82 250.70 191.10,
filtdisp= 0.00 0.96 1.94 2.93 3.92 4.91 5.88 6.87
ntpq> q
[ssuri@usporiainfrar00:~]' $ sudo ntpdate -u ntp.nova.org
13 Sep 03:51:50 ntpdate[87098]: step time server 10.16.64.124 offset 0.757074 sec
[ssuri@usporiainfrar00:~]' $ sudo ntpdate -d ntp.nova.org
13 Sep 03:51:59 ntpdate[87462]: ntpdate 4.2.6p5@1.2349-o' Tue May 3 15:12:51 UTC 2016 (1)
Looking for host ntp.nova.org and service ntp
host found : ntp.nova.org
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
transmit(10.16.64.124)
receive(10.16.64.124)
server 10.16.64.124, port 123
stratum 1, precision -20, leap 00, trust 000
refid [GPS], delay 0.02580, dispersion 0.00000
transmitted 4, in filter 4
reference time: dd6329d6.e0896c6a Wed, Sep 13 2017 3:51:50.877
originate timestamp: dd6329df.4e7f4739 Wed, Sep 13 2017 3:51:59.306
transmit timestamp: dd6329df.4d0d781a Wed, Sep 13 2017 3:51:59.300
filter delay: 0.02585 0.02580 0.02580 0.02585
0.00000 0.00000 0.00000 0.00000
filter offset: 0.005469 0.005474 0.005467 0.005453
0.000000 0.000000 0.000000 0.000000
delay 0.02580, dispersion 0.00000
offset 0.005474
13 Sep 03:51:59 ntpdate[87462]: adjust time server 10.16.64.124 offset 0.005474 sec
We also did some comparisons with the system hardware clock and system time set by ntp by running the following command snippet:
s(){ printf "\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n";}; s; printf '`date`:\t '; date; printf '`sudo hwclock`: ';sudo hwclock; s; printf '`sudo ntpq -pn`:\n\n'; sudo ntpq -pn; s; printf '`sudo ntpq -c as`:\n'; sudo ntpq -c as; for id in $(sudo ntpq -c as | awk '/^ ./{print $2}'); do s; printf "\`sudo ntpq -c \"rv $id\"\`:\n\n"; sudo ntpq -c "rv $id"; done
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `date`: Wed Sep 13 05:23:47 UTC 2017 `sudo hwclock`: Wed 13 Sep 2017 05:23:48 AM UTC -0.343942 seconds ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -pn`: remote refid st t when poll reach delay offset jitter ============================================================================== *10.16.64.124 .GPS. 1 u 14 64 377 0.160 428.657 142.242 +10.20.64.124 .GPS. 1 u 40 64 377 13.456 416.054 136.215 +10.16.64.11 10.16.64.15 3 u 30 64 377 0.214 411.056 141.797 +10.24.64.11 10.16.64.15 3 u 28 64 377 91.317 419.916 141.874 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c as`: ind assid status conf reach auth condition last_event cnt =========================================================== 1 33492 961a yes yes none sys.peer sys_peer 1 2 33493 943a yes yes none candidate sys_peer 3 3 33494 9414 yes yes none candidate reachable 1 4 33495 9414 yes yes none candidate reachable 1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33492"`: associd=33492 status=961a conf, reach, sel_sys.peer, 1 event, sys_peer, srcadr=ntp.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.305, refid=GPS, reftime=dd633f51.48ffa52c Wed, Sep 13 2017 5:23:29.285, rec=dd633f55.f6c4f89f Wed, Sep 13 2017 5:23:33.963, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=50, flash=00 ok, keyid=0, offset=428.657, delay=0.160, dispersion=0.980, jitter=142.242, xleave=0.033, filtdelay= 0.16 0.20 0.18 0.15 0.25 0.23 0.19 0.20, filtoffset= 428.66 397.34 365.10 333.82 302.01 269.84 237.60 205.38, filtdisp= 0.00 1.01 2.04 3.05 4.07 5.10 6.14 7.17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33493"`: associd=33493 status=943a conf, reach, sel_candidate, 3 events, sys_peer, srcadr=mtntime.emrets.net, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=1, precision=-20, rootdelay=0.000, rootdisp=0.458, refid=GPS, reftime=dd633f2b.eeaa5a6e Wed, Sep 13 2017 5:22:51.932, rec=dd633f3b.031f6da1 Wed, Sep 13 2017 5:23:07.012, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=21, flash=00 ok, keyid=0, offset=416.054, delay=13.456, dispersion=0.947, jitter=136.215, xleave=0.026, filtdelay= 13.46 13.42 13.43 13.45 13.42 13.50 13.50 13.47, filtoffset= 416.05 385.25 355.33 324.97 294.15 264.31 232.99 202.63, filtdisp= 0.00 0.99 1.95 2.93 3.92 4.88 5.88 6.86 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33494"`: associd=33494 status=9414 conf, reach, sel_candidate, 1 event, reachable, srcadr=usstlz-pinfdc15.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=3, precision=-6, rootdelay=62.500, rootdisp=121.170, refid=10.16.64.15, reftime=dd633ccb.357101ca Wed, Sep 13 2017 5:12:43.208, rec=dd633f45.f8454467 Wed, Sep 13 2017 5:23:17.969, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=32, flash=00 ok, keyid=0, offset=411.056, delay=0.214, dispersion=16.513, jitter=141.797, xleave=0.018, filtdelay= 0.21 0.27 0.27 0.22 0.23 0.26 0.27 0.26, filtoffset= 411.06 377.74 348.50 307.74 276.47 246.26 226.04 197.64, filtdisp= 15.63 16.60 17.61 18.57 19.57 20.58 21.57 22.54 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `sudo ntpq -c "rv 33495"`: associd=33495 status=9414 conf, reach, sel_candidate, 1 event, reachable, srcadr=gblonz-pinfdc07.emrsn.org, srcport=123, dstadr=10.16.216.184, dstport=123, leap=00, stratum=3, precision=-6, rootdelay=109.375, rootdisp=122.223, refid=10.16.64.15, reftime=dd633c4d.a4ae07e5 Wed, Sep 13 2017 5:10:37.643, rec=dd633f47.0cce5653 Wed, Sep 13 2017 5:23:19.050, reach=377, unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=33, flash=00 ok, keyid=0, offset=419.916, delay=91.317, dispersion=16.524, jitter=141.874, xleave=0.010, filtdelay= 91.32 91.61 91.40 91.13 91.36 92.86 91.49 91.38, filtoffset= 419.92 388.51 354.44 316.21 298.64 270.20 226.80 195.30, filtdisp= 15.63 16.62 17.62 18.61 19.60 20.58 21.58 22.57
We disabled slew mode for ntp as shown below by updating the OPTIONS parameter in etc/sysconfig/ntpd file:
# grep -i ntpd ps ntp 1082 0.0 0.0 26520 1980 ? Ss 03:09 0:00 ntpd -x -u ntp:ntp -p /var/run/ntpd.pid -g -4
# grep -v "^#" /etc/sysconfig/ntpd
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g -4"
After that we restarted the service:
# ps -ef | grep -E ^ntp
ntp 59539 1 0 04:48 ? 00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g -4
After this we did a stop and start of the service a few times and a manual update with the ntp server to finally fix the high offset problem.
sudo ntpdate -u ntp.nova.org
13 Sep 02:20:49 ntpdate[19667]: step time server 10.16.64.124 offset -42.674761 sec
Just a note that apart from the troubleshooting we did here to fix the high offset, this problem could also be caused by network latency so that is worth checking out.
In the second scenario we were again facing high offsets on one of our linux servers but in this case the time displayed by the hardware clock was accurate.
We did a few stop, start and restart operations along with a manual update with the ntp server but with no success.
[ssuri@usporinfradr00:~] $ sudo nptdate -u ntpls.nova.org
7 Oct 09:13:38 ntpdate[14262]: step time server 10.20.64.124 offset -173.793480 sec
[ssuri@usporinfradr00:~] $ sudo ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*ntp.nova.org .GPS. 1 u 4971 64 1 0.087 -173811 0.000
10.16.64.124 .INIT. 16 u - 64 0 0.000 0.000 0.000
usporz-infrac21 10.16.64.15 3 u 4971 64 1 0.183 -173819 0.000
gblonz-infrac07 10.16.64.15 3 u 4971 64 1 101.189 -173806 0.000
The system hardware clock was accurate though.
[ssuri@usporinfradr00:~] $ sudo hwclock -r
Sat 07 Oct 2017 09:18:34 AM UTC -0.485089 seconds
[ssuri@usporinfradr00:~] $ date
Sat Oct 7 09:16:05 UTC 2017
We then tried to sync the system and hardware clocks by executing hwclock -s but to our surprise now the hardware clock was also out of sync.
[ssuri@usporinfradr00:~] $ sudo hwclock -s
[ssuri@usporinfradr00:~] $ date
Sat Oct 7 09:20:18 UTC 2017
[root@usporinfradr00 ~]# hwclock -r
Sat 07 Oct 2017 09:20:38 AM UTC -0.265782 seconds
We did a manual update with our ntp server again followed by a restart of the service.
[root@usporinfradr00 ~]# sudo ntpdate -u ntpls.nova.org
7 Oct 09:18:05 ntpdate[88723]: step time server 10.20.64.124 offset -173.991557 sec
[root@usporinfradr00 ~]# sudo ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
ntpserver.nova .GPS. 1 u 4971 64 37 0.076 82.388 123032.
10.16.64.124 .INIT. 16 u - 64 0 0.000 0.000 0.000
usporz-infrac21 10.16.64.15 3 u 4971 64 37 0.161 70.839 123031.
gblonz-infrac07 10.16.64.15 3 u 4971 64 37 101.189 -173806 150569.
[root@usporinfradr00 ~]# sudo ntpq -pservice ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
This finally corrected the offset and brought the system time in sync with the hardware clock as well.
[root@usporinfradr00 ~]# date
Sat Oct 7 09:18:56 UTC 2017
[root@usporinfradr00 ~]# hwclock -r
Sat 07 Oct 2017 09:19:26 AM UTC -0.031508 seconds
[root@usporinfradr00 ~]# date
Sat Oct 7 09:19:30 UTC 2017
[root@usporinfradr00 ~]# sudo ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*ntpls.nova.org .GPS. 1 u 14 64 3 0.081 9.357 32.137
10.16.64.124 .INIT. 16 u - 64 0 0.000 0.000 0.000
usporz-infrac21 10.16.64.15 3 u 9 64 3 0.170 10.491 32.264
gblonz-infrac07 10.16.64.15 3 u 10 64 3 101.359 22.452 32.468
[root@usporinfradr00 ~]#
[root@usporinfradr00 ~]# date
Sat Oct 7 09:20:21 UTC 2017
No comments:
Post a Comment