On some abusive NTP clients

You have been referred to this document for one of a number of reasons.

You are responsible for a machine or machines connected to the Internet appear to be behaving badly.
You are responsible for a network on which there are hosts that are behaving badly.
You are an ISP or ASP who is being nagged to provide NTP service for your customers.
You are ultimately responsible for some site which uses broken software. That is, you are the boss of of one of the people listed above.
You provide NTP services to the public and some of your clients are behaving badly, and would like to see what others are doing about it.
You are just curious.

Although this document should be read by all of those people listed above, its language is directed to the the site or network administrator with the abusive NTP client.

NTP overview

Everything you need to know about NTP can be found at www.ntp.org and ntp.isc.org I'm providing a quick (and far from complete) overview of the bits of NTP needed for this document.

It's about time

NTP is designed to keep various computers in time synchronization with each other. It is nice, for example, to have all of the machines in an organization agree on what time it is.
NTP is not only designed to keep machines in sync with each other but also with the correct time.
Most importantly, NTP is designed to do this in a some decentralized (but hierarchical) was so that burden of providing accurate time information is shared.
NTP is robust. It can keep times correctly in sync even if there are some servers providing incorrect time or through network outages.

Hierarchy

An NTP server is like a web-server: It answers NTP requests. An NTP client is like a web browser it makes NTP requests. But unlike how the WWW works, all (almost) NTP servers are also clients. An NTP server that is providing time information to clients also is itself a client which requests time information from other servers (which in turn are probably clients as well).

The only NTP servers that aren't also clients are those that are directly connected to a reference clock, such as a cesium atomic clock or a GPS device. Those servers are at the top of the hierarchy and are called stratum 0 servers. Stratum 0 servers are not publicly available on the Internet. NTP servers/clients which get their time from stratum 0 servers are called stratum 1 servers. There are a few dozen public NTP stratum 1 servers. Likewise, those that get their time from stratum 1 servers are called stratum 2 servers. Again there are only a few hundred publicly available stratum 2 servers.

NTP is designed to take into account network delays and jittery clocks, so even a stratum 7 or 8 server should be within a few tens of milliseconds of the correct time.

Hierarchy and Organization

Suppose you run an organization with about 100 computers, all of which you (very correctly) want to have set to the correct time. Should all 100 of those machines be making repeated NTP requests to public NTP servers? Or course not. The organization should just run one (or a very few) NTP servers which get their time from the public Internet, and then use those servers to serve time information for all of the machines within the organization. But setting up a hierarchy this way, the organization will keep just as accurate time, but will reduce their network traffic. Furthermore they will reduce the burden on the public service provided by those who run public NTP servers. Even for a smaller organization this makes sense. It also has the advantage of keeping clocks in sync with each other even if the connection to the Internet fails.

Many ISPs do (and all should) provide NTP servers for their customers. ISPs will already be running NTP servers to keep their own machines in sync. By making NTP servers available to their customers they will reduce their network traffic. The end user will probably get better time (because of reduced network delay) by using their ISPs NTP server than by using a low stratum server that has to pass through much more network to reach.

Peering

In addition to having a client and server relationship between two NTP servers, they can also be set up as peers. Having peers within an organization can help keep clocks even more accurate without adding any external network traffic.

Polling intervals

NTP clients send requests to servers at regular intervals. Under normal configurations these range from about one per minute to about once every 17 minutes. NTP clients are very smart and will adjust the polling intervals as the client gathers more information. Before the client has had a chance to study the accuracy of the internal clock on the machine it is running on it may need to poll more frequently (once per minute or so). But as things settle into a pattern the polling intervals will grow longer and longer while still keeping very accurate time.

For example, as I write this, ntp0.goldmark.org is polling three lower stratum (higher on the hierarchy) servers every 68 minutes. Still my system estimates that its time is within 30 milliseconds of the time on the reference clocks. Furthermore, I would be polling even less frequently if I weren't running a public NTP server. The bottom line is that you can get all of the clock accuracy you could ever want with a polling rate of several requests per day instead of several per hour or several per minute.

The abuse

Few people would deliberately engage in abusive practices. There is nothing to gain from it. Abusive practices waste the bandwidth of the abusing networks at least as much as it wastes things for the server. It generally falls into two types. (1) Failure to use the hierarchical structure (so many machines within an organization or a network each querying external servers. (2) Setting polling intervals that are far to short.

Consequences

NTP is an extremely lightweight service. My server is handling on average 17 requests per second. The load that this has put on my small machine is far lower than the script that is running to collect those statistics. So NTP abuse is unlikely to bring down an NTP server.

However, NTP abuse has brought down networks on which the server sits. NTP abuse results in a pattern of traffic to a host or a network that looks like and has similar effects as a DDoS attack. An enormous number of requests coming at great frequency from many parts of the Internet can bring networks to a standstill. Several people have withdrawn their NTP servers from the pool for exactly this reason.

There is other (though less dramatic) harm caused by abuse NTP clients, but the bottom line is that the success of the pool project depends on volunteers willing to dedicate some network, machine and personal resources (time for managing things) to the effort. If there were no NTP abuse the number of current voluntary servers would easily be sufficient, but with the abuse, some are forced to drop out making matters worse for everybody.

So please reconfigure your NTP clients appropriately. Once you have done that, consider contribuing an NTP server to the pool. Instructions can be found at http://www.pool.ntp.org/.

Email template

Occasionally I will block (at my firewall) NTP queries from networks that I find abusive. If I can find an appropriate address to email a notice to, this is the template for the mail that I send:

To: abuse@[DOMAIN_OF_ABUSIVE_IP]
Subject: NTP abuse from [ABUSIVE_IP]
Hello,

I operate one of the many pool.ntp.org NTP servers.  Mine is
ntp0.goldmark.org.  An IP on your network, [ABUSIVE_IP], is making an
inordinate number of requests.  It has made more than [TOTAL_REQUESTS]
requests at rates of [RATE]

This is a clear sign of poor NTP configuration on (or behind) that IP
address.  Please see

  http://www.goldmark.org/netrants/ntp-abuse/

for details.

I have started to block NTP requests to my server from [BLOCKED_NET]
(This won't effect other servers in pool.ntp.org because, like mine,
they are all run by different volunteers on different parts of the
network.)

I will remove the blocking as soon as I hear that you are going to look
into the problem.

Thank you

-j

-- 
Jeffrey Goldberg                        http://www.goldmark.org/jeff/

Version: $Revision: 1.8 $
Last Modified: $Date: 2007/03/29 16:13:25 $ UTC
First established: 2007/02/28
Author: Jeffrey Goldberg