Measurement, Meaning and Purpose: Exploring the M-Lab NDT Dataset
44 Pages Posted: 3 Aug 2021 Last revised: 11 Aug 2021
Date Written: August 2, 2021
The speed of a data transfer over the Internet connection is a measure of great interest. It can directly influence quality of experience, it can serve as a measure of equitable access, and it can reveal whether or not the speed matches that which was advertised. One of earliest and longest-standing tools to contain a speed test is the Network Diagnostic Tool (NDT), currently supported by Measurement Lab (M-Lab). NDT, in its various forms, has been used across the globe since 2003. There are billions of measurements archived by M-Lab, with a rich collection of metadata for each measurement. This data allows an in-depth analysis of each measurement, and potentially supports analysis across aggregates of measurements.
The original purpose of NDT was diagnostic: why is my connection operating as it is? However, the archive of this data invites its use in aggregate form to draw conclusions about the overall behavior of the Internet. Such use, however, is confounded by the fact that the individual measurements are triggered by users for a range of reasons: simple curiosity, debugging, anger, bragging rights, or automated checking of operational status (in some cases as often as once a minute). NDT measures network speed, but it equally - if indirectly - measures human behavior.
The goals of this paper are to explore the archived NDT data in order to provide insights about how it can be interpreted; and to distinguish between appropriate uses of the data from uses that may lead to unwarranted conclusions.
Keywords: Internet measurement, Internet speed test, network diagnostic tool, measurement lab
Suggested Citation: Suggested Citation