Usability of Broadband Performance Measurements for Statistical Surveys

28 Pages Posted: 3 Apr 2012 Last revised: 31 Oct 2012

See all articles by Hendrik Rood

Hendrik Rood

Stratix BV

David Yoshikawa

Stratix Consulting BV

René Wevers

Stratix Consulting BV

René Post

iPing Research BV

Arend-Jan Tetteroo

iPing Research BV

Andries Kuipers

Statistics Netherlands

Rik Schurmann

Statistics Netherlands

Date Written: March 31, 2012

Abstract

This paper addresses the usability of a broadband performance measurement application for end users, as introduced in Europe since 2003 by iPing Research B.V. The small Application, executes various automated performance measurements at regular intervals in the background of a Laptop/PC and data was acquired on bandwidth, price, technology, location, as well as voluntarily supplied gender and age of the App-installer. Panels were acquired in cooperation with Consumer Associations and by general magazine publicity (self-selection), and via a market research bureau (a-select panel).

The dataset for the Netherlands for 2003-2010 is compared with data acquired via household surveys by Statistics Netherlands (CBS) and regulatory filings of ISPs to Dutch regulator OPTA. We evaluated the representativeness for the self-select and a-select panels from iPing Research by comparing them with the results for a-select CBS household survey panels on geography, technology, gender and age as well as broadband capacities from the OPTA filings.

Using Chi-square testing for geography and technology, we had to reject the hypothesis that iPing and CBS data measure the same population for iPing’s self-select panels. Instead iPing’s a-select panel was within 95% confidence limits of CBS. It was found that in the self-select panels users of higher bitrate offerings are overrepresented, as well as males and persons aged 30 – 60. However, there was no significant difference in gender for average measured bitrates.

A comparison of these outcomes and performance measurements results from Speedtest.net and Google M-Labs tools for the Netherlands, reveal implications for proper panel selection that extends to hardware based performance measurements now operational in the UK, USA and EU, as they also use volunteers and risk the found self-selection bias.

The recommendation for good statistical data quality on broadband is joint use of a performance measurement tool in combination with a survey among an a-select drawn panel. This allows for the actual broadband access in use to be verified through measurement as part of the survey answer and then a proper weighing of the panel data. Software tools included in an online survey have attractions in cost and less intrusiveness in comparison to hardware.

Funding for this research has been provided by the Statistics Netherlands program Impact ICT and the Netherlands Ministry of Economic affairs, Agriculture and Innovation.

Keywords: Broadband, Performance Measurements, Statistics, Panel Selection, representativeness, usability

JEL Classification: C12, C23, C42, D12

Suggested Citation

Rood, Hendrik and Yoshikawa, David and Wevers, René and Post, René and Tetteroo, Arend-Jan and Kuipers, Andries and Schurmann, Rik, Usability of Broadband Performance Measurements for Statistical Surveys (March 31, 2012). 2012 TRPC, Available at SSRN: https://ssrn.com/abstract=2032598 or http://dx.doi.org/10.2139/ssrn.2032598

Hendrik Rood (Contact Author)

Stratix BV ( email )

Utrechtseweg 29
Hilversum, North Holland 1213TK
Netherlands

David Yoshikawa

Stratix Consulting BV ( email )

Utrechtseweg 29
Hilversum, North Holland 1213TK
Netherlands

René Wevers

Stratix Consulting BV ( email )

Utrechtseweg 29
Hilversum, North Holland 1213TK
Netherlands

René Post

iPing Research BV ( email )

Julianastraat 9
Leiderdorp, 2351 GV
Netherlands

Arend-Jan Tetteroo

iPing Research BV ( email )

Julianastraat 9
Leiderdorp, 2351 GV
Netherlands

Andries Kuipers

Statistics Netherlands ( email )

Henri Faasdreef 312
The Hague, 2492 JP
Netherlands

Rik Schurmann

Statistics Netherlands ( email )

Henri Faasdreef 312
The Hague, 2492 JP
Netherlands

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
95
Abstract Views
1,853
Rank
493,626
PlumX Metrics