Architecture Matters: Investigating the Influence of Differential Privacy on Neural Network Design

https://openreview.net/forum?id=Hp8sjc8LCwO

6 Pages Posted: 14 Feb 2022

See all articles by Felix Morsbach

Felix Morsbach

Karlsruhe Institute of Technology

Tobias Dehling

Karlsruhe Institute of Technology

Ali Sunyaev

University of Cologne; Karlsruhe Institute of Technology

Date Written: December 14, 2021

Abstract

One barrier to more widespread adoption of differentially private neural networks is the entailed accuracy loss. To address this issue, the relationship between neural network architectures and model accuracy under differential privacy constraints needs to be better understood. As a first step, we test whether extant knowledge on architecture design also holds in the differentially private setting. Our findings show that it does not; architectures that perform well without differential privacy, do not necessarily do so with differential privacy. Consequently, extant knowledge on neural network architecture design cannot be seamlessly translated into the differential privacy context. Future research is required to better understand the relationship between neural network architectures and model accuracy to enable better architecture design choices under differential privacy constraints.

Keywords: neural networks, neural network architecture, differential privacy, privacy-preserving machine learning

Suggested Citation

Morsbach, Felix and Dehling, Tobias and Sunyaev, Ali, Architecture Matters: Investigating the Influence of Differential Privacy on Neural Network Design (December 14, 2021). https://openreview.net/forum?id=Hp8sjc8LCwO, Available at SSRN: https://ssrn.com/abstract=3978767

Felix Morsbach (Contact Author)

Karlsruhe Institute of Technology ( email )

Kaiserstraße 12
Karlsruhe, Baden Württemberg 76131
Germany

Tobias Dehling

Karlsruhe Institute of Technology ( email )

Kaiserstraße 12
Karlsruhe, Baden Württemberg 76131
Germany

Ali Sunyaev

University of Cologne ( email )

Albertus-Magnus-Platz
Cologne, 50923
Germany

HOME PAGE: http://www.isq.uni-koeln.de

Karlsruhe Institute of Technology ( email )

Kaiserstraße 12
Karlsruhe, Baden Württemberg 76131
Germany

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
9
Abstract Views
146
PlumX Metrics