Depth-2 Neural Networks Under a Data-Poisoning Attack

32 Pages Posted: 2 Aug 2022

See all articles by Anirbit Mukherjee

Anirbit Mukherjee

Wharton (UPenn), Department of Statistics

Sayar Karmakar

University of Florida

Theodore Papamarkou

The University of Manchester

Abstract

In this work, we study the possibility of defending against data-poisoning attacks while training ashallow neural network in a regression setup. We focus on doing supervised learning for a class ofdepth-2 finite-width neural networks, which includes single-filter convolutional networks. In thisclass of networks, we attempt to learn the network weights in the presence of a malicious oracle doingstochastic, bounded and additive adversarial distortions on the true output during training. Forthe non-gradient stochastic algorithm that we construct, we prove worst-case near-optimal trade-offs among the magnitude of the adversarial attack, the weight approximation accuracy, and theconfidence achieved by the proposed algorithm. As our algorithm uses mini-batching, we analyzehow the mini-batch size affects convergence. We also show how to utilize the scaling of the outerlayer weights to counter output-poisoning attacks depending on the probability of attack. Lastly,we give experimental evidence demonstrating how our algorithm outperforms stochastic gradientdescent under different input data distributions, including instances of heavy-tailed distributions.

Keywords: Convolutional Neural Networks, stochastic algorithms, data poisoning, robust regression

Suggested Citation

Mukherjee, Anirbit and Karmakar, Sayar and Papamarkou, Theodore, Depth-2 Neural Networks Under a Data-Poisoning Attack. Available at SSRN: https://ssrn.com/abstract=4179863

Anirbit Mukherjee (Contact Author)

Wharton (UPenn), Department of Statistics ( email )

Wharton School
Philadelphia, PA 19104
United States

HOME PAGE: http://https://sites.google.com/view/anirbit/home

Sayar Karmakar

University of Florida ( email )

PO Box 117165, 201 Stuzin Hall
Gainesville, FL 32610-0496
United States

Theodore Papamarkou

The University of Manchester ( email )

United Kingdom

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
18
Abstract Views
99
PlumX Metrics