Learning and Leveraging Generalizable Knowledge of 2d Transformations

Abstract

The existing deep learning models suffer from out-of-distribution ( o.o.d. ) performance drop in computer vision tasks. In comparison, humans have a remarkable ability to interpret images, even if the scenes in the images are rare, thanks to the generalizability of acquired knowledge. This work focuses on the acquisition and utilization of generalizable knowledge about 2D transformations. We demonstrate that deep neural networks can learn generalizable knowledge with a new training methodology based on synthetic datasets. The generalizability is reflected in the results that, even when the knowledge is learned from random noise, the networks can still achieve stable performance in parameter estimation tasks. Furthermore, to demonstrate that the learned knowledge can be leveraged for image classification tasks, a novel architecture called "InterpretNet" is devised, and it consists of an Estimator and an Identifier, in addition to a Classifier. By emulating the "hypothesis-verification" process in human visual perception, our InterpretNet improves the classification accuracy significantly on test sets under covariate shift.

Keywords: Deep learning, Knowledge Acquisition, O.O.D. Generalization, Explainability, Computer Vision