Unsupervised Continual Learning with Multi-View Data Fusion for Dynamic Network Embedding

Abstract

Dynamic network embedding (DNE) is a tough task in graph representation learning because data arrives in a streaming fashion and without any supervisory signals. Traditional DNE models often resort to parameter updating to cope with such a task, but they suffer from serious catastrophic forgetting upon learning new patterns on incremental data due to being unable to preserve historical knowledge. Recent graph neural networks (GNNs) attempt to tackle the challenge via model retraining and matrix factorization, but they are confronted with inefficient model training and insufficient incremental knowledge learning. Inspired by continual learning, we propose an unsupervised Continual Learning scheme with multi-view data fusion in this paper to overcome catastrophic forgetting in DNE (termed CLDNE). At the heart of CLDNE is a streaming graph auto-encoder that implements multi-view data fusion and captures the global and local features of the input graph. It also equips with an experience replay buffer and a knowledge distillation module. We quantify the capacity of CLDNE through the link prediction task and evaluate the scalability of CLDNE via the node classification task. Extensive experiments elucidate that CLDNE alleviates the catastrophic forgetting problem and shrinks the training time by 80% without a significant loss in incremental knowledge learning.

Keywords: Continual Learning, Multi-view Data Fusion, Dynamic Network Embedding, Unsupervised Learning, Graph Auto-Encoder