Two-Level Attention Mechanism for Heterogenous Graph Embedding

Abstract

This paper introduces a novel method for generating node representations in heterogeneous graphs. The main idea is to construct homogeneous subgraphs using meta-structure and meta-path techniques to extract complex hidden structures of heterogeneous graphs. The method incorporates a node attention mechanism to determine the impacts of neighboring nodes in each subgraph and considers the graph structure when extracting the embeddings. The importance of each subgraph in extracting node representations is determined using another layer of attention mechanism that combines the extracted node representations of the subgraphs with different weights. By employing these techniques, our method method attains enhanced results in generating node representations in heterogeneous graphs. In compare with the currently state-of-the-art embedding methods, the proposed method captures the heterogeneity and semantic information in the graph more comprehensively, leading to improved node representation generation. The effectiveness of our method was evaluated in three different applications, including link prediction, data classification, and clustering. The experimental results demonstrated the superiority of the proposed method in comparison with a set of baseline and state-ofthe-art graph embedding methods.

Keywords: Graph neural network, attention mechanism, Meta-Path, Meta-Structure