Fusing Semantic Information for Syntax-Guided Paraphrase Generation

Abstract

Syntax-guided paraphrase generation (SGPG) refers to generating a paraphrase sentence that satisfies the given syntactic structure without changing the source sentences’ semantics. The commonly utilized syntactic structures are part-of-speech (POS) sequence, constituency parse tree, and masked template, with constituency parse tree achieving State-of-The-Art (SOTA) performance because of its rich syntactic information. As a result, further mining of syntactic information in parse trees has grown popular, yet fewer works pay attention to investigating semantic information in source sentences. A sentence is made up of two parts: syntax and semantics. Multiple studies have shown that improving the model’s ability to learn semantic information is critical for paraphrase construction as well as syntax learning. In this paper, we propose Fusing Semantic Information for Syntax-guided Paraphrase Generation (FS-SPG). Specifically, we propose a transformer-based semantic encoder to obtain detailed semantics from source sentences. This encoder contains a Semantics-Aware Attention mechanism for mining semantic information. In addition, we apply contrastive learning to improve the accuracy of parse tree nodes’ guidance to semantic sentences. Experiments on ParaNMT and QQP-Pos show that our model beats the SOTA model SI-SCP by 4.92% in syntactic metrics and 1.35% in semantic metrics.

Keywords: Paraphrase Generation, contrastive learning, Semantics and Syntax, Transformer