Rouge Metric Evaluation for Text Summarization Techniques

31 Pages Posted: 26 May 2022

See all articles by Marcello Barbella

Marcello Barbella

affiliation not provided to SSRN

Genoveffa Tortora

University of Salerno

Abstract

Approaches to Automatic Text Summarization try to extract key information from one or more input texts and generate summaries whilst preserving content meaning. These strategies are separated into two groups: Extractive and Abstractive, which differ in terms of how they work. The former extracts sentences from the document text directly, whereas the latter creates a summary by interpreting the text and rewriting sentences often with new words. So, it is important to assess and confirm how similar a summary is to the original text. The question is: how can the quality of these methodologies of summaries be evaluated? For the evaluation of text summarization results, various metrics and scores have been proposed in the literature, but the most used is ROUGE.Then, the primary goal of this study is to accurately estimate the ROUGE metric's behavior. We conducted a first experiment to compare the metric efficiency for evaluating Abstractive versus Extractive Text Summarization algorithms, and a second one to compare the obtained score for two different summary approaches: a simple execution of a summarization algorithm versus a multiple execution of different algorithms on the same text. Our results have shown that: ROUGE does not obtain impressive results since it behaves in a similar way both on the Abstractive and Extractive algorithms; furthermore, in most cases, a multiple execution is better than a single one.

Keywords: Automatic Text Summarization Algorithms, Extractive, Abstractive, ROUGE metric, Bert

Suggested Citation

Barbella, Marcello and Tortora, Genoveffa, Rouge Metric Evaluation for Text Summarization Techniques. Available at SSRN: https://ssrn.com/abstract=4120317 or http://dx.doi.org/10.2139/ssrn.4120317

Marcello Barbella (Contact Author)

affiliation not provided to SSRN ( email )

No Address Available

Genoveffa Tortora

University of Salerno ( email )

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
662
Abstract Views
1,417
Rank
77,955
PlumX Metrics