Evaluation Measures for Text Summarization

Authors

  • Josef Steinberger
  • Karel Ježek

Abstract

We explain the ideas of automatic text summarization approaches and the taxonomy of summary evaluation methods. Moreover, we propose a new evaluation measure for assessing the quality of a summary. The core of the measure is covered by Latent Semantic Analysis (LSA) which can capture the main topics of a document. The summarization systems are ranked according to the similarity of the main topics of their summaries and their reference documents. Results show a high correlation between human rankings and the LSA-based evaluation measure. The measure is designed to compare a summary with its full text. It can compare a summary with a human written abstract as well; however, in this case using a standard ROUGE measure gives more precise results. Nevertheless, if abstracts are not available for a given corpus, using the LSA-based measure is an appropriate choice.

Downloads

Download data is not yet available.

Downloads

Published

2012-01-26

How to Cite

Steinberger, J., & Ježek, K. (2012). Evaluation Measures for Text Summarization. COMPUTING AND INFORMATICS, 28(2), 251–275. Retrieved from https://www.cai.sk/ojs/index.php/cai/article/view/37