Automatic Evaluation of Sentence Simplification
I present the first meta-evaluation of automatic metrics for Automatic Sentence Simplification focused on simplicity.
I present the first meta-evaluation of automatic metrics for Automatic Sentence Simplification focused on simplicity.
I provide an overview of how automatically-simplified texts are currently evaluated, highlighting issues with resources and metrics for automatic evaluation, and proposing solutions based on recent work.
I discuss how evaluating automatic simplifications is difficult, since we do not have metrics that measure simplicity with high confidence, nor human judgements majorly agree in what simple is.
I present work that attempts to incorporate the multi-operation nature of Text Simplication into automatic models and evaluation resources.
I present the results of the first meta-evaluation of automatic metrics for Sentence Simplification, focused on simplicity judgements.
I explain the limitations of evaluation resources for Text Simplification when assessing multi-operation simplifications, and present some of our latest work on overcoming them.
I review our results on a manual analysis of professionally-produced simplifications, which sheds light on the transformation operations that humans perform at the document-level.