Automatic Evaluation of Sentence Simplification
I present the first meta-evaluation of automatic metrics for Automatic Sentence Simplification focused on simplicity.
I am a Lecturer (~Assistant Professor) at the School of Computer Science and Informatics at Cardiff University. My research focuses on technologies that apply Artificial Intelligence for education and information accessibility. In particular, my work employs Natural Language Processing approaches to facilitate reading and understanding. I am especially interested in studying the real capabilities of systems for several Natural Language Generation tasks, such as Text Simplification, Summarisation and Machine Translation. In order to do that, my collaborators and I create language resources, design evaluation methodologies or metrics, and implement models using machine learning techniques.
Previously, I was a Research Associate at SheffieldNLP (2020-2021), working with Prof. Lucia Specia for the APE-QUEST and Bergamot projects on Quality Estimation for Machine Translation. Before that, I worked as Adjunct Professor at the Pontifical Catholic University of Peru (2013-2016), where I was a member of the Artificial Intelligence Group IA-PUCP. During my Masters, I was also a member of the Interinstitutional Center for Computational Linguistics at the University of São Paulo.
PhD in Computer Science
University of Sheffield
MSc in Computer Science
University of Sao Paulo
BSc in Informatics Engineering
Pontifical Catholic University of Peru
I present the first meta-evaluation of automatic metrics for Automatic Sentence Simplification focused on simplicity.
I provide an overview of how automatically-simplified texts are currently evaluated, highlighting issues with resources and metrics for automatic evaluation, and proposing solutions based on recent work.
I discuss how evaluating automatic simplifications is difficult, since we do not have metrics that measure simplicity with high confidence, nor human judgements majorly agree in what simple is.
I present work that attempts to incorporate the multi-operation nature of Text Simplication into automatic models and evaluation resources.
I present the results of the first meta-evaluation of automatic metrics for Sentence Simplification, focused on simplicity judgements.