Improving Explainability of Sentence-level Metrics via Edit-level Attribution for Grammatical Error Correction

Key Points

• The aim is to enhance the explainability of sentence-level metrics in grammatical error correction by attributing scores to individual edits.
• Proposed a method to attribute metric scores to specific edits using cooperative game theory concepts such as the Shapley value.
• Conducted experiments to assess the utility of attribution for fidelity and explainability.
• Evaluated the agreement of attributed scores with human judgments on edit quality.
• Demonstrated that the proposed attribution method achieves about 70% agreement with human assessments of edit quality.
• Confirmed that score attribution allows for finer explanations at both sentence and corpus levels.
• Revealed biases in scoring related to the characteristics of edits.