University of Sussex
Browse
EDM12.pdf (396.05 kB)

Using edit distance to analyse errors in a natural language to logic translation corpus

Download (396.05 kB)
presentation
posted on 2023-06-08, 12:17 authored by Dave Barker-Plummer, Robert Dale, Richard Cox
We have assembled a large corpus of student submissions to an automatic grading system, where the subject matter involves the translation of natural language sentences into propositional logic. Of the 2.3 million translation instances in the corpus, 286,000 (approximately 12%) are categorized as being in error. We want to understand the nature of the errors that students make, so that we can develop tools and supporting infrastructure that help students with the problems that these errors represent. With this aim in mind, this paper describes an analysis of a significant proportion of the data, using edit distance between incorrect answers and their corresponding correct solutions, and the associated edit sequences, as a means of organising the data and detecting categories of errors. We demonstrate that a large proportion of errors can be accounted for by means of a small number of relatively simple error types, and that the method draws attention to interesting phenomena in the data set.

History

Publication status

  • Published

Page range

134-141

Presentation Type

  • paper

Event name

The 5th International Conference on Educational Data Mining (EDM 2012)

Event location

Chania, Greece

Event type

conference

Event date

19-21st June

Department affiliated with

  • Informatics Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date

2012-08-22

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC