University of Sussex
Browse

File(s) not publicly available

An Empirical Study of Errors in Translating Natural Language into Logic

presentation
posted on 2023-06-08, 09:56 authored by Dave Barker-Plummer, Richard Cox, Robert Dale, John Etchemendy
Every teacher of logic knows that the ease with which a student can translate a natural language sentence into formal logic depends, amongst other things, on just how that natural language sentence is phrased. This paper reports findings from a pilot study of a large scale corpus in the area of formal logic education, where we used a very large dataset to provide empirical evidence for specific characteristics of natural language problem statements that frequently lead to students making mistakes. We developed a rich taxonomy of the types of errors that students make, and implemented tools for automatically classifying student errors into these categories. In this paper, we focus on three specific phenomena that were prevalent in our data: Students were found (a) to have particular difficulties with distinguishing the conditional from the biconditional, (b) to be sensitive to word-order effects during translation, and (c) to be sensitive to factors associated with the naming of constants. We conclude by considering the implications of this kind of large-scale empirical study for improving an automated assessment system specifically, and logic teaching more generally.

History

Publication status

  • Published

Publisher

Cognitive Science Society

Pages

6.0

Presentation Type

  • paper

Event name

Proceedings of the 30th Annual Conference of the Cognitive Science Society

Event location

Washington, DC

Event type

conference

ISBN

978-0-976-83184-6

Department affiliated with

  • Informatics Publications

Full text available

  • No

Peer reviewed?

  • Yes

Editors

BC Love, VM Sloutsky, K McRae

Legacy Posted Date

2012-02-06

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC