Generating English explanations of logical formulas: measuring the quality of the generated sentences

Abstract

Logic languages such as First Order Logic can describe complex ideas, with the downside of being too abstract. In order to make them clearer, it is possible to translate sentences in them to Natural Language. In order to examine this kind of system further, we chose to focus on one domain: Tarski’s World. We created a system that has a First Order Logic formula that has a quantifier as input and an English sentence as output. Our focus was generating a more natural-sounding sentence and explore the quality of the sentence generated. This was possible through the development of two metrics: Naturalness (how natural a sentence is) and Clarity (how grammatically clear a sentence is). Both showed promising results but were ultimately flawed and required further improvements. During the development of the metrics, it became apparent that this type of sentence construction presents scope ambiguity. In order to determine the acceptable readings, a survey was conducted. From the results we concluded a few things: that sentences with adjectives had nocuous ambiguity; quantifiers do not present scope ambiguity; and sentences that contain numbers instead of quantifiers present scope ambiguity.