Subjetividade em correção de redações: detecção autoḿatica através de ĺexico de operadores de víes linguístico

Translated title of the contribution: Subjectivity in essay grading: automatic detection through language bias operator lexicon

Marcia Cancado, Luana Amaral, Evelin Amorim, Adriano Veloso, Heliana Mello

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

Essays are very important assessment tools for Brazilian students. Therefore, it is expected that the grading of these texts will be made with as little subjectivity as possible. However, in an analysis of a sample of grading sheet comments by evaluators, we have noticed a high degree of subjectivity in these texts. From this first analysis, carried manually, we proposed the hypothesis that this genre is more subjective than one would expect. In order to corroborate this hypothesis, we have drawn up a list of linguistic bias markers, divided into four categories: Argumentative operators, presupposition operators, modalization operators, and opinion and value operators. This list was applied to a corpus of essay grading sheet comments by evaluators, using an automatic language bias detection methodology. From this, we were able to quantify the linguistic bias markers present in these texts. These bias markers were also analyzed in two other corpora: Abstracts and product reviews published on internet sales sites. We have compared the percentage of these markers in evaluators' comments with the percentage numbers of these markers in genres admittedly less subjective (abstracts) and admittedly more subjective (reviews). For such comparison, we have used boxplots, a statistical tool widely used in data comparison analysis. Our results indicated that the grading sheets, as for the number of bias markers, are closer to more subjective texts than to less subjective texts. This corroborates our hypothesis and indicates that these grading sheets present a high degree of subjectivity, closer to the degree of a more subjective text. Thus, we conclude that these grading sheets reflect the personal views of the evaluator, deviating from the correction criteria, which raises doubts about considering this genre an exempt and fair assessment instrument.
Translated title of the contributionSubjectivity in essay grading: automatic detection through language bias operator lexicon
Original languagePortuguese
Pages (from-to)63-79
Number of pages17
JournalLinguamatica
Volume12
Issue number1
DOIs
Publication statusPublished - Jun 2020
Externally publishedYes

Keywords

  • Automatic detection
  • Bias operator lexicon
  • Essay grading
  • Subjectivity

Fingerprint

Dive into the research topics of 'Subjectivity in essay grading: automatic detection through language bias operator lexicon'. Together they form a unique fingerprint.

Cite this