Modeling Noise in Paraphrase Detection

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussaKonferenssiartikkeliTieteellinenvertaisarvioitu


Noisy labels in training data present a challenging issue in classification tasks, misleading a model towards incorrect decisions during training. In this paper, we propose the use of a linear noise model to augment pre-trained language models to account for label noise in fine-tuning. We test our approach in a paraphrase detection task with various levels of noise and five different languages. Our experiments demonstrate the effectiveness of the additional noise model in making the training procedures more robust and stable. Furthermore, we show that this model can be applied without further knowledge about annotation confidence and reliability of individual training examples and we analyse our results in light of data selection and sampling strategies.
OtsikkoProceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022)
ToimittajatNicoletta Calzolari, Frédéric Béchet, Philippe Blache, et al.
KustantajaEuropean Language Resources Association (ELRA)
Julkaisupäivä20 kesäk. 2022
ISBN (elektroninen)979-10-95546-72-6
TilaJulkaistu - 20 kesäk. 2022
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Conference on Language Resources and Evaluation - Marseille, Ranska
Kesto: 20 kesäk. 202225 kesäk. 2022
Konferenssinumero: 13


  • 6121 Kielitieteet
  • 113 Tietojenkäsittely- ja informaatiotieteet

Siteeraa tätä