Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman; Hande Celikkanat; Sami Virpioja; Markus Heinonen; Jörg Tiedemann

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman, Hande Celikkanat, Sami Virpioja, Markus Heinonen, Jörg Tiedemann

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

Abstrakti

This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

Alkuperäiskieli	englanti
Otsikko	Proceedings of the 24th Nordic Conference on Computational Linguistics
Toimittajat	Tanel Alumäe , Mark Fishel
Sivumäärä	8
Julkaisupaikka	Tartu
Kustantaja	University of Tartu Library
Julkaisupäivä	2023
Sivut	358-365
ISBN (elektroninen)	978-99-1621-999-7
Tila	Julkaistu - 2023
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisuussa
Tapahtuma	Nordic Conference on Computational Linguistics - Tórshavn, Färsaaret Kesto: 22 toukok. 2023 → 24 toukok. 2023 Konferenssinumero: 24

Julkaisusarja

Nimi	NEALT Proceedings Series
Kustantaja	University of Tartu Library
Numero	52
ISSN (painettu)	1736-8197
ISSN (elektroninen)	1736-6305

Tieteenalat

6121 Kielitieteet
113 Tietojenkäsittely- ja informaatiotieteet

Pääsy asiakirjaan

2023.nodalida-1.37Lopullinen julkaistu versio, 338 KBLisenssi: CC BY

Muut tiedostot ja linkit

https://aclanthology.org/2023.nodalida-1.37

Uncertainty-aware neural language models
Tiedemann, J., Celikkanat, H., Virpioja, S. P. & Vazquez , R.
Academy of Finland, Suomen Akatemia Projektilaskutus
01/01/2022 → 01/10/2025
Projekti: Tutkimusprojekti
FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding
Tiedemann, J., Celikkanat, H., Raganato, A., Silfverberg, M., Sulubacak, U., Vazquez , R., Apidianaki, M., Attieh, J., Aulamo, M., Boggia, M., Celikkanat, H., De Gibert Bonet, O., Grönroos, S., Mickus, T., Raganato, A., Scherrer, Y., Silfverberg, M., Sjöblom, E. I., Talman, A., Vazquez , R., Virpioja, S. P., Yli-Jyrä, A. & Zosa, E.
01/09/2018 → 29/02/2024
Projekti: EU Horizon 2020: European Research Council: Consolidator Grant (H2020-ERC-COG)

Siteeraa tätä

@inproceedings{8baa06ac175b4d1697c19025bc05a5bf,

title = "Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging",

abstract = "This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.",

keywords = "6121 Languages, 113 Computer and information sciences",

author = "Aarne Talman and Hande Celikkanat and Sami Virpioja and Markus Heinonen and J{\"o}rg Tiedemann",

year = "2023",

language = "English",

series = "NEALT Proceedings Series",

publisher = "University of Tartu Library",

number = "52",

pages = "358--365",

editor = "{Alum{\"a}e }, {Tanel } and Fishel, {Mark }",

booktitle = "Proceedings of the 24th Nordic Conference on Computational Linguistics",

address = "Estonia",

note = "Nordic Conference on Computational Linguistics , NoDaLiDa 2023 ; Conference date: 22-05-2023 Through 24-05-2023",

}

Talman, A , Celikkanat, H , Virpioja, S, Heinonen, M & Tiedemann, J 2023, Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. julkaisussa T Alumäe & M Fishel (toim), Proceedings of the 24th Nordic Conference on Computational Linguistics. NEALT Proceedings Series, Nro 52, University of Tartu Library, Tartu, Sivut 358-365, Nordic Conference on Computational Linguistics , Tórshavn, Färsaaret, 22/05/2023.

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. / Talman, Aarne ; Celikkanat, Hande ; Virpioja, Sami et al.

Proceedings of the 24th Nordic Conference on Computational Linguistics. toim. / Tanel Alumäe ; Mark Fishel. Tartu : University of Tartu Library, 2023. s. 358-365 (NEALT Proceedings Series; Nro 52).

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

TY - GEN

T1 - Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

AU - Talman, Aarne

AU - Celikkanat, Hande

AU - Virpioja, Sami

AU - Heinonen, Markus

AU - Tiedemann, Jörg

N1 - Conference code: 24

PY - 2023

Y1 - 2023

N2 - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

AB - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

KW - 6121 Languages

KW - 113 Computer and information sciences

UR - https://aclanthology.org/2023.nodalida-1.37

M3 - Conference contribution

T3 - NEALT Proceedings Series

SP - 358

EP - 365

BT - Proceedings of the 24th Nordic Conference on Computational Linguistics

A2 - Alumäe , Tanel

A2 - Fishel, Mark

PB - University of Tartu Library

CY - Tartu

T2 - Nordic Conference on Computational Linguistics

Y2 - 22 May 2023 through 24 May 2023

ER -

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Abstrakti

Julkaisusarja

Tieteenalat

Pääsy asiakirjaan

Muut tiedostot ja linkit

Projektit

Uncertainty-aware neural language models

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

Siteeraa tätä