Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman; Hande Celikkanat; Sami Virpioja; Markus Heinonen; Jörg Tiedemann

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman, Hande Celikkanat, Sami Virpioja, Markus Heinonen, Jörg Tiedemann

Forskningsoutput: Kapitel i bok/rapport/konferenshandling › Konferensbidrag › Vetenskaplig › Peer review

Sammanfattning

This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

Originalspråk	engelska
Titel på värdpublikation	Proceedings of the 24th Nordic Conference on Computational Linguistics
Redaktörer	Tanel Alumäe , Mark Fishel
Antal sidor	8
Utgivningsort	Tartu
Förlag	University of Tartu Library
Utgivningsdatum	2023
Sidor	358-365
ISBN (elektroniskt)	978-99-1621-999-7
Status	Publicerad - 2023
MoE-publikationstyp	A4 Artikel i en konferenspublikation
Evenemang	Nordic Conference on Computational Linguistics - Tórshavn, Färöarna Varaktighet: 22 maj 2023 → 24 maj 2023 Konferensnummer: 24

Publikationsserier

Namn	NEALT Proceedings Series
Förlag	University of Tartu Library
Nummer	52
ISSN (tryckt)	1736-8197
ISSN (elektroniskt)	1736-6305

Vetenskapsgrenar

6121 Språkvetenskaper
113 Data- och informationsvetenskap

Åtkomst av dokument

2023.nodalida-1.37Slutlig publicerad version, 338 KBLicens: CC BY

Andra filer och länkar

https://aclanthology.org/2023.nodalida-1.37

Uncertainty-aware neural language models
Tiedemann, J., Celikkanat, H., Virpioja, S. P. & Vazquez , R.
Academy of Finland, Suomen Akatemia Projektilaskutus
01/01/2022 → 01/10/2025
Projekt: Forskningsprojekt
FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding
Tiedemann, J., Celikkanat, H., Raganato, A., Silfverberg, M., Sulubacak, U., Vazquez , R., Apidianaki, M., Attieh, J., Aulamo, M., Boggia, M., Celikkanat, H., De Gibert Bonet, O., Grönroos, S., Mickus, T., Raganato, A., Scherrer, Y., Silfverberg, M., Sjöblom, E. I., Talman, A., Vazquez , R., Virpioja, S. P., Yli-Jyrä, A. & Zosa, E.
01/09/2018 → 29/02/2024
Projekt: EU Horizon 2020: European Research Council: Consolidator Grant (H2020-ERC-COG)

Citera det här

@inproceedings{8baa06ac175b4d1697c19025bc05a5bf,

title = "Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging",

abstract = "This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.",

keywords = "6121 Languages, 113 Computer and information sciences",

author = "Aarne Talman and Hande Celikkanat and Sami Virpioja and Markus Heinonen and J{\"o}rg Tiedemann",

year = "2023",

language = "English",

series = "NEALT Proceedings Series",

publisher = "University of Tartu Library",

number = "52",

pages = "358--365",

editor = "{Alum{\"a}e }, {Tanel } and Fishel, {Mark }",

booktitle = "Proceedings of the 24th Nordic Conference on Computational Linguistics",

address = "Estonia",

note = "Nordic Conference on Computational Linguistics , NoDaLiDa 2023 ; Conference date: 22-05-2023 Through 24-05-2023",

}

Talman, A , Celikkanat, H , Virpioja, S, Heinonen, M & Tiedemann, J 2023, Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. i T Alumäe & M Fishel (red), Proceedings of the 24th Nordic Conference on Computational Linguistics. NEALT Proceedings Series, nr. 52, University of Tartu Library, Tartu, s. 358-365, Nordic Conference on Computational Linguistics , Tórshavn, Färöarna, 22/05/2023.

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. / Talman, Aarne ; Celikkanat, Hande ; Virpioja, Sami et al.

Proceedings of the 24th Nordic Conference on Computational Linguistics. red. / Tanel Alumäe ; Mark Fishel. Tartu : University of Tartu Library, 2023. s. 358-365 (NEALT Proceedings Series; Nr. 52).

Forskningsoutput: Kapitel i bok/rapport/konferenshandling › Konferensbidrag › Vetenskaplig › Peer review

TY - GEN

T1 - Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

AU - Talman, Aarne

AU - Celikkanat, Hande

AU - Virpioja, Sami

AU - Heinonen, Markus

AU - Tiedemann, Jörg

N1 - Conference code: 24

PY - 2023

Y1 - 2023

N2 - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

AB - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

KW - 6121 Languages

KW - 113 Computer and information sciences

UR - https://aclanthology.org/2023.nodalida-1.37

M3 - Conference contribution

T3 - NEALT Proceedings Series

SP - 358

EP - 365

BT - Proceedings of the 24th Nordic Conference on Computational Linguistics

A2 - Alumäe , Tanel

A2 - Fishel, Mark

PB - University of Tartu Library

CY - Tartu

T2 - Nordic Conference on Computational Linguistics

Y2 - 22 May 2023 through 24 May 2023

ER -

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Sammanfattning

Publikationsserier

Vetenskapsgrenar

Åtkomst av dokument

Andra filer och länkar

Projekt

Uncertainty-aware neural language models

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

Citera det här