Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman; Hande Celikkanat; Sami Virpioja; Markus Heinonen; Jörg Tiedemann

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Aarne Talman, Hande Celikkanat, Sami Virpioja, Markus Heinonen, Jörg Tiedemann

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Abstract

This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

Original language	English
Title of host publication	Proceedings of the 24th Nordic Conference on Computational Linguistics
Editors	Tanel Alumäe , Mark Fishel
Number of pages	8
Place of Publication	Tartu
Publisher	University of Tartu Library
Publication date	2023
Pages	358-365
ISBN (Electronic)	978-99-1621-999-7
Publication status	Published - 2023
MoE publication type	A4 Article in conference proceedings
Event	Nordic Conference on Computational Linguistics - Tórshavn, Faroe Islands Duration: 22 May 2023 → 24 May 2023 Conference number: 24

Publication series

Name	NEALT Proceedings Series
Publisher	University of Tartu Library
Number	52
ISSN (Print)	1736-8197
ISSN (Electronic)	1736-6305

Fields of Science

6121 Languages
113 Computer and information sciences

Access to Document

2023.nodalida-1.37Final published version, 338 KBLicence: CC BY

Cite this

@inproceedings{8baa06ac175b4d1697c19025bc05a5bf,

title = "Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging",

abstract = "This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.",

keywords = "6121 Languages, 113 Computer and information sciences",

author = "Aarne Talman and Hande Celikkanat and Sami Virpioja and Markus Heinonen and J{\"o}rg Tiedemann",

year = "2023",

language = "English",

series = "NEALT Proceedings Series",

publisher = "University of Tartu Library",

number = "52",

pages = "358--365",

editor = "{Alum{\"a}e }, {Tanel } and Fishel, {Mark }",

booktitle = "Proceedings of the 24th Nordic Conference on Computational Linguistics",

address = "Estonia",

note = "Nordic Conference on Computational Linguistics , NoDaLiDa 2023 ; Conference date: 22-05-2023 Through 24-05-2023",

}

Talman, A , Celikkanat, H , Virpioja, S, Heinonen, M & Tiedemann, J 2023, Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. in T Alumäe & M Fishel (eds), Proceedings of the 24th Nordic Conference on Computational Linguistics. NEALT Proceedings Series, no. 52, University of Tartu Library, Tartu, pp. 358-365, Nordic Conference on Computational Linguistics , Tórshavn, Faroe Islands, 22/05/2023.

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging. / Talman, Aarne ; Celikkanat, Hande ; Virpioja, Sami et al.

Proceedings of the 24th Nordic Conference on Computational Linguistics. ed. / Tanel Alumäe ; Mark Fishel. Tartu : University of Tartu Library, 2023. p. 358-365 (NEALT Proceedings Series; No. 52).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

AU - Talman, Aarne

AU - Celikkanat, Hande

AU - Virpioja, Sami

AU - Heinonen, Markus

AU - Tiedemann, Jörg

N1 - Conference code: 24

PY - 2023

Y1 - 2023

N2 - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

AB - This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

KW - 6121 Languages

KW - 113 Computer and information sciences

UR - https://aclanthology.org/2023.nodalida-1.37

M3 - Conference contribution

T3 - NEALT Proceedings Series

SP - 358

EP - 365

BT - Proceedings of the 24th Nordic Conference on Computational Linguistics

A2 - Alumäe , Tanel

A2 - Fishel, Mark

PB - University of Tartu Library

CY - Tartu

T2 - Nordic Conference on Computational Linguistics

Y2 - 22 May 2023 through 24 May 2023

ER -

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Abstract

Publication series

Fields of Science

Access to Document

Other files and links

Uncertainty-aware neural language models

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

Cite this

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

Abstract

Publication series

Fields of Science

Access to Document

Other files and links

Projects

Uncertainty-aware neural language models

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

Cite this