Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

Iiro Rastas; Yann Ciarán Ryan; Iiro Lassi Ilmari Tiihonen; Mohammadreza Qaraei; Liina Repo; Rohit Babbar; Eetu Mäkelä; Mikko Tolonen; Filip Ginter

Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

Iiro Rastas, Yann Ciarán Ryan, Iiro Lassi Ilmari Tiihonen, Mohammadreza Qaraei, Liina Repo, Rohit Babbar, Eetu Mäkelä, Mikko Tolonen, Filip Ginter

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

Abstrakti

In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents. The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts. We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years absolute error. We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.

Alkuperäiskieli	englanti
Otsikko	PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022)
Toimittajat	Nina Tahmasebi, Syrielle Montariol, Andrey Kutuzov, Simon Hengchen, Haim Dubossarsky, Lars Borin
Sivumäärä	10
Julkaisupaikka	Stroudsburg
Kustantaja	The Association for Computational Linguistics
Julkaisupäivä	2022
Sivut	68–77
ISBN (elektroninen)	978-1-955917-42-1
Tila	Julkaistu - 2022
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisuussa
Tapahtuma	Workshop on Computational Approaches to Historical Language Change - [Hybrid event], Dublin, Irlanti Kesto: 26 toukok. 2022 → 27 toukok. 2022 Konferenssinumero: 3

Tieteenalat

6121 Kielitieteet
113 Tietojenkäsittely- ja informaatiotieteet

Pääsy asiakirjaan

2022.lchange-1.7Lopullinen julkaistu versio, 268 KBLisenssi: CC BY

https://aclanthology.org/2022.lchange-1.7.pdfLisenssi: CC BY

HPC-HD: High Performance Computing for the Detection and Analysis of Historical Discourses
Tolonen, M., Mäkelä, E., Mathioudakis, M., Ginter, F. & Babbar, R.
Academy of Finland, Suomen Akatemia Projektilaskutus
01/01/2022 → 31/12/2024
Projekti: Tutkimusprojekti

Siteeraa tätä

Rastas, I., Ryan, Y. C., Tiihonen, I. L. I., Qaraei, M., Repo, L., Babbar, R., Mäkelä, E., Tolonen, M., & Ginter, F. (2022). Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model. teoksessa N. Tahmasebi, S. Montariol, A. Kutuzov, S. Hengchen, H. Dubossarsky, & L. Borin (Toimittajat), PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022) (Sivut 68–77). The Association for Computational Linguistics. https://aclanthology.org/2022.lchange-1.7.pdf

Rastas, Iiro ; Ryan, Yann Ciarán ; Tiihonen, Iiro Lassi Ilmari et al. / Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model. PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022). Toimittaja / Nina Tahmasebi ; Syrielle Montariol ; Andrey Kutuzov ; Simon Hengchen ; Haim Dubossarsky ; Lars Borin. Stroudsburg : The Association for Computational Linguistics, 2022. Sivut 68–77

@inproceedings{f66511ff58364037b168f1add86955b6,

title = "Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model",

abstract = "In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents. The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts. We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years absolute error. We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.",

keywords = "6121 Languages, 113 Computer and information sciences",

author = "Iiro Rastas and Ryan, {Yann Ciar{\'a}n} and Tiihonen, {Iiro Lassi Ilmari} and Mohammadreza Qaraei and Liina Repo and Rohit Babbar and Eetu M{\"a}kel{\"a} and Mikko Tolonen and Filip Ginter",

year = "2022",

language = "English",

pages = "68–77",

editor = "Tahmasebi, {Nina } and Syrielle Montariol and Kutuzov, {Andrey } and Hengchen, {Simon } and Haim Dubossarsky and Borin, {Lars }",

booktitle = "PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022)",

publisher = "The Association for Computational Linguistics",

address = "United States",

note = "Workshop on Computational Approaches to Historical Language Change ; Conference date: 26-05-2022 Through 27-05-2022",

}

Rastas, I, Ryan, YC, Tiihonen, ILI, Qaraei, M, Repo, L, Babbar, R, Mäkelä, E , Tolonen, M & Ginter, F 2022, Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model. julkaisussa N Tahmasebi, S Montariol, A Kutuzov, S Hengchen, H Dubossarsky & L Borin (toim), PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022). The Association for Computational Linguistics, Stroudsburg, Sivut 68–77, Workshop on Computational Approaches to Historical Language Change, Dublin, Irlanti, 26/05/2022. <https://aclanthology.org/2022.lchange-1.7.pdf>

Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model. / Rastas, Iiro; Ryan, Yann Ciarán; Tiihonen, Iiro Lassi Ilmari et al.

PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022). toim. / Nina Tahmasebi; Syrielle Montariol; Andrey Kutuzov; Simon Hengchen; Haim Dubossarsky; Lars Borin. Stroudsburg : The Association for Computational Linguistics, 2022. s. 68–77.

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

TY - GEN

T1 - Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

AU - Rastas, Iiro

AU - Ryan, Yann Ciarán

AU - Tiihonen, Iiro Lassi Ilmari

AU - Qaraei, Mohammadreza

AU - Repo, Liina

AU - Babbar, Rohit

AU - Mäkelä, Eetu

AU - Tolonen, Mikko

AU - Ginter, Filip

N1 - Conference code: 3

PY - 2022

Y1 - 2022

N2 - In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents. The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts. We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years absolute error. We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.

AB - In this paper, we describe a BERT model trained on the Eighteenth Century Collections Online (ECCO) dataset of digitized documents. The ECCO dataset poses unique modelling challenges due to the presence of Optical Character Recognition (OCR) artifacts. We establish the performance of the BERT model on a publication year prediction task against linear baseline models and human judgement, finding the BERT model to be superior to both and able to date the works, on average, with less than 7 years absolute error. We also explore how language change over time affects the model by analyzing the features the model uses for publication year predictions as given by the Integrated Gradients model explanation method.

KW - 6121 Languages

KW - 113 Computer and information sciences

M3 - Conference contribution

SP - 68

EP - 77

BT - PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022)

A2 - Tahmasebi, Nina

A2 - Montariol, Syrielle

A2 - Kutuzov, Andrey

A2 - Hengchen, Simon

A2 - Dubossarsky, Haim

A2 - Borin, Lars

PB - The Association for Computational Linguistics

CY - Stroudsburg

T2 - Workshop on Computational Approaches to Historical Language Change

Y2 - 26 May 2022 through 27 May 2022

ER -

Rastas I, Ryan YC, Tiihonen ILI, Qaraei M, Repo L, Babbar R et al. Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model. julkaisussa Tahmasebi N, Montariol S, Kutuzov A, Hengchen S, Dubossarsky H, Borin L, toimittajat, PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON COMPUTATIONAL APPROACHES TO HISTORICAL LANGUAGE CHANGE 2022 (LCHANGE 2022). Stroudsburg: The Association for Computational Linguistics. 2022. s. 68–77

Explainable Publication Year Prediction of Eighteenth Century Texts with the BERT Model

Abstrakti

Tieteenalat

Pääsy asiakirjaan

Projektit

HPC-HD: High Performance Computing for the Detection and Analysis of Historical Discourses

Siteeraa tätä