Analysing concatenation approaches to document-level NMT in two different domains

Yves Scherrer; Jörg Tiedemann; Sharid Loáiciga

doi:10.18653/v1/D19-6506

Analysing concatenation approaches to document-level NMT in two different domains

Yves Scherrer, Jörg Tiedemann, Sharid Loáiciga

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Abstract

In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.

Original language	English
Title of host publication	The Fourth Workshop on Discourse in Machine Translation : Proceedings of the Workshop
Number of pages	11
Place of Publication	Stroudsburg
Publisher	The Association for Computational Linguistics
Publication date	1 Nov 2019
Pages	51-61
ISBN (Electronic)	978-1-950737-74-1
DOIs	https://doi.org/10.18653/v1/D19-6506
Publication status	Published - 1 Nov 2019
MoE publication type	A4 Article in conference proceedings
Event	Workshop on Discourse in Machine Translation - Hong Kong, China Duration: 3 Nov 2019 → 3 Nov 2019 Conference number: 4

Fields of Science

113 Computer and information sciences
6121 Languages

Access to Document

10.18653/v1/D19-6506Licence: CC BY

D19-6506Final published version, 259 KBLicence: CC BY

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding
Tiedemann, J., Attieh, J., Aulamo, M., Boggia, M., Celikkanat, H., De Gibert Bonet, O., Grönroos, S., Mickus, T., Raganato, A., Scherrer, Y., Silfverberg, M., Sjöblom, E. I., Talman, A., Vazquez , R., Virpioja, S. P., Yli-Jyrä, A., Zosa, E., Celikkanat, H., Raganato, A., Silfverberg, M., Sulubacak, U. & Vazquez , R.
01/09/2018 → 31/03/2024
Project: EU Horizon 2020: European Research Council: Consolidator Grant (H2020-ERC-COG)
MeMAD: Methods for Managing Audiovisual Data: Combining Automatic Efficiency with Human Accuracy
Hirvonen, M., Tiedemann, J., Tiittula, L., Sulubacak, U., Vazquez , R. & Koponen, M.
European Commission / Horizon 2020
01/01/2018 → 31/03/2021
Project: Research project

Document-level Machine Translation Benchmark
Scherrer, Y. (Creator), Tiedemann, J. (Creator) & Loáiciga, S. (Creator), University of Helsinki, 1 Nov 2019
DOI: 10.5281/zenodo.3525366
Dataset

Cite this

@inproceedings{3ba0164606504d9d9e4bb71b91298ab8,

title = "Analysing concatenation approaches to document-level NMT in two different domains",

abstract = "In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.",

keywords = "113 Computer and information sciences, 6121 Languages",

author = "Yves Scherrer and J{\"o}rg Tiedemann and Sharid Lo{\'a}iciga",

year = "2019",

month = nov,

day = "1",

doi = "10.18653/v1/D19-6506",

language = "English",

pages = "51--61",

booktitle = "The Fourth Workshop on Discourse in Machine Translation",

publisher = "The Association for Computational Linguistics",

address = "United States",

note = "Workshop on Discourse in Machine Translation ; Conference date: 03-11-2019 Through 03-11-2019",

}

Scherrer, Y , Tiedemann, J & Loáiciga, S 2019, Analysing concatenation approaches to document-level NMT in two different domains. in The Fourth Workshop on Discourse in Machine Translation: Proceedings of the Workshop. The Association for Computational Linguistics, Stroudsburg, pp. 51-61, Workshop on Discourse in Machine Translation, Hong Kong, China, 03/11/2019. https://doi.org/10.18653/v1/D19-6506

Analysing concatenation approaches to document-level NMT in two different domains. / Scherrer, Yves ; Tiedemann, Jörg; Loáiciga, Sharid.
The Fourth Workshop on Discourse in Machine Translation: Proceedings of the Workshop. Stroudsburg: The Association for Computational Linguistics, 2019. p. 51-61.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Analysing concatenation approaches to document-level NMT in two different domains

AU - Scherrer, Yves

AU - Tiedemann, Jörg

AU - Loáiciga, Sharid

N1 - Conference code: 4

PY - 2019/11/1

Y1 - 2019/11/1

N2 - In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.

AB - In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.

KW - 113 Computer and information sciences

KW - 6121 Languages

U2 - 10.18653/v1/D19-6506

DO - 10.18653/v1/D19-6506

M3 - Conference contribution

SP - 51

EP - 61

BT - The Fourth Workshop on Discourse in Machine Translation

PB - The Association for Computational Linguistics

CY - Stroudsburg

T2 - Workshop on Discourse in Machine Translation

Y2 - 3 November 2019 through 3 November 2019

ER -

Analysing concatenation approaches to document-level NMT in two different domains

Abstract

Fields of Science

Access to Document

Projects

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

MeMAD: Methods for Managing Audiovisual Data: Combining Automatic Efficiency with Human Accuracy

Datasets

Document-level Machine Translation Benchmark

Cite this