I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures

Timothee Mickus; Raul Vazquez; Joseph Attieh

I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures

Timothee Mickus, Raul Vazquez, Joseph Attieh

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

Abstrakti

Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.

Alkuperäiskieli	englanti
Otsikko	Proceedings of the Fifth Workshop on Insights from Negative Results in NLP
Toimittajat	Shabnam Tafreshi, Arjun Akula, João Sedoc, Aleksandr Drozd, Anna Rogers, Anna Rumshisky
Sivumäärä	7
Julkaisupaikka	Kerrville
Kustantaja	The Association for Computational Linguistics
Julkaisupäivä	1 kesäk. 2024
Sivut	34-40
ISBN (elektroninen)	979-8-89176-102-5
Tila	Julkaistu - 1 kesäk. 2024
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisuussa
Tapahtuma	Workshop on Insights from Negative Results in NLP - Mexico City, Meksiko Kesto: 20 kesäk. 2024 → 20 kesäk. 2024 Konferenssinumero: 5

Tieteenalat

6121 Kielitieteet
113 Tietojenkäsittely- ja informaatiotieteet

Pääsy asiakirjaan

2024.insights-1.5Lopullinen julkaistu versio, 506 KBLisenssi: CC BY

https://aclanthology.org/2024.insights-1.5Lisenssi: CC BY

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding
Tiedemann, J. (Projektinjohtaja), Attieh, J. (osallistuja), Aulamo, M. (osallistuja), Celikkanat, H. (osallistuja), De Gibert Bonet, O. (osallistuja), Grönroos, S.-A. (osallistuja), Mickus, T. (osallistuja), Raganato, A. (osallistuja), Scherrer, Y. (osallistuja), Silfverberg, M. (osallistuja), Sjöblom, E. I. (osallistuja), Talman, A. (osallistuja), Vazquez , R. (osallistuja), Virpioja, S. P. (osallistuja), Yli-Jyrä, A. (osallistuja), Zosa, E. (osallistuja), Celikkanat, H. (Osallistuja), Raganato, A. (Osallistuja), Silfverberg, M. (Osallistuja), Sulubacak, U. (Osallistuja) & Vazquez , R. (Osallistuja)
01/09/2018 → 31/03/2024
Projekti: EU Horizon 2020: European Research Council: Consolidator Grant (H2020-ERC-COG)

Best paper Award
Varjonen, S. (Vastaanottaja), 7 jouluk. 2011
Palkinto: Palkinnot ja kunnianosoitukset

Siteeraa tätä

Mickus, T., Vazquez, R., & Attieh, J. (2024). I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures. teoksessa S. Tafreshi, A. Akula, J. Sedoc, A. Drozd, A. Rogers, & A. Rumshisky (Toimittajat), Proceedings of the Fifth Workshop on Insights from Negative Results in NLP (Sivut 34-40). The Association for Computational Linguistics. https://aclanthology.org/2024.insights-1.5

Mickus, Timothee ; Vazquez, Raul ; Attieh, Joseph. / I Have an Attention Bridge to Sell You : Generalization Capabilities of Modular Translation Architectures. Proceedings of the Fifth Workshop on Insights from Negative Results in NLP. Toimittaja / Shabnam Tafreshi ; Arjun Akula ; João Sedoc ; Aleksandr Drozd ; Anna Rogers ; Anna Rumshisky. Kerrville : The Association for Computational Linguistics, 2024. Sivut 34-40

@inproceedings{eb7699c9fd5f4c8086e414179b7ec8cf,

title = "I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures",

abstract = "Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.",

keywords = "6121 Languages, 113 Computer and information sciences",

author = "Timothee Mickus and Raul Vazquez and Joseph Attieh",

year = "2024",

month = jun,

day = "1",

language = "English",

pages = "34--40",

editor = "Shabnam Tafreshi and Arjun Akula and Jo{\~a}o Sedoc and Aleksandr Drozd and Anna Rogers and Anna Rumshisky",

booktitle = "Proceedings of the Fifth Workshop on Insights from Negative Results in NLP",

publisher = "The Association for Computational Linguistics",

address = "United States",

note = "Workshop on Insights from Negative Results in NLP ; Conference date: 20-06-2024 Through 20-06-2024",

}

Mickus, T , Vazquez, R & Attieh, J 2024, I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures. julkaisussa S Tafreshi, A Akula, J Sedoc, A Drozd, A Rogers & A Rumshisky (toim), Proceedings of the Fifth Workshop on Insights from Negative Results in NLP. The Association for Computational Linguistics, Kerrville, Sivut 34-40, Workshop on Insights from Negative Results in NLP, Mexico City, Meksiko, 20/06/2024. <https://aclanthology.org/2024.insights-1.5>

I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures. / Mickus, Timothee ; Vazquez, Raul ; Attieh, Joseph.
Proceedings of the Fifth Workshop on Insights from Negative Results in NLP. toim. / Shabnam Tafreshi; Arjun Akula; João Sedoc; Aleksandr Drozd; Anna Rogers; Anna Rumshisky. Kerrville: The Association for Computational Linguistics, 2024. s. 34-40.

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussa › Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

TY - GEN

T1 - I Have an Attention Bridge to Sell You

T2 - Workshop on Insights from Negative Results in NLP

AU - Mickus, Timothee

AU - Vazquez, Raul

AU - Attieh, Joseph

N1 - Conference code: 5

PY - 2024/6/1

Y1 - 2024/6/1

N2 - Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.

AB - Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.

KW - 6121 Languages

KW - 113 Computer and information sciences

M3 - Conference contribution

SP - 34

EP - 40

BT - Proceedings of the Fifth Workshop on Insights from Negative Results in NLP

A2 - Tafreshi, Shabnam

A2 - Akula, Arjun

A2 - Sedoc, João

A2 - Drozd, Aleksandr

A2 - Rogers, Anna

A2 - Rumshisky, Anna

PB - The Association for Computational Linguistics

CY - Kerrville

Y2 - 20 June 2024 through 20 June 2024

ER -

Mickus T , Vazquez R , Attieh J. I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures. julkaisussa Tafreshi S, Akula A, Sedoc J, Drozd A, Rogers A, Rumshisky A, toimittajat, Proceedings of the Fifth Workshop on Insights from Negative Results in NLP. Kerrville: The Association for Computational Linguistics. 2024. s. 34-40

I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures

Abstrakti

Tieteenalat

Pääsy asiakirjaan

Projektit

FoTran: Found in Translation - Natural Language Understanding with Cross-Lingual Grounding

Palkinnot

Best paper Award

Siteeraa tätä