Toward the Modular Training of Controlled Paraphrase Adapters

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussaKonferenssiartikkeliTieteellinenvertaisarvioitu

Abstrakti

Controlled paraphrase generation often focuses on a specific aspect of paraphrasing, for instance syntactically controlled paraphrase generation. However, these models face a limitation: they lack modularity. Consequently adapting them for another aspect, such as lexical variation, needs full retraining of the model each time. To enhance the flexibility in training controlled paraphrase models, our proposition involves incrementally training a modularized system for controlled paraphrase generation for English. We start by fine-tuning a pretrained language model to learn the broad task of paraphrase generation, generally emphasizing meaning preservation and surface form variation. Subsequently, we train a specialized sub-task adapter with limited sub-task specific training data. We can then leverage this adapter in guiding the paraphrase generation process toward a desired output aligning with the distinctive features within the sub-task training data. The preliminary results on comparing the fine-tuned and adapted model against various competing systems indicates that the most successful method for mastering both general paraphrasing skills and task-specific expertise follows a two-stage approach. This approach involves starting with the initial fine-tuning of a generic paraphrase model and subsequently tailoring it for the specific sub-task.

Alkuperäiskielienglanti
OtsikkoProceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024)
ToimittajatRaul Vazquez, Timothee Mickus, Jörg Tiedemann, Ivan Vulic, Ahmet Üstün
Sivumäärä6
JulkaisupaikkaStroudsburg
KustantajaAssociation for Computational Linguistics (ACL)
Julkaisupäivämaalisk. 2024
Sivut1-6
ISBN (elektroninen)979-8-89176-084-4
TilaJulkaistu - maalisk. 2024
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaWorkshop on Modular and Open Multilingual NLP: MOOMIN 2024 - St. Julian's, Malta
Kesto: 21 maalisk. 202421 maalisk. 2024
Konferenssinumero: 1

Lisätietoja

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Tieteenalat

  • 6121 Kielitieteet
  • 113 Tietojenkäsittely- ja informaatiotieteet

Siteeraa tätä