Abstrakti
Controlled paraphrase generation often focuses on a specific aspect of paraphrasing, for instance syntactically controlled paraphrase generation. However, these models face a limitation: they lack modularity. Consequently adapting them for another aspect, such as lexical variation, needs full retraining of the model each time. To enhance the flexibility in training controlled paraphrase models, our proposition involves incrementally training a modularized system for controlled paraphrase generation for English. We start by fine-tuning a pretrained language model to learn the broad task of paraphrase generation, generally emphasizing meaning preservation and surface form variation. Subsequently, we train a specialized sub-task adapter with limited sub-task specific training data. We can then leverage this adapter in guiding the paraphrase generation process toward a desired output aligning with the distinctive features within the sub-task training data. The preliminary results on comparing the fine-tuned and adapted model against various competing systems indicates that the most successful method for mastering both general paraphrasing skills and task-specific expertise follows a two-stage approach. This approach involves starting with the initial fine-tuning of a generic paraphrase model and subsequently tailoring it for the specific sub-task.
Alkuperäiskieli | englanti |
---|---|
Otsikko | Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024) |
Toimittajat | Raul Vazquez, Timothee Mickus, Jörg Tiedemann, Ivan Vulic, Ahmet Üstün |
Sivumäärä | 6 |
Julkaisupaikka | Stroudsburg |
Kustantaja | Association for Computational Linguistics (ACL) |
Julkaisupäivä | maalisk. 2024 |
Sivut | 1-6 |
ISBN (elektroninen) | 979-8-89176-084-4 |
Tila | Julkaistu - maalisk. 2024 |
OKM-julkaisutyyppi | A4 Artikkeli konferenssijulkaisuussa |
Tapahtuma | Workshop on Modular and Open Multilingual NLP: MOOMIN 2024 - St. Julian's, Malta Kesto: 21 maalisk. 2024 → 21 maalisk. 2024 Konferenssinumero: 1 |
Lisätietoja
Publisher Copyright:© 2024 Association for Computational Linguistics.
Tieteenalat
- 6121 Kielitieteet
- 113 Tietojenkäsittely- ja informaatiotieteet