MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning

Aina Gari Soler, Marianna Apidianaki

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


We present the MULTISEM systems submitted to SemEval 2020 Task 3: Graded Word Similarity in Context (GWSC). We experiment with injecting semantic knowledge into pre-trained BERT models through fine-tuning on lexical semantic tasks related to GWSC. We use existing semantically annotated datasets, and propose to approximate similarity through automatically generated lexical substitutes in context. We participate in both GWSC subtasks and address two languages, English and Finnish. Our best English models occupy the third and fourth positions in the ranking for the two subtasks. Performance is lower for the Finnish models which are mid-ranked in the respective subtasks, highlighting the important role of data availability for fine-tuning.
Original languageEnglish
Title of host publicationProceedings of the Fourteenth Workshop on Semantic Evaluation
EditorsAurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
Number of pages8
Place of PublicationStroudsburg
PublisherThe Association for Computational Linguistics
Publication date2020
ISBN (Electronic)978-1-952148-31-6
Publication statusPublished - 2020
MoE publication typeA4 Article in conference proceedings
EventInternational Workshop on Semantic Evaluation - [online], Barcelona, Spain
Duration: 12 Dec 202013 Dec 2020
Conference number: 14

Fields of Science

  • 113 Computer and information sciences
  • 6121 Languages

Cite this