Investigating the Utility of Surprisal from Large Language Models for Speech Synthesis Prosody

Tutkimustuotos: Artikkeli kirjassa/raportissa/konferenssijulkaisussaKonferenssiartikkeliAmmatillinen

Abstrakti

This paper investigates the use of word surprisal, a measure of the predictability of a word in a given context, as a feature to aid speech synthesis prosody. We explore how word surprisal extracted from large language models (LLMs) correlates with word prominence, a signal-based measure of the salience of a word in a given discourse. We also examine how context length and LLM size affect the results, and how a speech synthesizer conditioned with surprisal values compares with a baseline system. To evaluate these factors, we conducted experiments using a large corpus of English text and LLMs of varying sizes. Our results show that word surprisal and word prominence are moderately correlated, suggesting that they capture related but distinct aspects of language use. We find that length of context and size of the LLM impact the correlations, but not in the direction anticipated, with longer contexts and larger LLMs generally underpredicting prominent words in a nearly linear manner. We demonstrate that, in line with these findings, a speech synthesizer conditioned with surprisal values provides a minimal improvement over the baseline with the results suggesting a limited effect of using surprisal values for eliciting appropriate prominence patterns.
Alkuperäiskielienglanti
OtsikkoProceedings of the 12th ISCA Speech Synthesis Workshop (SSW)
ToimittajatThomas Hueber, Damien Lolive, Nicolas Obin, Olivier Perrotin
Sivumäärä7
JulkaisupaikkaBaixas
KustantajaISCA - International Speech Communication Association
Julkaisupäiväelok. 2023
Sivut127-133
DOI - pysyväislinkit
TilaJulkaistu - elok. 2023
OKM-julkaisutyyppiD3 Artikkeli ammatillisessa konferenssijulkaisussa
TapahtumaSpeech Synthesis Workshop - MaCI (« Maison de la Création et de l’Innovation »), Grenoble, Ranska
Kesto: 26 elok. 202328 elok. 2023
Konferenssinumero: 12
https://ssw2023.org/

Tieteenalat

  • 213 Sähkö-, automaatio- ja tietoliikennetekniikka, elektroniikka
  • 113 Tietojenkäsittely- ja informaatiotieteet
  • 6121 Kielitieteet

Siteeraa tätä