VoiceTTA Demo

Abstract

Recently, zero-shot text-to-speech (TTS) models have enabled high-fidelity, expressive speech synthesis. However, they often struggle to imitate unseen speaking styles when prompted with reference speech from uncommon scenarios (e.g., crosstalk, game commentary), limiting their applicability. Moreover, fine-tuning pretrained TTS models typically requires large, high-quality datasets, making personalized online deployment inconvenient and inefficient. To address these challenges, we propose VoiceTTA, a reinforcement learning-based test-time adaptation (TTA) method that enhances the voice imitation ability of pretrained zero-shot TTS models while preserving efficient inference. VoiceTTA adopts the group relative preference optimization (GRPO) algorithm to optimize a LoRA adapter before inference, guided by intelligibility and style rewards, improving style transfer with negligible computational overhead. Extensive experiments demonstrate that VoiceTTA significantly improves performance on uncommon speech prompts, outperforming the state-of-the-art.

Note: The content on this page is for research purposes only.

Framework Overview

Figure 1: The overview of the proposed VoiceTTA method. We first apply the GRPO algorithm to optimize the LoRA adapter, guided by intelligibility and style rewards. Then, we use the adapted model to synthesize new speech.

Accent

Reference	Text	CosyVoice	MaskGCT	Vevo	F5-TTS	Ours
Now if y'all don't pay me one billion dollar US I will proceed to make the entire world go kaboom!	I don't really care what you call me. I've been a silent spectator, watching species evolve, empires rise and fall.
Great, great, fantastic, thanks a lot. Oh, and don't forget to bring the briefing documents and insurance papers. Yes, great, thank you.	At the end of the day, our choices in food reflect our personal preferences and sometimes, even our lifestyle or belief system.
I woke up this morning to see some basher that stole my chair. Anyway, are you coming with me later to see Plymouth Road Guile? Nice! Great hour, mate!	Perhaps they are driven by the delicious blend of flavors, or it could be the appealing visual presentation.
Yuri! Hello! It's Nikolai here! I will never forget that time in Moscow when Mikhail stole Dmitry's vodka and then he started singing Soviet national anthem. Remember that time in Volograd when Dmitry stole the police motorbike and crashed into ice cream parlor?	Your safety and the pack's reputation are at stake. Your bravery is admirable, but sometimes bravery is knowing when to retreat.
I tried to tell him before, but there was a communication issue. There was nothing I could do. I'm so very sorry that this happened. You need to go compare yourself. I regretfully inform you that it is now outside my responsibility.	Please, consider returning with me. We can work out a plan, but only if you're willing to listen.

Children

Reference	Text	CosyVoice	MaskGCT	Vevo	F5-TTS	Ours
Um, love is when you love somebody. And it's when you love somebody and it's really when you love somebody.	Suddenly, the atmosphere became gloomy. At first glance, all the troubles seemed to surround me.
Because there's usually a lot of like cute poodles there and I like poodles.	I frowned, feeling that pressure, but I know I can't give up, can't admit defeat.
Because they have a very beautiful palace.	The emperor's complexion did not change, remaining as still as a sculpture, and a touch of touching warmth flashed in his eyes.
I learned about bugs.	His voice was low and firm, leaving a faint hint of helplessness and tenderness in the air.
We learned in math about quadru-rata-val- oh I said that wrong. I think, I don't know how to say it, but we know that it's a shape with four sides.	But always remember, I am mighty and enduring. Respect me and I'll nurture you; ignore me and you shall face the consequences.

Slurred

Reference	Text	CosyVoice	MaskGCT	Vevo	F5-TTS	Ours
The bridge didn't know all the problems. My grandfather, well. He is nearly 93 years old. Yeah, he still thinks as swiftly as ever.	He gave way to the others very readily and retreated unperceived by the Squire and Mistress Fitzooth to the rear of the tent.
So that I can see if the aqua- aquaverse makes a difference next week.	One day, I hope you understand the reasons behind my actions. Until then, Anna, please, bear with me.
I really just wanted to recur record the way I look.	Dealing with family secrets is never easy. Yet, sometimes, omission is a form of protection.

Chinese Sketch

Reference	Text	CosyVoice	MaskGCT	Vevo	F5-TTS	Ours
我这心里究竟怎么回事你就跟我说呗，老折磨我干啥呀这是。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
不打针不吃药，坐这就是跟你唠，用谈话的方式治疗，也叫化疗。	各位乡亲，各位父老，下面播送个广告，本人虽说村长落选，但思想工作仍然要搞，在家开个心理诊所，专门治疗人的大脑，欢迎大家前来就诊，有钱给点没钱拉倒。临江村大明白心理诊所，主治医生赵大宝。电话动拐动拐动动拐。
在苍茫的大海上，狂风卷积着乌云。	突然，身边一阵笑声。我看着他们，意气风发地挺直了胸膛。
就我长这样，那浴室的门都老尊重我了，我往那一站，那浴室的门自动唰就开了。	皇上的面色未变，宛如雕塑般静止，他的眼中闪过一丝动人的温度。他深深地看了那位忠心耿耿的臣子一眼，终于开口：“诺，我会再考虑考虑的。”

Chinese Dialects

Reference	Text	CosyVoice	MaskGCT	Vevo	F5-TTS	Ours
保证金将作为团社团的公用资金。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
嗯其中的中医药院校的直属附属医院六十四个。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
作为本科生的王毅能做到这一点。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
基本上一年每个人月薪会减少八百。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
并且睡眠不足还会造成孩子精神不振。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
大哥呀我家炖的大鹅你过来吃不。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
其最终带到新加坡的钱只有二千六百万元。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。
店正常购买的进口车便算是大贸车的一种。	你的安全以及族群的声誉都危在旦夕。你的勇敢令人钦佩，但有时候勇敢在于懂得何时撤退。拜托，考虑一下和我一起回去吧。我们可以制定一个计划，但前提是你愿意倾听。