Controllable Accent Normalization via Discrete Diffusion


Abstract

Existing accent normalization methods do not typically offer control over accent strength, yet many applications—such as language learning and dubbing—require tunable accent retention. We propose DLM-AN, a controllable accent normalization system built on masked discrete diffusion over self-supervised speech tokens. A Common Token Predictor identifies source tokens that likely encode native pronunciation; these tokens are selectively reused to initialize the reverse diffusion process. This provides a simple yet effective mechanism for controlling accent strength: reusing more tokens preserves more of the original accent. DLM-AN further incorporates a flow-matching Duration Ratio Predictor that automatically adjusts the total duration to better match the native rhythm. Experiments on multi-accent English data show that DLM-AN achieves the lowest word error rate among all compared systems while delivering competitive accent reduction and smooth, interpretable accent strength control.

You can download all audio files on this page by cloning this github repository.

General System Comparison

Each column corresponds to a different speaker utterance and each row to a different system. The first row contains the original accented source audio; subsequent rows show the accent-normalized outputs from various systems.

Naming convention: -1 systems are duration-free (no total-duration constraint); -2 systems are duration-controlled (preserving the source total duration).

DLM-AN-2 threshold: Controls accent strength via token reuse — threshold = 1.0 generates entirely from scratch (strongest normalization), while threshold = 0.0 resynthesizes with all source tokens (original accent preserved).

ABA (Arabic) YBAA (Arabic) ZHAA (Arabic) SKA (Arabic) RRBI (Hindi) TNI (Hindi) ASI (Hindi) SVBI (Hindi) ERMS (Spanish) EBVS (Spanish) NJS (Spanish) MBMPS (Spanish) LXC (Chinese) BWC (Chinese) NCC (Chinese) TXHC (Chinese) HJK (Korean) YDCK (Korean) YKWK (Korean) HKK (Korean) HQTV (Vietnamese) PNV (Vietnamese) THV (Vietnamese) TLV (Vietnamese) BDL (American) RMS (American) SLT (American) CLB (American) ABA (Arabic) YBAA (Arabic) ZHAA (Arabic) SKA (Arabic) RRBI (Hindi) TNI (Hindi) ASI (Hindi) SVBI (Hindi) ERMS (Spanish) EBVS (Spanish) NJS (Spanish) MBMPS (Spanish) LXC (Chinese) BWC (Chinese) NCC (Chinese) TXHC (Chinese) HJK (Korean) YDCK (Korean) YKWK (Korean) HKK (Korean) HQTV (Vietnamese) PNV (Vietnamese) THV (Vietnamese) TLV (Vietnamese) BDL (American) RMS (American) SLT (American) CLB (American)
Text

I was brought up the way most girls in Hawaii are brought up

I want to know how all this is possible

Without them he could not run his empire

The other felt a sudden wave of irritation rush through him

I had been born with no organic chemical predisposition toward alcohol

But there came no promise from the bow of the canoe

They likewise are disinclined to being eaten

But he reconciled himself to it by an act of faith

Bassett was a fastidious man

But Martin smiled a superior smile

I never allow what can't be changed to annoy me

All operations have been carried on from Montreal and Toronto

It was his intention to return to Eileen and her father

Do you know that you are shaking my confidence in you

His immaculate appearance was gone

They are his tongue by which he makes his knowledge articulate

It occurred to me that there would have to be an accounting

It was beating and waiting in the ambush of those black pits

Suddenly his fingers closed tightly over the handkerchief

There's not an iota of truth in it

He moved away as quietly as he had come

The girl faced him her eyes shining with sudden fear

He saw only the effect in a general sketchy way

Down there the earth was already swelling with life

Thus had the raw wilderness prepared him for this day.

It was not a large lake, and almost round.

He was smooth-shaven, and his hair and eyes were black.

One by one the boys were captured.

From my earliest recollection my sleep was a period of terror

Of course much grumbling went on and little outbursts were continually occurring

He caught himself with a jerk

He looked like one who had passed through an uncomfortable hour or two

The steward has just tendered me a respectful bit of advice

There is not an iota of truth in it certainly not

The very thought of the effort to swim over was nauseating

He was sure now of but few things

Besides had he not whipped the big owl in the forest

Philip knew that she was not an Indian

She is essentially the life giving life conserving female of the species

Philip bent low over Pierre

Scarcely had he uttered the name when Pierre's closing eyes shot open

They edged nearer and stood shoulder to shoulder facing their world

In her haste to get away she had forgotten these things

But how are you going to do it

A bush chief had died a natural death

What an excited whispering and conferring took place

He made sure that the magazine was loaded and resumed his paddling

But already he had composed himself

Obviously it was a disease that could be contracted by contact

It lived in perpetual apprehension of that quarter of the compass

To my dearest and always appreciated friend I submit myself

I just do appreciate it without being able to express my feelings

Now, you understand.

They will search for us between their camp and Churchill.

I was sick once -- typhoid.

Mab, she said.

Source
TokAN-1
TokAN-2
CosyAccent-1
CosyAccent-2
DLM-AN-1
DLM-AN-2
(Threshold=1.0)
DLM-AN-2
(Threshold=0.3)
DLM-AN-2
(Threshold=0.0)

↔ Scroll horizontally to explore additional columns.

Accent Strength Control — Proportion of Token Reuse

This panel demonstrates smooth accent strength control by varying the proportion of source tokens reused during generation. A higher proportion retains more of the original accent; a lower proportion yields stronger normalization.

  • Proportion = 1.00 (top row) — Full resynthesis: all source tokens are reused, preserving the original accent.
  • Proportion = 0.00 (bottom row) — Generation from scratch: no source tokens are reused, maximizing accent normalization.
ABA (Arabic) YBAA (Arabic) ZHAA (Arabic) SKA (Arabic) RRBI (Hindi) TNI (Hindi) ASI (Hindi) SVBI (Hindi) ERMS (Spanish) EBVS (Spanish) NJS (Spanish) MBMPS (Spanish) LXC (Chinese) BWC (Chinese) NCC (Chinese) TXHC (Chinese) HJK (Korean) YDCK (Korean) YKWK (Korean) HKK (Korean) HQTV (Vietnamese) PNV (Vietnamese) THV (Vietnamese) TLV (Vietnamese) BDL (American) RMS (American) SLT (American) CLB (American)
Text

From my earliest recollection my sleep was a period of terror

He was sure now of but few things

There's not an iota of truth in it

I was brought up the way most girls in Hawaii are brought up

I had been born with no organic chemical predisposition toward alcohol

They will search for us between their camp and Churchill

They likewise are disinclined to being eaten

He looked like one who had passed through an uncomfortable hour or two

Besides had he not whipped the big owl in the forest

Philip knew that she was not an Indian

He moved away as quietly as he had come

But already he had composed himself

Scarcely had he uttered the name when Pierre's closing eyes shot open

Do you know that you are shaking my confidence in you

His immaculate appearance was gone

It lived in perpetual apprehension of that quarter of the compass

A bush chief had died a natural death

It was his intention to return to Eileen and her father

He made sure that the magazine was loaded and resumed his paddling

Bassett was a fastidious man

Obviously it was a disease that could be contracted by contact

The girl faced him her eyes shining with sudden fear

He saw only the effect in a general sketchy way

Down there the earth was already swelling with life

Thus had the raw wilderness prepared him for this day.

It occurred to me that there would have to be an accounting.

I was sick once -- typhoid.

One by one the boys were captured.

Source
Proportion=1.00
(Resynthesis)
Proportion=0.90
Proportion=0.70
Proportion=0.50
Proportion=0.30
Proportion=0.10
Proportion=0.00
(From-scratch)

↔ Scroll horizontally to explore additional columns.

Accent Strength Control — Thresholding Token Reuse

This panel demonstrates accent strength control via thresholding. The Common Token Predictor assigns each source token a probability of being native-like; only tokens whose probability exceeds the threshold are reused.

  • Threshold = 0.00 (top row) — All tokens pass the threshold and are reused (resynthesis, original accent preserved).
  • Threshold = 1.00 (bottom row) — No tokens pass the threshold (generation from scratch, strongest normalization).
ABA (Arabic) YBAA (Arabic) ZHAA (Arabic) SKA (Arabic) RRBI (Hindi) TNI (Hindi) ASI (Hindi) SVBI (Hindi) ERMS (Spanish) EBVS (Spanish) NJS (Spanish) MBMPS (Spanish) LXC (Chinese) BWC (Chinese) NCC (Chinese) TXHC (Chinese) HJK (Korean) YDCK (Korean) YKWK (Korean) HKK (Korean) HQTV (Vietnamese) PNV (Vietnamese) THV (Vietnamese) TLV (Vietnamese) BDL (American) RMS (American) SLT (American) CLB (American)
Text

From my earliest recollection my sleep was a period of terror

He was sure now of but few things

There's not an iota of truth in it

I was brought up the way most girls in Hawaii are brought up

I had been born with no organic chemical predisposition toward alcohol

They will search for us between their camp and Churchill

They likewise are disinclined to being eaten

He looked like one who had passed through an uncomfortable hour or two

Besides had he not whipped the big owl in the forest

Philip knew that she was not an Indian

He moved away as quietly as he had come

But already he had composed himself

Scarcely had he uttered the name when Pierre's closing eyes shot open

Do you know that you are shaking my confidence in you

His immaculate appearance was gone

It lived in perpetual apprehension of that quarter of the compass

A bush chief had died a natural death

It was his intention to return to Eileen and her father

He made sure that the magazine was loaded and resumed his paddling

Bassett was a fastidious man

Obviously it was a disease that could be contracted by contact

The girl faced him her eyes shining with sudden fear

He saw only the effect in a general sketchy way

Down there the earth was already swelling with life

Thus had the raw wilderness prepared him for this day.

It occurred to me that there would have to be an accounting.

I was sick once -- typhoid.

One by one the boys were captured.

Source
Threshold=0.00
(Resynthesis)
Threshold=0.01
Threshold=0.10
Threshold=0.30
Threshold=0.50
Threshold=0.70
Threshold=0.90
Threshold=0.99
Threshold=1.00
(From-scratch)

↔ Scroll horizontally to explore additional columns.

Duration Control — Scaling the Total Duration

This panel showcases DLM-AN's ability to control the total duration of generated speech. The Duration Ratio Predictor scales the source utterance length by a given factor: ×0.75 produces faster speech, ×1.0 preserves the original duration, and ×1.25 produces slower speech — all while maintaining the same level of accent normalization.

ABA (Arabic) YBAA (Arabic) ZHAA (Arabic) SKA (Arabic) RRBI (Hindi) TNI (Hindi) ASI (Hindi) SVBI (Hindi) ERMS (Spanish) EBVS (Spanish) NJS (Spanish) MBMPS (Spanish) LXC (Chinese) BWC (Chinese) NCC (Chinese) TXHC (Chinese) HJK (Korean) YDCK (Korean) YKWK (Korean) HKK (Korean) HQTV (Vietnamese) PNV (Vietnamese) THV (Vietnamese) TLV (Vietnamese) BDL (American) RMS (American) SLT (American) CLB (American)
Text

From my earliest recollection my sleep was a period of terror

He was sure now of but few things

There's not an iota of truth in it

I was brought up the way most girls in Hawaii are brought up

I had been born with no organic chemical predisposition toward alcohol

They will search for us between their camp and Churchill

They likewise are disinclined to being eaten

He looked like one who had passed through an uncomfortable hour or two

Besides had he not whipped the big owl in the forest

Philip knew that she was not an Indian

He moved away as quietly as he had come

But already he had composed himself

Scarcely had he uttered the name when Pierre's closing eyes shot open

Do you know that you are shaking my confidence in you

His immaculate appearance was gone

It lived in perpetual apprehension of that quarter of the compass

A bush chief had died a natural death

It was his intention to return to Eileen and her father

He made sure that the magazine was loaded and resumed his paddling

Bassett was a fastidious man

Obviously it was a disease that could be contracted by contact

The girl faced him her eyes shining with sudden fear

He saw only the effect in a general sketchy way

Down there the earth was already swelling with life

Thus had the raw wilderness prepared him for this day.

It occurred to me that there would have to be an accounting.

I was sick once -- typhoid.

One by one the boys were captured.

Source
Duration x0.75
Duration x1.0
Duration x1.25

↔ Scroll horizontally to explore additional columns.