Automatic real-time transcription language detection in Agent Workspace

If enabled for your account, you can use automatic real-time transcription (RTT) language detection in your VCC. This feature automatically detects the spoken language (or languages) within the first few seconds of the call and applies it to real-time transcription.

The detected language is also used for related features, such as sentiment analysis and summarization, which are generated in the same detected language.

Only languages configured for the interaction are identified. Speech in other languages is not transcribed. You can configure up to three supported languages per interaction, one main and up to two alternatives.

Examples:

Languages configured	Scenario	Real-time transcription

Languages configured

Scenario

Real-time transcription

en-US (main)
es-US (alternative)

A contact center in the U.S. handles both English- and Spanish-speaking customers.
A customer calls, and the agent answers.
The customer speaks in Spanish, and the agent continues the conversation in Spanish.

Automatic real-time transcription language detection recognizes Spanish and transcribes the conversation in real-time in Spanish.

The same detected language is used for sentiment analysis and summarization.

en-US (main)
es-US (alternative)

A contact center in the U.S. handles both English- and Spanish-speaking customers.
A customer calls, and the agent answers.
The customer speaks in Spanish, and the agent continues the conversation in English.

Automatic real-time transcription language detection recognizes both English and Spanish and transcribes the conversation in real-time in English and Spanish.

The same detected languages are used for sentiment analysis and summarization.

This feature is only available if real-time transcription is also enabled for your account and available for agents.
The underlying AI model controls language-specific punctuation, and it cannot be modified in VCC.
Automatic RTT language detection does not apply to post-call transcripts available in Conversation Analyzer.

You can turn on this feature for an entire interaction plan or a branch within an interaction plan.

The feature can handle the following:

Call participants speaking two languages simultaneously: the caller and the agent can conduct their conversation in two different languages if both languages are configured for the interaction.
For example, the caller can ask questions about their order in German, and the agent can answer those questions in English.
Another agent joining the call: if a new agent joins the call, their spoken language will be detected immediately and applied to the call transcription.

Changing languages mid-call

If the caller or agent switches to another configured language mid-call, it can take up to five minutes for the system to re-detect the new language. During this time, transcription may temporarily pause or continue in the previous language.
If a new agent joins after a language switch, their speech is detected and transcribed immediately. The conversation between the previous agent and the caller in the switched language may not appear in the transcript for up to five minutes.

Configuration for an interaction plan

In Vonage Contact Center:

Navigate to Interaction plans.
Click Manager.
Click Destinations.
Click the Edit button next to the interaction plan for which you want to turn on the language detection.
In the Data source section, add one or two language parameters. Provide both parameter names and language codes.
Click Save mapping.

Configuration for a branch within an interaction plan

In Vonage Contact Center:

Navigate to Interaction plans.
Click Architect.
Choose the interaction plan with the branch for which you want to set language detection.
Click Display Selection.
In your interaction plan, find the Set Data Source applet that starts the desired branch.
In the Data Source Configuration section, add one or two language parameters. Provide both parameter names and language codes.
Click Update.