A voice agent can sound fluent in two languages and still book the wrong date. Multilingual testing needs to check what the agent understands, what it says and what it changes in connected systems.
Start with the language pairs and caller groups your business actually serves. This guide uses English and Spanish examples to explain nine tests that apply more broadly. The objective is successful, verifiable tasks across supported languages, with a clear fallback when the agent cannot continue reliably.
TL;DR
- Test recognition, reasoning, speech output and backend actions separately.
- Include switching languages during a call, accents, noise, names and number formats.
- Have fluent reviewers assess the audio, not only the transcript.
- Confirm critical dates, amounts and identifiers before taking action.
- Report results per language and scenario so a strong English average cannot hide failures elsewhere.
What each test covers
| Test | Main risk to expose |
|---|---|
| Initial language detection | Agent starts in the wrong language |
| Mid-call language switching | Context or intent is lost |
| Accents and speech variation | Supported callers are misunderstood |
| Names and domain terminology | Important entities are changed |
| Dates numbers and identifiers | Correct conversation creates the wrong record |
| Interruptions and corrections | Agent acts on information the caller replaced |
| Speech and transcript consistency | Written and spoken answers differ |
| Tool actions across languages | Backend receives an incorrect value |
| Escalation and unsupported languages | Caller cannot reach a suitable fallback |
Vapi's multilingual documentation describes the roles of transcription, model instructions and speech output providers. Test the actual provider combination in your application; language coverage in one component does not prove the whole workflow is supported.
1 Test initial language detection
Begin calls in each supported language with both a full sentence and a short greeting. Include a caller who uses a product name or personal name from another language. Check whether the opening response and subsequent dialogue follow the intended language policy.
ElevenLabs documents language detection behavior that should be considered when configuring and testing that platform. Detection details are provider specific, so record the configuration rather than assuming automatic switching is universal.
Pass evidence: The audio and recorded language match the intended handling, or the agent politely asks for clarification when the opening is ambiguous.
2 Switch languages during the same call
Start in English, request Spanish, then continue the same task. Reverse the direction in a separate test. Include a natural mixed-language sentence and an explicit request to stay in one language.
Check whether the agent preserves the appointment, customer context and consent state through the switch. It should not restart intake unnecessarily or repeat a completed action.
Pass evidence: The task continues with the same authorized context and the requested supported language. Tradeoff: Automatic switching can feel convenient but may react incorrectly to a quoted foreign phrase.
3 Include accents and speech variation
Recruit fluent reviewers and representative speakers for the regions served. Include ordinary variation in pronunciation, speaking speed and background noise. Keep demographic assumptions out of the pass criteria; assess whether the actual spoken request is handled correctly.
Use consented or appropriately created test recordings and avoid retaining unnecessary customer audio. Synthetic voices can supplement the set but should not be the only source of speech variation.
Pass evidence: Important intent and entities survive realistic audio conditions, or the agent requests clarification instead of guessing.
4 Test names and domain terminology
Include proper names, street addresses, company names and business vocabulary that should not be translated. For example, an English product name may appear inside a Spanish sentence.
Review both recognition and pronunciation. A readable transcript can hide a spoken name that is difficult to understand. Check that the CRM or booking record preserves the confirmed spelling and uses the correct customer identifier.
Pass evidence: The confirmed entity is stored correctly and the caller hears an understandable response. Tradeoff: Pronunciation tuning requires ongoing maintenance as the vocabulary grows.
5 Test dates numbers and identifiers
Use ambiguous dates, currency amounts, phone numbers, appointment times and time zones. A phrase such as “next Friday” requires the current date and business time zone to be resolved deliberately.
Where regional formats differ, ask for confirmation in unambiguous language before executing a booking or other important action. Do not rely on translating a formatted string and hoping the downstream parser reaches the same interpretation.
Pass evidence: The confirmed value matches the backend value and the spoken confirmation. The test record includes the applicable locale and time zone.
6 Interrupt and correct important information
Have the caller replace a date or name while the agent is speaking. Include a correction after a tool lookup but before the final confirmation. Check whether the system cancels or updates its pending action correctly.
A correction such as “Tuesday, sorry, Wednesday” should be reflected in the final confirmed request. Retest corrections in each language rather than assuming the English behavior transfers unchanged.
Pass evidence: The system acts on the confirmed corrected value and avoids duplicate changes. Use the voice prompt design guide for related conversation controls.
7 Compare speech with the transcript
Listen to the actual caller experience. Review number pronunciation, negation, pauses and whether the agent reads punctuation or internal formatting aloud. Confirm that a refusal, limitation or important condition remains clear after translation.
Store the review language and reviewer notes with the result. Automated transcript scoring helps locate problems but can miss pronunciation and timing issues.
Pass evidence: The spoken answer communicates the same intended facts and conditions as the verified text. Tradeoff: Human listening takes time, so prioritize high-impact scenarios and known failure areas.
8 Verify backend actions in every language
Run the same booking, lead qualification or support task through each language path. Compare tool arguments, authorization checks and final state. A Spanish conversation should not weaken validation or bypass a required approval.
Check that free-text notes retain the necessary meaning and that structured fields use a stable schema. The CRM integration guide provides context for separating conversational text from system fields.
Pass evidence: The final record, permitted action and audit trail are correct, regardless of the language used to request them.
9 Test escalation and unsupported languages
Ask for a human in each supported language, then try a language outside the configured scope. Check the message, routing destination and what happens when no suitable person is available.
Microsoft's multilingual voice configuration guidance illustrates why channel and agent language settings need to align. Verify the actual staffing and transfer configuration used by your organization.
Pass evidence: The caller receives an understandable limitation and a usable approved fallback. The agent does not claim language support that the deployed workflow cannot provide.
A sample review record
| Field | Example value |
|---|---|
| Scenario | Reschedule after a language switch |
| Languages | English to Spanish |
| Critical entity | Confirmed appointment time |
| Audio condition | Mobile call with moderate background noise |
| Required outcome | One authorized reschedule with correct time zone |
| Evidence | Call audio transcript tool arguments and final booking |
| Reviewer decision | Pass fail or clarification required |
These entries describe an illustrative test design, not results from a deployed agent. Use the same core scenario across languages, then add language-specific edge cases identified by fluent reviewers.
Pros and cons of multilingual automation
The advantage is consistent availability across the configured language set. The operating cost is maintaining speech configurations, localized content, review coverage and a suitable human fallback. A broader advertised language count is less useful than a smaller set of workflows that have been tested well.
Use the bilingual receptionist comparison when selecting options, then apply these tests to the exact configuration you plan to deploy.
Frequently asked questions
Is transcript accuracy enough
No. Evaluate audio intelligibility, critical entity accuracy, completed tasks and backend state. A transcript alone cannot show everything a caller hears.
Should every supported language have the same tests
Use a shared core suite, then add locale-specific cases. Different number formats, names and accents need targeted coverage.
Can synthetic callers replace fluent reviewers
They can increase repeatable coverage, but fluent reviewers are still useful for naturalness, pronunciation and ambiguous meaning.
What should happen when language detection is uncertain
Ask a clear clarification question or use the approved fallback. Guessing is especially risky before an action involving dates, money or identity.
Expand languages after verifying the workflow
Launch with the languages and tasks the team can support and monitor. Explore LuMay Voice Agents to discuss multilingual call design and integrations, and use voice agent KPIs to track the resulting service.
Sources and further reading
- Multilingual support | Vapi
- Language detection | ElevenLabs Documentation
- Configure multilingual voice agents | Microsoft Learn
Sources and linked destinations checked on 1 November 2026. Product capabilities and documentation can change.
