State of conversational voice agents · Healthcare first, with notes for financial services

October 2026 edition

Voice agents at the front door

Voice agents are now good enough to take routine calls in regulated contact centers: scheduling, billing questions, balance and status checks. They are not yet consistent enough to run unsupervised on clinical or high-risk financial conversations. In one vendor-run benchmark, the leading platform still failed about one in four test scenarios when each was run three times.#

The one-minute read

What we know. The best platform passed 75.6% of test scenarios on all three tries; the top three are within the margin of error. Every published deployment result is administrative and self-reported.

What to watch. Demos overstate readiness, cloned voices defeat voice-only identity checks, and scores move so fast that any model bet goes stale in a quarter.

What to do. Build a golden test set from your own calls and gate every change on it. Pilot narrow, judged on cost per resolved call. Separate silent decisions from the conversation, and build a governed fact store; trial limited-memory models only once they beat retrieval.

56%

Lower call abandonment at a large health system within two weeks of go-live [1]

self-reported

75.6%

Best platform's repeat reliability: share of 82 scenarios passed on all three tries (about 65% to 84% at 95% confidence) [2]

vendor-reported

Of CX, IT and operations leaders can quantify AI's impact on business outcomes, though 98% have deployed it in the customer journey [3]

vendor-reported

Over half

Of fraud attempts at healthcare contact centers now involve AI-generated elements such as synthetic voice [4]

vendor-reported

Thesis 1Reliability is the gap

In one benchmark, single-run scores beat repeat reliability by 18 to 38 points on six of seven platforms. Test repeatability on your own calls.

Thesis 2A thinker-responder agent

A fast System-1 thinker decides silently; a conversational LLM speaks. People don't narrate every thought before they answer.

Thesis 3Invest in governed memory

A governed fact store pays off today. Limited-memory models that look facts up are promising but research-stage: trial them against retrieval first.

Evidence labels used throughout: independent vendor or self-reported unverified author's view, arithmetic or illustration Charts and tools you can use carry interactive

How to read this report: method, limits and disclosure

Method. A review of public sources up to the research cutoff. I did not run my own tests; every figure links to its source.
Which vendors. Platforms and models appear when they publish benchmark results or deployment outcomes, or run at large health systems or banks. Sierra is included in the market analysis wherever evidence exists, because of its healthcare footprint. It was not part of the seven-platform reliability test, and that gap is noted rather than filled in.
How much weight. Peer-reviewed and independent tests first, then vendor benchmarks with a published method, then customer-reported outcomes, then vendor claims. The label on each figure says which it is.
Illustrative tools. The router, latency, economics and limited-memory demos are models for exploring trade-offs, not measurements of a product.
Recommendations. Thresholds, pilot gates and suggested paths are proposals to adapt to your risk, call mix and baseline.
Disclosure. Written in a personal capacity; the views are my own, not any employer's. No commercial relationship with any vendor named, and no sponsorship.
Corrections. Post them in the comments or as a GitHub issue; changes are logged in the changelog.

Executive summary · about ten minutes

What this report argues, in plain terms

Voice agents can now take routine calls in regulated contact centers. What separates the teams that succeed is not which model they buy, but how they test, how the agent decides, what it remembers, who it is listening to, and how it proves who the caller is. Nine points, each linked to the evidence.

Ready for routine calls, not for unsupervised high-risk ones

Agents now handle scheduling, billing questions and status checks well. In one public benchmark, the best platform completed over 90% of scenarios on a single try, but passed only 75.6% when each scenario had to succeed three times in a row.[2] The gap is consistency, not capability. Health systems report large gains, such as waits cut from 6.2 to 2.4 minutes and 44% of repetitive calls automated, but every published result is administrative and self-reported.[5][6]

Reliability and results

Evaluation is the asset that lasts

Models change every month; the top score on one voice benchmark rose from 30% to 67% in eight months.[7] What you keep is a golden test set: hundreds of real, de-identified call scenarios with known right answers, including noisy lines and upset callers, run three times on every change. A drop in score blocks the release, the way failing tests block code. Build it before choosing a vendor: it is how you compare vendors, gate releases and hold contracts to account.

The golden test set

Think silently, then respond

People size up a question before they answer it. A voice agent should too. A small, fast "System 1" model answers typed questions in tens of milliseconds, each with a confidence score: which team owns this, is there fraud or clinical risk, is the caller finished, is it even the caller speaking. It never writes prose. A conversational LLM then responds, with the right data already fetched.

This is faster and cheaper: in one test, routing first kept accuracy at about 84% while cutting cost from $2.42 to about $0.69 per 1,000 requests.[8] It is also safer, because every decision is a logged record, and fraud, clinical and complaint signals always go to a person by rule. It works beside any voice stack, and it can cut the audio mid-sentence if it spots risk.

The thinker-responder model

Hear the right caller

Real calls come from cars, kitchens and waiting rooms. In one vendor test, background speech pushed transcription errors to 22% to 31%, and isolating the main speaker cut them by up to 43%.[9] Diarization separates the voices, so the agent locks onto the caller and checks identity again if a different voice takes over before a sensitive step. Test with noise and side conversations, not clean demo audio.

Noise and diarization

Emotion and persona: worth piloting, not yet proven

Detecting frustration in real time lets the agent slow down or hand off to a person, and scoring sentiment on every call replaces quality reviews that sample a few percent. A persona is the brand's voice: offer callers a choice, then keep each caller's preferred voice consistent. The evidence is still mostly vendor case studies, and the EU requires telling people when emotion recognition is used,[10] so run both as experiments behind the golden test set.

Emotion and persona

Build governed memory now; train limited-memory models later

Turn plans, policies, fees and provider details into a governed fact store: versioned rows with owners, access control and deletion. It powers retrieval today. Limited-memory models go further: they are trained to look facts up in that store instead of memorizing them. In research, a 382M-parameter model approached a 7B model on factual precision, and deleting a row made the model forget the fact.[11] That research is early (Wikipedia facts, small models, no voice or production use), so trial it on one domain once the store is ready, and scale only if it beats retrieval. Live data such as balances and appointment slots always comes from your systems through tools.

Governed memory

A voice is no longer proof of identity

Over half of fraud attempts at healthcare contact centers now involve AI-generated elements such as synthetic voice.[4] Match the check to the risk: passive signals for status checks, a one-time code for account changes, voiceprint plus liveness detection for irreversible actions, and always a person as the fallback. Card numbers should never reach the agent at all. For outbound calls, consent belongs to the current subscriber, so check lists against the FCC's Reassigned Numbers Database.[12]

Trust and identity

The money is in resolution, not token price

Savings come from calls the agent truly resolves, meaning no repeat contact on any channel within 48 hours. The calls left for people are the harder ones, so honest models show smaller savings than vendor pitches. With this report's default assumptions, the agent breaks even at about 40% resolution, and each extra point of resolution is worth far more than cheaper tokens. Contract on price per resolved call, with rights to export logs and evaluation data.

The economics model

What to do in the next 90 days

Pick two or three inbound administrative call types in one market. Build the golden test set (200+ scenarios). Set the go-live bar by risk: about 85% repeat reliability for informational calls, 95% for anything that writes data or moves money, and zero critical failures. Transfer to a person with full context. At day 90, judge on cost per resolved call, abandonment and zero wrong-account actions. Whatever the outcome, keep the test set, routing logs and recordings: they are the most valuable thing the pilot produces.

The pilot playbook

1 · Reliability

A demo call is not a production call

The same test agent was run on seven voice platforms: 82 realistic caller scenarios (appointments and Medicare), three repeats each, on live calls. Each provider chose its own configuration, except OpenAI, which Cekura configured directly. Single-run task completion looks ready. Passing every repeat is the number that matters for customers and patients.[2][13]

Passed all 3 tries (pass^3) Single-run task completion The demo gap

Cekura is a voice-testing vendor and the scenarios are generic. Sierra, a leading enterprise agent platform, was not among the platforms tested; its own tau-voice benchmark reports single-run scores, not repeat reliability.[7] A scenario counts as failed if any of its three runs fails the evaluator's checks or the call does not connect; the benchmark does not break failures down by type. With 82 scenarios, each score carries a margin of about plus or minus 9 points, so the top three platforms are effectively tied. Treat this as directional, then rerun the test on your own call types, including any Sierra deployment.

2 · Published results

Where it is already working: access and service, not advice

The strongest published results so far come from health systems, and every one is administrative. All figures are reported by the organization or its vendor, with no control group, so expect some regression at scale.

Tampa General · Hyro 2.4 min

Average wait, down from 6.2 minutes, within two weeks

Wait, before

6.2 min

Wait, after

2.4 min

Abandon, before

34%

Abandon, after

14.9%

Also 21% more appointments scheduled. [1][5]

self-reported

WellSpan · Hippocratic AI 160K+

Patient calls a month handled by an AI agent

About 7,000 hours of conversation; partnership expanded July 30, 2026. [6]

self-reported

Intermountain · Hyro + Agentforce Health 44%

Of repetitive calls automated

Call abandonment down 85% on in-scope lines such as patient portal support. [6]

self-reported

For financial services, I expect the same pattern: balance, card status, payment dates and dispute intake first; advice, fraud adjudication and hardship conversations stay with people. author's view CX, IT and operations leaders report the same gap: 98% have deployed AI in the customer journey, but only 5% can quantify its impact on business outcomes.[3] vendor-reported

Enterprise agent platforms are moving into healthcare too. Sierra says it serves most Fortune 50 healthcare companies, and announced new voice models at Sierra Summit 2026. No healthcare outcome figures have been published yet.[14] vendor-reported

3 · Pace of change

2026 moved fast. Snapshots go stale in a quarter.

On Sierra's voice benchmark, the best score rose from 30% to 67% in eight months.[7] Lock in your data and evaluations, not a model.

4 · Economics

The lever is resolution rate, not token price

Model your own contact center. Every in-scope call reaches the agent first; calls it cannot resolve go to a person, and those tend to be the longer, harder ones. Some calls the agent closes come back as repeat calls. Defaults use published per-minute pricing[24] and a fully loaded human cost per call[25]. Platform, integration and team costs are a fixed annual line.

Interactive · drag sliders or pick a preset

Daily inbound calls, all queues

Share in scope for the agent Scheduling, status checks, billing questions, routing

Average minutes per call

AI cost per minute, all-in Managed stacks run about $0.08 to $0.14

Resolved end to end by the agent Count a call only if there is no repeat contact on any channel within 48 hours; hang-ups are not resolutions

Human cost per call A published illustration: about $6 at six calls an hour, $9 at four

Escalated calls cost more than average The calls left for people are the harder ones

Agent-closed calls that come back Repeat calls go to a person

Fixed annual cost: platform, integration, team

Updates live · chart dot follows the resolution slider

Human only, monthly

AI + human fallback

Net monthly savings

Break-even resolution

Agent cost per call it resolves

Human cost per call

author's arithmetic Illustrative only. The escalation and repeat-call defaults are assumptions, not sourced figures; use your own. Worst case: resolution 15 points lower, escalations 15 points costlier, 5 points more repeat calls. Best case: resolution 10 points higher, escalations 10 points cheaper, 3 points fewer repeat calls. A published comparison found the pricier stack with higher resolution ($0.14/min, 70% resolved) costs less per 1,000 calls than the cheaper one ($0.09/min, 55%).[24] Cost models that count only LLM token savings miss most of the money.

5 · Think silently, then respond

A thinker-responder model for conversational agents

People don't say every thought out loud before they answer. We size up the question, decide what it's about, recall what we need, and then speak. A voice agent should work the same way: a fast System-1 model is the thinker, making typed decisions silently, and a conversational LLM is the responder, turning those decisions into a natural reply.[22][20][19]

Caller: "I got charged twice for my visit last month."

What the thinker decides (System 1, silent, typed)

Intent: which sub-agent owns this turn (billing, scheduling, complaint)
Risk: fraud, clinical, vulnerable-caller or complaint flags that force a handoff
Tools: which MCP servers to expose, and which facts to fetch before speaking
Turn state: is the caller done, correcting themselves, or just saying "mm-hmm"
Speaker: is this the caller, or someone else in the room
Emotion: frustration or confusion that should change the tone, slow things down or hand off
Confidence: a probability on each, so low-confidence turns hand off instead of guessing

Why it works

Faster: decisions take tens to hundreds of milliseconds, so the responder starts with data already fetched.
Cheaper: the expensive model only writes the reply, not the routing.
Safer: the "thought" is a structured record, not prose, so it can be logged, audited and gated by rules. The responder can't narrate a tool result it never received.[23]
A deliberate difference: OpenAI and Sierra put the fast model on the talking side and a slower reasoner behind it.[26][14] This design puts the fast model on the deciding side. The two combine: a fast talker, a fast silent decider, and a slower reasoner when needed.
Works with any voice stack: the thinker does not need a speech-to-text pipeline. Beside a speech-to-speech or full-duplex model it reads the transcript stream and decides which tools and policies apply, so the voice model keeps its natural turn-taking. Because a full-duplex model starts speaking within a few hundred milliseconds, the thinker also needs a stop control: when it flags risk mid-turn, it cuts the audio output at once, before protected data or advice reaches the caller.
Limit: the thinker handles bounded choices. Open-ended reasoning still needs a deeper planner the responder can call.

Pick a caller turnInteractive · tap a turn, then move the threshold

Confidence threshold Below the threshold, the router hands off instead of guessing

illustrative Probabilities are examples of a calibrated multi-label router, not output from a specific product.

Routing speed, the cost of a cascade and a check of this thesis against the evidence are in the technical appendix (T5).

6 · What separates a good agent

Two foundations, and two experiments worth running

Models change every month. Two capabilities decide whether an agent holds up on real calls, and they carry over from one model or vendor to the next. Two more are promising but early: pilot them behind the golden test set.

Foundation · A golden test set and an evaluation framework

The most valuable asset a team builds. Hundreds of real, de-identified call scenarios, each with a known right answer, run three times on every change to models, prompts, tools or personas.

Common guidance: 200+ scenarios before production, 500+ for multilingual enterprise use, [27] vendor-reported To detect a 5-point change in a pass rate near 80%, plan for several hundred runs per version, not the 100 sometimes quoted. author's arithmetic
Mix happy paths with edge cases, errors, adversarial callers and noisy audio, and add every production failure as a new case.
Block a release when a score drops, the same way failing tests block code.

Foundation · Hearing the right caller

Real callers phone from cars, kitchens and waiting rooms, with a TV on or someone talking beside them. How the agent performs there matters more than how it performs in a quiet demo.

Background speech pushed streaming speech-to-text error rates to 22% to 31%; isolating the main speaker cut errors by up to 43%.[9] vendor-reported
Separate the voices (diarization), lock onto the caller, and re-verify identity if a different voice takes over before a sensitive step.[28]
Test at several noise levels and with side conversations, not just clean audio.

Experiment · Emotion in real time, sentiment on every call

During the call, frustration or confusion should change how the agent speaks or trigger a warm handoff. After the call, sentiment scored on every call replaces quality reviews that sample a few percent.

Treat emotion scores as a signal for routing and review, not a judgment of the caller. Accuracy varies, so check that they work equally across accents and age groups.
In the EU, people must be told when an emotion recognition system is used.[10]

Experiment · Persona is the brand

Callers judge the organization by how its agent sounds. Voice, pace, warmth and word choice deserve the same care as a brand guide.

One subscription business reported resolution rates up more than 50% after tuning its voice persona.[14] vendor-reported
Offer callers a choice of a few personas, then keep each caller's preferred voice consistent on every call. author's view
Every persona says it is an AI, never implies a professional license, and gives the same facts and policies.[29]

author's view The experiments have little evidence so far: mostly vendor case studies, and none yet shows emotion detection or personas improving AI-agent outcomes at scale.

7 · Where to start

Readiness against risk, by call type

Healthcare shown, because that is where results are published. Green is the pilot zone. Red stays human until there is independent safety evidence.

Interactive · tap a call type for details

Start here Next wave Clinically supervised only Not yet

Call type	Zone	Reported outcome	Example vendors
Confirm, cancel, reschedule	Start here	Abandonment down 56%, appointments up 21% at Tampa General	Hyro, Assort, Hello Patient, Zocdoc Zo
Routing and FAQ, portal help	Start here	44% of repetitive calls automated, abandonment down 85% at Intermountain	Hyro, Talkdesk, Sierra
Ambient documentation	Next wave	Adjacent category, widely piloted for clinical notes and handoffs	Abridge, Microsoft/Nuance, Commure
Refill status	Next wave	Included in Tampa General scope; no isolated metric found	Hyro, Assort
Billing and revenue cycle	Next wave	Billing agent live in 40 days at one urgent care network (vendor-reported)	Hello Patient, Infinitus, Prosper
Prior auth and payer calls	Next wave	Positioned by vendors; no outcome metric found	Infinitus, Prosper, Sierra
Post-discharge follow-up	Supervised only	9.0/10 patient rating reported by vendor; heavy clinical supervision	Hippocratic AI
Screening outreach (outbound)	Supervised only	Higher kit requests among Spanish-speaking patients (vendor-reported); consent rules apply	Hippocratic AI
Nurse triage lines	Not yet	No validated autonomous deployment found; vendors exclude diagnosis	None recommended

8 · Trust and identity

Over half

of fraud attempts at healthcare contact centers now involve AI-generated elements such as synthetic voice, bots and IVR reconnaissance.[4] vendor-reported

A voice is no longer proof of identity

This is a brand question as much as a security one. Customers will be targeted by callers pretending to be you, and your agent will hear callers pretending to be customers.[30]

Liveness and deepfake detection in the contact center platform, before any protected data.
Step-up verification (one-time code to the number on file), not birth date or security questions alone.
For payments, prescription changes and other authorizations, add voiceprint verification with liveness detection; a voiceprint alone can be fooled by a clone.[31]
Match the friction to the risk: passive signals for status checks, a one-time code for account changes, voiceprint plus liveness only for irreversible actions. Liveness checks over phone-quality audio reject some genuine callers, and texted codes lose some older or distressed callers, so always offer a person as the fallback. author's view
A published "we will never ask you for" policy, owned by marketing.
In the US, outbound AI-voice calls count as artificial-voice calls under the TCPA since the FCC's 2024 ruling, so they generally need prior express consent; healthcare exemptions are narrow, so check reminder programs with counsel.[32] Consent belongs to the current subscriber, so check outbound lists against the FCC Reassigned Numbers Database before every campaign.[12]

9 · What comes next

Models that look facts up instead of memorizing them

A Limited Memory Language Model (LMLM) is trained so that specific facts, such as a plan's coverage or a card's limit, live in an external database, not in the model's weights. During training the model is not rewarded for memorizing those values; it learns to issue a lookup and copy the answer. Language and reasoning stay in the model. Facts stay in a table people can inspect, correct and delete.[11] It suits slow-changing knowledge such as benefits, policies and fees; live data such as balances, claims and appointment slots stays on tool and MCP calls.

Caller asks

"What's the coverage limit on my plan, member 4471?"

Agent answers

illustrative The deletion behavior mirrors the paper's unlearning result: removing records from the store made the model forget them with no loss of general ability, where gradient-based unlearning degraded the model.[11] Logs, transcripts, caches and backups still need their own deletion.

Why a governed fact store fits regulated conversations

Facts become governed data. Each one is a row with access control, versioning and change review.
Deletion from the model is real. Removing rows forgets them without retraining.[11] Copies in logs, transcripts and backups need their own retention rules.
Less sensitive data in weights. Researchers have extracted names, phone numbers and emails from model weights, even from data seen once.[33][34]
Every answer has provenance. Each lookup is a logged call with the value it returned.
Room to shrink the model. In research, a 382M limited-memory model came close to a 7B model on factual precision; details are in the technical appendix (T6).[11]

My view: build the fact store now, and treat limited-memory training as a gated bet

Retrieval on a frontier model is the safe default today, and the evidence for limited-memory models is still research-stage: Wikipedia facts, models under 400M parameters, no dialog, voice or production use. A governed fact store pays off either way. Training is a bet to stage behind gates, not a commitment.

Now · no-regretBuild the governed fact store

Turn plans, policies, products, providers and fees into versioned rows with owners, access control and deletion. It powers retrieval today and LMLM tomorrow.

When the store is ready · trialTrial a small model on one domain

Start from the open LMLM code (MIT) on one de-identified domain, such as benefits or product FAQs. Go further only if it beats retrieval on factual precision, deletion, latency and total cost.[35]

Only if it wins · scalePut it behind the responder

Expand domains for static knowledge. Live balances, claims and appointment slots always stay on tools.

Compute is not the hard part: the paper trained each model in at most 8 H100-days[11], roughly $400 to $500 at $2 to $2.69 an hour for rented H100s[36]. author's arithmetic Budget for data engineering, store quality and evaluation instead.

What the research does and does not show yet, claim by claim, is in the technical appendix (T6).

10 · The decision

Four ways to play the next 12 months

Interactive · tap an option

Buy a platform or vertical vendor Sierra, Hyro, Hippocratic AI

Pros

Fastest time to value
Integrations and compliance contracts are standard
Vendor carries tuning, safety layers and ops

Cons

Opaque pricing
Less control over evaluation, routing and data
Lock-in at the integration layer

ROI logic: Abandonment and scheduling lift convert to revenue; labor avoided on in-scope queues. Verdict: Strong for the pilot itself.

Build on a platform Retell, LiveKit, Pipecat

Pros

Full transparency and model choice
Lowest per-minute cost
Own every trace and the router

Cons

Needs a dedicated team
You own integration and safety
Slower first launch

ROI logic: About $0.08 to $0.14 a minute against roughly $6 a human call; break-even depends on resolution rate. Verdict: Right later, not first.

Hybrid (recommended) Buy now, own the router and evals

Pros

Quick wins from a vertical vendor
You build the test suite, routing layer, logs and governed fact store in parallel
Builds the governed fact store, with the option to trial limited-memory models
Keeps the exit door open

Cons

Two workstreams to govern
Needs contract terms for log and eval export

ROI logic: Captures near-term savings while building leverage for the next contract cycle. Verdict: Recommended.

Wait Revisit in 2027

Pros

No execution risk
Models and benchmarks keep improving

Cons

Peers compound gains
IVR frustration persists
No internal evaluation muscle

ROI logic: Negative: forgone abandonment and scheduling gains. Verdict: Not recommended.

11 · A pilot playbook

A 90-day pilot with gates set before launch

Teams are at different points on this path. If you are starting out, expanding to new call types or resetting a stalled rollout, this is one way to run it; if you are already in production, the golden test set, gates and kill criteria work as an audit of what you have. The shape: one market or line of business, inbound only, two or three call types, warm transfer with full context. The pre-launch bar is set above the best public benchmark on purpose, because public scenarios are generic.

Weeks 0 to 4Build the golden test set

200+ scenarios from your own de-identified calls: accents, older callers, Spanish, number dictation, mid-call corrections, background noise and chatter, upset callers. Silent shadow test on recorded audio.

Gate · end of week 4Pass^3 bar set by risk

About 85% for informational calls, 95% for anything that writes data or moves money, and zero critical failures at any rate. Every scenario run three times on phone-grade audio. Router thresholds calibrated on held-out data. Identity controls signed off.

Weeks 5 to 12Live in one market

Every failure becomes a regression test. Weekly audit of tool-call write-backs and misroutes (transfers after routing).

Day 90Scale, adjust, or stop

Decide against the criteria below, not against a demo.

scale ifSuccess at day 90

Abandonment down 30% or more on in-scope queues
50% or more of in-scope calls resolved without a person, with no repeat contact on any channel within 48 hours (hang-ups don't count)
Zero wrong-customer or wrong-account write-backs on audit
Satisfaction at or above the human baseline
Recording, transcript, routing decision and tool trace on 100% of calls

stop ifKill criteria, any time

Any protected data disclosed to an unverified caller
Any advice beyond approved content: clinical guidance, or a financial recommendation outside an approved script
Wrong-action rate above 0.5% of tool calls on audit
A vendor refuses to export logs or evaluation data

author's view Starting points tied to the published results above, not an industry standard. The pre-launch bar is a risk-management choice, not a benchmark-derived number: a narrow golden set of two or three call types is easier than a broad generic benchmark, so it can sit above the best public score (75.6%), and the bar rises with the harm a failure can do. A high overall pass rate never offsets a critical failure, such as protected data given to the wrong caller or a wrong payment; one of those blocks launch. With 200 scenarios, a measured 90% carries a margin of about plus or minus 4 points. 30% lower abandonment is about half what peers report (56% to 85%). 50% resolution is in the range of reported automation (44% at one health system). Tighten the bar for higher-risk call types, and lower the resolution target for complex queues.

Gates met: scale in waves

Months 4 to 6Widen

Add two or three call types and one more market at a time, each through the same pass^3 gate
Retrain the router on pilot transcripts; you now have real labeled intents
Outbound only where consent is on file

Months 7 to 12Own

Move routing and evaluation in-house; keep vendors on the responder and voice stack
Renegotiate on price per resolved call, with log and routing export rights
Trial a limited-memory model on one knowledge domain, if the fact store is ready

OngoingGovern

Monthly review of misroutes, wrong actions and complaints
Change control: re-run the golden set before any model, prompt or persona change
Grow the golden set every week from real failures
Score sentiment on every call and review the angriest calls weekly
Report resolved calls, abandonment and satisfaction to the board quarterly

Some gates missed: fix, then re-run

Weeks 1 to 4Diagnose

Rank failures from the logs; most trace to audio, identity or integration, not the model
Narrow scope to the call types that passed

Weeks 5 to 8Fix

Swap the weak component (speech-to-text, turn detection, router thresholds) rather than the whole platform
Add every failure to the test suite

Weeks 9 to 20Re-run

Repeat the 90-day pilot on the narrowed scope with the same gates
Scale only after a clean pass

Kill criterion hit: stop and keep the learning

ImmediatelyContain

Turn the agent off for affected call types; route to people
Notify compliance and run an incident review

Next 30 daysLearn

Write a blameless post-mortem: what failed, why the gates did not catch it earlier
Keep the test suite, logs and recordings

Next editionRevisit

Re-evaluate when the failing capability improves; benchmarks move every quarter
Continue building the governed fact store; it pays off regardless

author's view Whatever the outcome, keep the test suite, routing logs and call recordings. They are the most valuable thing the pilot produces.

That is the executive read. The think tank and the technical appendix that follow are for readers who want the debate and the detail. Keep reading · About this report

12 · Think tank

Seven seats at the table

A structured debate on one question: what should a regulated enterprise do about voice agents in the next 12 months, and how much should it bet on fast routers and governed-memory models? The seven seats are synthetic perspectives written to stress-test the argument, not a survey of real people.

Interactive · tap a seat to hear that view

Chief Digital & CX Officer

Expand in phases. Voice is the front door, and it is now also a brand-trust channel.

Evidence: A large health system cut abandonment 56% and scheduled 21% more appointments in two weeks. Concern: One bad call becomes the story. Scammers posing as your brand erode trust in every legitimate call. Would change their mind: Pilot satisfaction falling below the human baseline.

CFO

Fund a pilot, not a platform. Pay for resolved calls.

Evidence: The higher-resolution stack is cheaper per 1,000 calls despite a higher per-minute rate. A decision-first cascade cut classification cost about 72% in one test. Concern: Vertical vendors rarely publish pricing. Early-access model prices may be subsidized. Would change their mind: Verified cost per resolved call below half the human cost at day 90.

CISO & Privacy Officer

Hosted APIs only under contract. Self-host the small models. Prefer facts in governed stores.

Evidence: Hosted decision APIs need a BAA or equivalent; self-hosted encoders avoid the question. Extraction attacks have pulled personal data from model weights. Concern: Voice cloning, prompt injection, retention terms that change. Would change their mind: A signed contract and SOC 2 Type II from every layer, testing vendor included.

Contact Center Operations

Start with scheduling, billing and status checks; warm transfer with full context.

Evidence: Intermountain: 44% of repetitive calls automated and 85% lower abandonment. Concern: Wrong write-backs, misroutes, authentication failures, staff morale. Would change their mind: Misroute or wrong-action rate above 0.5% on audit.

Clinical & Risk Safety Lead

Administrative only. No triage, no advice. Escalation by rule, not by threshold.

Evidence: No validated autonomous triage deployment found. Best models reach only about 53% on multi-intent benchmarks. Concern: Missed red flags, older and non-English callers, regulatory exposure. Would change their mind: An independent safety evaluation with prospective outcomes.

AI Platform Architect

Thinker-responder: tiered router in front, narrowed tools per sub-agent, and a governed fact store, with a limited-memory trial for static knowledge once it beats retrieval. On a managed platform such as Sierra, insist on exporting routing decisions and logs.

Evidence: A 0.1B fine-tuned classifier hits 93.5% on Banking77 in 22 ms. Tool retrieval more than tripled selection accuracy. A 382M LMLM approached a 7B model on factual precision. Concern: Lock-in at orchestration and integration layers; model churn every few weeks. Would change their mind: A vendor refusing to export logs or routing decisions.

Skeptic / red team

Do not trust vendor benchmarks or one-lab results. Demand your own repeat-reliability numbers.

Evidence: Jev's Banking77 score varies 10.7 points by lab. Benign context flipped 61% of correct decisions. LMLM is untested above 400M parameters or on dialog. Sierra publishes single-run voice scores, not repeat reliability. Concern: Every outcome figure here is customer- or vendor-reported, with no control group. Would change their mind: 90%+ pass^3 on your own scenarios, audited by an independent party.

Where the perspectives converge

Start with inbound administrative calls and warm human transfer.
Build a golden evaluation set before choosing a vendor, and gate every change on it.
Measure repeat reliability and cost per resolved call, not containment or demo quality.
Use a fast router, with calibrated thresholds and rule-based escalation.
Own the evaluation data, routing logs and call logs, whichever vendor wins.

Still unresolved

Fine-tuned classifier (cheapest, most accurate on fixed intents) versus a hosted decision API (no training data, adapts to new intents).
When a private limited-memory model beats retrieval on a frontier model, on total cost.
Speed versus safety on outbound calls, where consent rules bite at scale.

Part II · Technical appendix

For architects, platform and AI engineering teams

Architecture, latency, reliability measurement, failure modes, System-1 routing, limited-memory models, deployment and data posture, fraud controls, regulation, and a check of claims circulating in the market.

T1 · Architecture

Three generations, converging on "talk fast, think slow"

OpenAI's GPT-Live removes the separate turn detector and delegates search and reasoning to a frontier model behind an asynchronous boundary, so a slow tool call cannot stall audio.[26][37] Sierra's Tandem Voice pairs a real-time conversation layer with a reasoning layer.[14] Gemini 3.8 Live calls tools in the background.[38] A typed System-1 router fits beside any of these as the dispatch layer.

Dimension	Cascaded (STT, LLM, TTS)	Native speech-to-speech	Continuous full-duplex
Examples	Pipecat, LiveKit, Retell; Agentforce Voice[39]	gpt-realtime-2.1; Gemini Live API	OpenAI GPT-Live; Gemini 3.8 Live; Sierra Tandem Voice
Turn-taking	VAD plus turn detector	Turn detector still decides when to start	Model controls turns, can backchannel
Tone and prosody	Lost in transcription	Kept	Kept
Tools and reasoning	Inside the LLM step, blocking	In-model or blocking	Async delegation to a frontier model
Audit trail	Native text at every step	Transcripts available	Transcripts derived after the fact
Control and vendor choice	Highest	Medium	Lowest today (single vendor)
Fit for regulated work	Per-step audit and per-component contracts	Simpler flows with one covered vendor	Watch closely; GPT-Live API announced as upcoming[26]

Production reality: calls arrive over SIP trunks as compressed 8 kHz audio, with jitter and packet loss, so test on that audio rather than clean recordings. Integrations with scheduling, CRM and EHR systems are usually the slowest and most brittle part of a launch; budget for them first. author's view

Reference flow (cascaded, with a System-1 router)

Audio in

Caller (phone, 8 kHz)→Echo cancel→Isolate caller's voice + diarize→Speaker-aware VAD + turn model→Streaming STT

Thinker · System 1

Router: intent, risk flags (typed probabilities)→Pick sub-agent + its MCP servers→Fraud, clinical, complaint or low confidence: escalate

Responder · LLM

Sub-agent LLM with narrowed tools→Facts from governed store or system of record→Read-back before any write→TTS

Always

One correlation ID across VAD, STT, router, LLM, tool and TTS events→Replayable trace

T2 · Latency budget

Where the seconds go in a turn

Human turn gaps across ten languages cluster around a quarter second.[40] Measured platform turns in the benchmark averaged 1.27 s to 3.08 s.[2] Adjust the stages, or pick a preset.

Be candid about the trade-off: a cascade (speech-to-text, LLM, text-to-speech) takes turns like a walkie-talkie. Even with a fast router and prefetch it lands near 1.5 s a turn, and cutting endpointing further clips callers mid-sentence. If turns under 800 ms are a hard requirement, use a speech-to-speech or full-duplex model and accept a coarser, after-the-fact audit trail. The thinker fits beside either.

illustrative Sourced anchors: fixed silence endpointing of 500 to 800 ms is still common[23]; the fastest LLM first-token time in Regal's 27-model test was 584 ms[41]; a self-hosted router costs 20 to 50 ms. Run the router in parallel with end-of-turn detection, and route once at call start or on topic shift, not every turn. Commit a route only on the finished turn, and route again when the caller corrects themselves ("cancel... no, just move it"); any prefetch is speculative and dropped on a re-route.

Caller experience beats raw model quality

In Regal's benchmark (27 models, 80 scenarios, 316,311 scored samples), the ten slowest LLMs scored about 128 points lower on a 1,000-point scale despite slightly better task progression.[41] vendor-reported

T3 · Measuring reliability

pass^k: why single-run numbers flatter every vendor

pass^k counts a scenario only when all k runs succeed. It came from Sierra's tau-bench and now appears in Cekura Bench and tau2-bench voice mode. Even assuming independent runs, small per-call error rates compound quickly.

Interactive · drag the slider

Per-call success probability

Same model, different harness

In Cekura's speech-to-speech benchmark (11 models, 2,952 calls), GPT Realtime 2.1 alone reached 79.3% pass^3, while its mini sibling scored 50.0% and saved all fields correctly on only 64.5% of data-saving calls versus 92.0%.[42]

Recovery is its own axis

A 2026 interruption benchmark found task completion and recovery quality only partly related.[43] Test barge-in and correction separately from happy paths.

Release scorecard

Perception: word error rate on phone-grade audio, by accent and language, and on noisy, multi-speaker recordings (TV, car, a family member talking); entity accuracy for names, dates, phone and account numbers.
Routing: intent accuracy, out-of-scope recall, misroute rate (transfers after routing), calibration error.
Turn-taking: false and missed barge-ins, backchannel handling.
Latency: P95 turn time, not mean; dead air over 10 s.
Action correctness: exact argument match on every tool call. This is the hard gate.[23]
Repeatability: pass^3 on a golden set of 200+ scenarios; every production failure becomes a regression case; block a release if task completion drops more than 3 points or latency rises more than 10%.[27]
Emotion: frustration and confusion caught in real time on labeled calls, checked across accents and age groups; sentiment scored on every call after it ends.
Persona: each persona passes the same golden set; tone may differ, facts and policies may not.

T4 · Failure modes

Agents mostly break on audio, identity and integration

Failure	Mechanism	Evidence	Mitigation
Background noise and chatter	TV, traffic or other people in the room get transcribed as the caller, causing inserted words, false turns and false barge-ins	Background speech pushed streaming STT error rates to 22% to 31%; isolating the main speaker cut them by up to 43%, mostly by removing inserted words[9] vendor-reported	Isolate the caller's voice before STT and VAD; avoid generic noise suppression, which can distort the caller's speech and raise errors[44]
Wrong speaker answered	A caregiver or bystander answers a verification question, or the agent follows the wrong voice	Speaker-conditioned VAD detects the target speaker frame by frame and ignores others[28][45]	Diarize, lock onto the caller's voice in the first seconds, and re-verify if the voice changes before any sensitive step
Self-reply to echo	Agent's own speech leaks back, is transcribed as the caller, and gets answered	Seen in a public WebRTC test; rates varied widely by client[46]	Strong echo cancellation; compare input against agent playback
False barge-in	Echo treated as interruption; agent stops mid-word	A turn model marked echo "complete" at p=0.954[46]	Speaker or echo attribution before turn detection
VAD threshold trade-off	Lower thresholds speed replies but admit short echo bursts	500 → 384 ms saved 116 ms, added 3 failures[46]	Tune per channel; validate on real devices
Misroute	Router picks the wrong sub-agent, or one intent of two	Best models reach only 53.4% on a multi-intent benchmark[47]	Multi-label output, thresholds, frontier fallback
Context-flipped decision	Extra benign text changes a correct routing decision	61.4% of correct decisions flipped[48]	Route on the caller's turn only, not retrieved or third-party text
Entity drift	Transcript correct, different value reaches the tool	Observed phone number mismatch[49]	Read-back confirmation; server-side validation
Wrong saved data	Fields written incorrectly	64.5% vs 92.0% all-correct by model[42]	Exact-match audits on write-backs
Dead air	10+ s silence from stalled provider or reasoning	3.3% to 5.3% of calls by model[42]	Async tools; filler speech; timeouts with handoff
Connection failures	Call never completes cleanly	Clean calls as low as 72.4%[2]	Telephony redundancy; count against reliability
Misheard identity details	Names, emails, IDs garbled; verification fails and cascades	Context-aware ASR cut major errors up to 15%[50]	Bias recognition with the record on file; spell-back

T5 · System-1 routing

Typed decision models and routers, compared

A System-1 model returns typed answers (choice, score, yes/no) with probabilities in one pass and never writes prose. Three families now compete for the routing job: hosted decision APIs, open decision models, and plain fine-tuned classifiers.

Option	Type	Output	Latency	Cost	Evidence
OpenAI Decisions API	Closed, hosted; beta since Oct 6, 2026	Predicate, choice, score + probabilities	~10x faster than Responses vendor-reported	$0.10/M in; output free	Text and images; HIPAA for eligible customers with a BAA[22]
TypeSafe Jev	Closed, hosted	Choice, Score, Noul (a 0 to 1 "is this true?" answer) + probabilities	p50 175 ms; p95 270 ms	$0.042/M in; ~$0.11 per 1k calls	Banking77 81.0% vs frontier 84.4%; no public BAA[8][19]
Laya (Convai)	Open, Apache-2.0; 421M and 322M multilingual	Same typed interface	33 to 40 ms (T4); CPU OK	Self-host	Routed base checkpoint 42.5% on Banking77; 76.6% on its own typed-decisions benchmark after fine-tuning on that benchmark's training split[20]
Arch-Router-1.5B (Katanemo)	Open weights	Route name	51 ms (L40S)	Self-host	93.2% routing accuracy vs 92.8% for a frontier model[51]
Fine-tuned ModernBERT-base	Open, 0.1B	Intent label	22.5 ms (T4)	Self-host	Banking77 93.5%[52]
semantic-router (Aurelio)	Open, MIT	Route or none	One embedding call	Embedding cost	Returns "none" below threshold; good for out-of-scope[53]
Sierra Curie	Closed, inside the Sierra platform	Orchestration across Sierra's model "constellation"	Not published	In platform pricing	Announced Oct 2026; Sierra also runs Fleming to detect bot and agent callers[14] vendor-reported
RouteLLM	Open	Strong vs weak model	n/a	n/a	Cuts cost "over 2x" without quality loss (model routing, not intent)[54]

Banking77 intent accuracy (77 banking intents)

ModernBERT 0.1B, fine-tuned

93.5

bge-small + logistic reg.

93.3

Frontier LLM (Opus 5)

84.4

Jev, zero-shot

81.0

Laya base, routed

42.5

Different labs and setups; directional. Jev's own score ranges from 76.3% to 87.0% depending on who measured it.[52][55][8][20]

When a classifier wins, and when an LLM wins

Fine-tuned RoBERTa beat a small LLM on ATIS (95.9% vs 84.1%) and tied on CLINC150, at roughly three orders of magnitude lower cost and latency.[56]
The LLM won on out-of-scope recall (85.6% vs 58.1%) and on very noisy speech (92.5% vs 80.0%).[56]
A classifier trained on one app's intents scored 0% on a new app's; a schema-prompted model served both at about 94%.[56]
Implication: classifier for stable, high-volume intents; decision API or LLM for new, rare or out-of-scope ones.

Time to route one decision, against an 800 ms turn budget

Static chart. Measured by different labs on different hardware; directional. Sierra does not publish standalone latency for its orchestration model. Frontier figure is a large model classifying Banking77 intents.[8][52][20][51][57]

What a cascade buys

In one public test on 3,080 banking questions across 77 intents, a frontier model scored 84.4% at about $2.42 per 1,000 requests. Sending everything to a decision model first, and only the low-confidence cases (below 0.90) on to the frontier model, scored 84.0% at about $0.69 per 1,000.[8]

Frontier only

$2.42

Decision model first

$0.69

vendor-reported The cascade was scored on the same 3,080 examples used to measure accuracy, so treat the 72% saving as an upper bound.

The thinker-responder thesis, checked

holdsRouting a call to a sub-agent or toolset is closed-set classification. Small decision models do it in roughly 20 to 250 ms for fractions of a cent, versus about 2 s for a frontier LLM.[8]

holdsNarrowing tools first helps. Retrieving the right tools before selection lifted selection accuracy from 13.6% to 43.1% and cut prompt tokens by more than half in one study; Anthropic reports similar gains once agents have 10 or more tools.[58][59]

qualifyFor a fixed intent list, a plain fine-tuned classifier is often both the fastest and the most accurate router. Hosted decision APIs earn their place when intents change often or labeled data is scarce.[52][56]

qualifyDecision models are not uniformly "great at decisions." Published accuracy for the same model on the same benchmark varies by more than 10 points across labs, and benign-looking context flipped about 61% of correct decisions in one study.[55][48]

guardrailFraud, clinical concern, complaints and vulnerable-customer signals escalate by rule, never by a threshold. Multi-intent calls ("overcharged again and nobody calls back") need multi-label output.[47]

Routing to tools and MCP servers

MCP-Zero selected from about 2,800 tools across 308 servers with 98% fewer tokens on one benchmark.[60]
Anthropic reports 58 tools across 5 servers consume about 55K tokens before a conversation starts; tool search lifted accuracy from 49% to 74% on one model.[59]
With 4,027 tools, the right tool reached the top results only 56% to 64% of the time, so retrieval recall becomes the new bottleneck.[61]

Recommended router design

Tier 0: fine-tuned encoder or self-hosted decision model, multi-label (per-intent sigmoid), so "billing and complaint" returns both. In bilingual markets, start from a multilingual encoder such as XLM-R and test turns that mix languages; English-only encoders misroute them.
Stop control: when the thinker flags risk while the agent is already speaking, it can flush and mute the audio output immediately, then hand off.
Calibrate with temperature scaling on your own data; set per-intent thresholds on a held-out set, never the test set.
Gate: below threshold, out of scope, or two high-stakes intents goes to a frontier model with the full schema, or to a person.
Hard rules: fraud, clinical concern, complaints and vulnerable-customer signals always escalate.
Narrow tools: each sub-agent exposes only its own MCP servers, with a broader-tools fallback when retrieval scores are low.
Injection hygiene: classify the caller's turn only, without retrieved or third-party text in context.[48]
Who owns state: one call-state object per call, held by the orchestrator. The thinker runs on every finished turn and writes typed fields (intent, risk, speaker, emotion, confidence). The responder owns the conversation, slot filling and clarifying questions ("does Dr. Smith work Fridays?"), and reads the thinker's fields before it speaks. A new intent, topic shift or correction triggers a re-route; slots already filled carry over.
Audit: log input hash, label, probabilities, threshold, model version and final disposition; re-run evals on every model change.
Compliance: hosted decision APIs need a BAA or equivalent for protected data; self-hosted encoders avoid the question.

T6 · Limited Memory Language Models

How LMLM works, and what it has shown

Zhao et al., "Pre-training Limited Memory Language Models with Internal and External Knowledge," ICLR 2026. Code and 176M and 382M checkpoints are public under MIT.[11][35]

Mechanism

Training text is annotated with inline lookups: entity, relation, then the retrieved value.
Loss is zeroed on the retrieved value tokens. The model still trains on every other token, so it learns when and how to look up, not the value itself.
The store holds 54.6M (entity, relation, value) triples extracted from Wikipedia by an 8B annotator trained on GPT-4o-labeled seeds.
At inference, a lookup token pauses generation; the query is matched by embedding similarity (rejection threshold 0.6) and the value is inserted.
Budget: about 3B tokens, 8 epochs, at most 8 H100-days per model.

Follow-up: Co-LMLM (July 2026)

Replaces relational triples with vector keys and free-text values, so knowledge is no longer limited to entity attributes.
About 2.2 ms per retrieval versus 28.6 ms for a question-generating lookup, which fits inside a voice turn.
Reports that lookup timing is imperfect: forcing lookups added 5.9 points on SimpleQA.[17]

Model	Params	FactScore	T-REx EM	PopQA
LLaMA2 standard	382M	14.0	52.0	22.7
LLaMA2 LMLM	382M	31.9	58.1	50.8
LLaMA2 LMLM, store off	382M	12.8	38.5	n/a
LLaMA2	7B	34.0	60.5	29.2
LLaMA3.1	8B	40.3	67.3	29.4

Source: Zhao et al.[11] A 355M RAG baseline scored higher on T-REx than the same-size LMLM (75.8 vs 58.7) but lower on FactScore and PopQA, so retrieval-augmented frontier models remain the practical benchmark to beat.

How much of a model is fact storage?

LMLM 382M ≈ LLaMA2-7B

18x

RETRO 7.5B ≈ GPT-3 175B

25x

Atlas 11B > 540B on NQ

50x

Parameter reduction at comparable performance on knowledge-heavy tasks.[11][62][63] Models store about 2 bits of knowledge per parameter, falling toward 1 bit for rarely seen facts.[64] Question-answering accuracy tracks how often a fact appears in training data.[65] None of these measure the share of a frontier model's weights spent on facts; reasoning, code and multilingual skill need parameters too.

Claim often made	Verdict	Defensible version
Every fact it shares is correct; it cannot hallucinate	overstated	It cuts hallucination sharply and makes each fact traceable; errors from wrong lookups, missing rows and extraction remain.
It keeps non-repeating data out of the model	partly	It offloads factual attributes by type, common and rare alike; rare facts are where models learn worst.
Fact storage is ~99% of model bloat	unsupported	Retrieval models match models 18 to 50 times larger on knowledge tasks; the exact share is unmeasured.
It makes training cheap	partly	It still trains on the full corpus and adds lookup and store-building cost; the saving is fewer parameters for a given factual accuracy, which lowers serving cost.
It keeps sensitive data out of weights	plausible, untested	Designed to, and extraction attacks show why it matters, but no privacy-attack evaluation has been published.[33][34]

What has to be true for regulated contact centers

Extraction quality: turning CRM, EHR and policy text into rows relies on LLM annotators; their errors become confident wrong answers.
Beyond entity facts: contact center knowledge is often procedural ("if the plan is X and the claim is Y"). Free-text values help, unproven here.
Reliable lookups: the model must know when to look up and must not fall back on weights.
Live data still needs tools: balances and appointments change by the minute; LMLM covers static knowledge, not transactions.
Total cost versus RAG: pre-training a private small model plus building and refreshing the store must beat retrieval on a frontier API.
Evidence at scale: nothing yet on dialog, speech, models above about 400M, or enterprise data. A 2026 position paper argues for this direction but reports no deployments.[66]

T7 · Deployment and data posture

"Sovereign" in practice means private endpoints under contract

The case for keeping protected data and conversation logs under your control is strong. The case for running voice inference on your own GPUs is weak for 2026 to 2027, except for small models such as routers.

Posture	Status in 2026	Pros	Cons
Model API under a BAA or equivalent, modified or zero retention	OpenAI covers the Realtime API with modified retention[67]; Bedrock stores no prompts by default[68]; Anthropic covers its HIPAA-ready API[69]	Fastest; frontier quality; vendor carries ops	Terms can change; contracts needed for every component
Dedicated cloud tenancy over private network	Stanford runs Secure GPT this way on Azure[70]	No public internet path; enterprise audit	More integration work; not every voice model offered
Managed enterprise agent platform	Sierra runs the full voice stack (Tandem Voice, Curie, Fleming) and prices on outcomes[14]	Fastest to production; vendor owns tuning and safety layers	Least control over models, routing and logs; negotiate export rights
Self-hosted small models (routers, classifiers)	Encoders run in tens of ms on one GPU or CPU[52][20]	Cheap; no data leaves; no contract question	You own training, calibration and monitoring
Self-hosted large open-weight models	No named US health system found doing this for patient-facing voice	Full control	Beat the cheapest API in only 24% of 900 modeled scenarios[71]

Open versus closed models

Enterprise surveys put open-source models near 11% of LLM API share.[72][73] Open models win on cheap, high-volume work, which is exactly the router's job.

T8 · Fraud and identity controls

Design identity in from day one

Risk	Control	Runs in	Evidence
Cloned customer voice requesting protected data	Liveness detection plus passive voice, device and behavior signals	Contact center platform	Embedded in NICE CXone and Zoom Contact Center[74][75]
Knowledge-based answers stolen in breaches	Step-up verification to the number on file or app push	Agent workflow, deterministic	In one payer case, knowledge-based checks missed a fraud pattern across 1,200 targeted accounts[4] vendor-reported
Misheard identity details, wrong-account match	Recognition biased with the record on file; spell-back	STT layer	Up to 15% fewer major transcription errors[50] vendor-reported
Fraud intent buried in a routine call	Router hard rule: any fraud signal escalates regardless of confidence	System-1 router	author's view
Critical transactions and authorizations	Voiceprint match plus liveness and deepfake detection, then a one-time code for anything irreversible	Contact center platform + agent workflow	A reporter passed a bank's voice ID with an AI clone of his own voice in 2023[31]; liveness is now offered alongside voiceprints[4]
Card numbers spoken on the call	Pause recording and transcription, or take the card by keypad with the tones masked, so the agent never hears the number	Telephony and payment provider	PCI guidance on telephone payments[76]
Personal data in transcripts and logs	Redact names, account and ID numbers from the transcript stream before storage; keep raw audio only where retention rules allow	Speech-to-text and logging layer	author's view
Bots or AI agents calling in at scale	Detect machine callers before the agent engages	Agent platform	Sierra Fleming, announced Oct 2026[14] vendor-reported
Outbound AI calls without consent (US)	Prior express consent, identification, opt-out	Dialer policy	FCC 24-17, February 2024[32]
Scammers posing as your organization	Published callback and "we never ask" policy; customer education	Marketing and communications	Family-impersonation clone scams documented[30]

Risk tier	Examples	Checks	Friction to plan for
Low	Hours, directions, appointment or card status	Passive voice, device and behavior signals; caller ID match	None for the caller
Medium	Address change, refill request, payment date	Passive signals plus a one-time code to the number on file, or an app push	Some older or distressed callers drop; offer a person
High	Payments, prescription changes, authorizations	Voiceprint plus liveness and deepfake detection, then a one-time code	False rejects on phone-quality audio; a fast route to a person is required

author's view Tiers and examples are a starting point for fraud and compliance teams. Texted codes carry SIM-swap risk, so prefer an app push where customers have the app.

T9 · Regulation

Mostly disclosure and consent, and tightening

A summary of selected US rules for orientation, not legal advice and not a complete list. US healthcare focus; financial services adds its own regimes (GLBA, PCI DSS, UDAAP) not reviewed here. Items marked unverified come from secondary sources and need confirmation against the enacted text and by counsel.

Rule	What it requires	Status
TCPA, FCC 24-17	AI voices count as "artificial"; outbound calls generally need prior express consent, with narrow healthcare exemptions; consent belongs to the current subscriber, so check the Reassigned Numbers Database[12]; $500 to $1,500 per violation	In force since Feb 8, 2024[32][77]
Call recording consent (state wiretap laws)	Some states, including California, Florida, Maryland and Pennsylvania, require every party's consent to record; announce recording and transcription at the start of the call	In force; varies by state; check with counsel[78]
HIPAA BAAs	Every layer that touches PHI: model, router, STT, TTS, orchestration, telephony, testing	Available from major model providers with retention options[67][69]
California AB 3030	Disclaimer and a route to a human on generative AI clinical messages, unless clinician-reviewed	Effective Jan 1, 2025[29]
California AB 489	AI may not imply it holds a health care license	Effective Jan 1, 2026[29]
California AB 1609	Large businesses disclose chatbots and offer a human within set times	Reported signed Sept 28, 2026 unverified[79]
Texas TRAIGA (HB 149)	Health care providers disclose AI use to patients	Effective Jan 1, 2026[80]
Utah AI Policy Act	Proactive disclosure in high-risk interactions by regulated occupations	In force[79]
Colorado SB 26-189	Replaces the 2024 AI Act, which never took effect; covers automated decisions in consequential areas such as credit, employment and housing, with no general chatbot disclosure duty	Reported effective Jan 1, 2027 unverified[79]
Right to deletion (state privacy laws, GDPR)	Erase personal data on request	Easier to honor in the model when facts live in a store, not weights[11]; logs, transcripts and backups still need their own deletion

T10 · Claim check

Claims circulating about voice AI, checked

Benchmark figures, vendor claims and one widely shared decision-offload cost model, each traced to a source. Filter by verdict.

Interactive · filter by verdict

Claim	Verdict	What the evidence shows
Cekura Bench: 7 configurations, 82 scenarios, 3 repeats, 246 calls each	partial or vendor	Design confirmed; the live hub now lists 8 platforms[13]
pass^3 is the primary metric	verified	Share of scenarios where all three retained calls pass[2]
Retell, ElevenLabs, Vapi and Gemini Live figures	verified	Exact match on all four reported metrics[49]
LiveKit 70.73% and Pipecat 63.41% pass^3	verified	pass^3 confirmed; other metrics not seen[2]
GPT Realtime 64.63% pass^3	unverified or wrong	Not found; the separate speech-to-speech benchmark shows 79.3%[42]
Per-platform model stacks	unverified or wrong	Providers chose their own stacks; not independently confirmed[2]
Entity drift between transcript and tool call	verified	Documented instance with a phone number[49]
TypeSafe Jev exists, built by an ex-OpenAI researcher	verified	Released Sep 15, 2026[81]
Jev latency 70 to 150 ms	unverified or wrong	Vendor quotes 70 to 500 ms; one test measured p50 175 ms, p95 270 ms[8]
Jev $0.042 per million input tokens, free output	partial or vendor	Matches early-access vendor pricing[19]
Jev is calibrated and cannot hallucinate	partial or vendor	Output is always schema-valid; calibration weaker than claimed in independent tests[82]
Decision models are 400x cheaper than LLMs	unverified or wrong	Not in primary sources; real savings in one cascade test were about 72%[8]
OpenAI Decisions API is generally available	unverified or wrong	Public beta since Oct 6, 2026[22]
Open-source Laya is a drop-in router	unverified or wrong	Routed base checkpoint scores 42.5% on Banking77; fine-tuning required[20]
Open models hit 78.4% of AI Gateway token volume	verified	Exact post; one-day record on a developer-heavy gateway[83]
Regulated enterprises are moving inference off public APIs	partial or vendor	Private cloud endpoints common; no named US on-prem voice deployment[70]
Decision-offload simulator: about 730 ms vs 1,900 ms turns	unverified or wrong	Simulated, not measured[2]
Decision-offload simulator: $11,017 a month saved	partial or vendor	Plausible arithmetic, but about 6% of run rate with no infrastructure modeled[24]
Sierra serves 80% of Fortune 50 healthcare companies	partial or vendor	Vendor claim from Sierra Summit 2026; not independently verified[14]
Limited-memory models cannot hallucinate	unverified or wrong	Authors: does not guarantee perfect factuality[11]
Fact storage is about 99% of model size	unverified or wrong	No study measures it; retrieval models match 18x to 50x larger models on knowledge tasks[62]

T11 · Tooling and implementation

What to borrow, what to avoid

Frontend and developer tooling

Vercel's AI Voice Elements give React components for persona, mic capture, transcript and playback; useful for web prototypes, irrelevant to telephony.[84]

Why implementations fail

Abstract goals instead of concrete use cases, bots isolated from scheduling and CRM systems, no handoff path, messy knowledge, and no measurement after launch.[85] vendor-reported OpenAI's own rebuild relied on a silent production test to find failures synthetic load tests missed.[37][86]

Sources

Every number traces back here

Numbered in order of first use.

Interactive · filter by source type

[1]Becker's Hospital Review: Tampa General cuts call wait times 58% with AI rollout independent
https://www.beckershospitalreview.com/healthcare-information-technology/ai/tampa-general-cuts-call-wait-times-58-with-ai-rollout/
[2]Cekura Bench: voice agent benchmark hub (updated Oct 7, 2026) vendor
https://benchmarks.cekura.ai/
[3]Talkdesk / NewtonX: The State of Agentic Automation in CX (Aug 25, 2026) vendor
https://www.globenewswire.com/news-release/2026/08/25/3350556/0/en/companies-are-deploying-ai-in-customer-experience-faster-than-they-can-make-it-work.html
[4]Pindrop expands AI-powered deepfake detection to healthcare (Feb 26, 2026) vendor
https://finance.yahoo.com/news/pindrop-expands-ai-powered-deepfake-150000934.html
[5]Hyro: Tampa General Hospital case study vendor
https://www.hyro.ai/case-study/tampa-general-hospital/
[6]Becker's Hospital Review: 12 health systems expanding their use of agentic AI (2026) independent
https://www.beckershospitalreview.com/healthcare-information-technology/ai/12-health-systems-expanding-their-use-of-agentic-ai/
[7]Sierra: tau-voice, benchmarking real-time voice agents (May 1, 2026) vendor
https://sierra.ai/blog/tau-voice-benchmarking-real-time-voice-agents-on-real-world-tasks
[8]OpenRouter: Jev vs Claude Opus 5 on classification (Sep 22, 2026) vendor
https://openrouter.ai/blog/insights/jev-vs-claude-opus-5-classification
[9]ai-coustics: Voice Focus 2.0 deep dive, primary-speaker isolation benchmark (Mar 11, 2026) vendor
https://ai-coustics.com/blog/voice-focus-2.0-deepdive
[10]EU AI Act, Article 50: transparency obligations, including emotion recognition independent
https://artificialintelligenceact.eu/article/50/
[11]Zhao et al., Pre-training Limited Memory Language Models (arXiv 2505.15962; ICLR 2026) independent
https://arxiv.org/abs/2505.15962
[12]FCC: Reassigned Numbers Database independent
https://www.fcc.gov/reassigned-numbers-database
[13]Cekura: Booking and reservation flow testing for voice agents vendor
https://www.cekura.ai/discover/booking-reservation-flow-testing-voice-agents
[14]Sierra: Summit 2026 recap (Oct 2026) vendor
https://sierra.ai/blog/summit-recap-2026
[15]TechCrunch: ElevenLabs raises $500M at $11B valuation (Feb 4, 2026) independent
https://techcrunch.com/2026/02/04/elevenlabs-raises-500m-from-sequioia-at-a-11-billion-valuation/
[16]Flowith: GPT-Live in ChatGPT Voice guide independent
https://flowith.io/blog/gpt-live-chatgpt-voice-full-duplex-guide/
[17]Feldman, Zhao et al., Co-LMLM: Continuous-Query Limited Memory Language Models (arXiv 2607.07707) independent
https://arxiv.org/abs/2607.07707
[18]RuntimeWire: DevDay 2026 live updates independent
https://runtimewire.com/article/devday-2026-live-updates
[19]TypeSafe AI: Introducing System One models and Jev (Sep 15, 2026) vendor
https://typesafe.ai/blog/introducing-system-one-models-and-jev
[20]Convai Innovations: Laya model card, Apache-2.0 (Sep 2026) vendor
https://huggingface.co/convaiinnovations/laya
[21]InfoQ: OpenAI DevDay 2026 independent
https://www.infoq.com/news/2026/10/openai-devday-2026/
[22]OpenAI: Decisions API guide and changelog (public beta Oct 6, 2026) vendor
https://developers.openai.com/api/docs/guides/decisions
[23]The Nuanced Perspective: Where voice agents actually break (Oct 9, 2026) independent
https://thenuancedperspective.substack.com/p/where-voice-agents-actually-break
[24]beri.net: Voice AI per-minute pricing and cost per resolved call (Oct 2, 2026) independent
https://www.beri.net/article/voice-ai-per-minute-pricing-vapi-retell-elevenlabs-bland-cost-per-resolved-call
[25]Helpware: Healthcare call center costs (2026) vendor
https://helpware.com/blog/healthcare-call-center
[26]OpenAI: Continuous voice interaction with GPT-Live (Aug 3, 2026) vendor
https://openai.com/index/continuous-voice-interaction-with-gpt-live/
[27]Hamming: How to evaluate voice agents, framework, metrics and checklists (Jan 2026) vendor
https://hamming.ai/blog/how-to-evaluate-voice-agents
[28]Ding et al. (Google): Personal VAD, speaker-conditioned voice activity detection (2020) independent
https://arxiv.org/abs/1908.04284
[29]Clunic: California AI health care laws independent
https://clunic.com/regulations/california-ai-healthcare-laws
[30]TechCrunch: After a deepfake voice fooled her grandfather, this founder sprang into action (Sep 28, 2026) independent
https://techcrunch.com/2026/09/28/after-a-deepfake-voice-fooled-her-grandfather-this-founder-sprang-into-action/
[31]AI Incident Database #485: reporter accesses own bank account with AI-cloned voice (2023) independent
https://incidentdatabase.ai/cite/485/
[32]FCC: AI-generated voices in robocalls are illegal without consent (Feb 8, 2024) independent
https://www.fcc.gov/document/fcc-makes-ai-generated-voices-robocalls-illegal
[33]Carlini et al., Extracting Training Data from Large Language Models (USENIX Security 2021) independent
https://arxiv.org/abs/2012.07805
[34]Nasr et al., Scalable Extraction of Training Data from Production Language Models (2023) independent
https://arxiv.org/abs/2311.17035
[35]kilian-group/LMLM code and checkpoints (GitHub, MIT) independent
https://github.com/kilian-group/LMLM
[36]IntuitionLabs: Hosted AI API vs self-hosted LLM cost (H100 rental rates) independent
https://intuitionlabs.ai/articles/hosted-ai-api-vs-self-hosted-llm-cost
[37]InfoQ: OpenAI details GPT-Live's architecture (Sep 2, 2026) independent
https://www.infoq.com/news/2026/09/openai-gpt-live/
[38]MarkTechPost: Google releases Gemini 3.8 Live (Sep 15, 2026) independent
https://www.marktechpost.com/2026/09/15/google-releases-gemini-3-8-live-and-3-8-live-extended-thinking-for-production-grade-voice-agents/
[39]Salesforce Developers: Agentforce Voice for Agent Script (Oct 1, 2026) vendor
https://developer.salesforce.com/blogs/2026/10/agentforce-voice-for-agent-script-voice-enabling-best-practices
[40]Stivers et al., Universals and cultural variation in turn-taking, PNAS 2009 independent
https://pubmed.ncbi.nlm.nih.gov/19553212/
[41]Regal: Best LLMs for voice AI agents, Voice CX Benchmark (Oct 6, 2026) vendor
https://www.regal.ai/listicles/best-llms-for-voice-ai-agents
[42]Cekura: Speech-to-speech benchmark vendor
https://benchmarks.cekura.ai/speech-to-speech
[43]IHBench: interruption handling benchmark (arXiv 2606.19595) independent
https://arxiv.org/pdf/2606.19595
[44]Sanas: ASR-optimized noise cancellation for agentic AI (Dec 12, 2025) vendor
https://www.sanas.ai/science/inside-sanas-asr-optimized-noise-cancellation-for-agentic-ai
[45]Online target-speaker voice activity detection for speaker diarization (arXiv 2207.05920) independent
https://arxiv.org/abs/2207.05920
[46]webrtcHacks: Voice agents get confused by echo too (Oct 8, 2026) independent
https://webrtchacks.com/voice-agents-get-confused-by-echo-too/
[47]Multi-intent spoken language understanding with LLMs (arXiv 2403.04481) independent
https://arxiv.org/abs/2403.04481
[48]JevOut: Natural context can flip decision models (arXiv 2609.30243) independent
https://arxiv.org/abs/2609.30243
[49]Retell AI: Retell vs Bland vs Vapi vs ElevenLabs (cites Cekura Bench) vendor
https://www.retellai.com/blog/retell-vs-bland-vs-vapi-vs-elevenlabs
[50]Sierra: Voice AI is only as good as what it hears (May 18, 2026) vendor
https://sierra.ai/blog/voice-ai-is-only-as-good-as-what-it-hears
[51]Katanemo: Arch-Router, preference-aligned routing (arXiv 2506.16655) vendor
https://arxiv.org/abs/2506.16655
[52]sift-banking77-modernbert model card (Hugging Face) independent
https://huggingface.co/DarioFrK/sift-banking77-modernbert
[53]Aurelio Labs: semantic-router (GitHub, MIT) vendor
https://github.com/aurelio-labs/semantic-router
[54]LMSYS: RouteLLM, learning to route LLMs with preference data (arXiv 2406.18665) independent
https://arxiv.org/abs/2406.18665
[55]CreativeAINews: Open Jev clones and benchmark disagreement (Sep 2026) independent
https://creativeainews.com/articles/open-jev-clones-benchmark-disagreement-2026
[56]Rodrigues and Vas: classifiers vs LLMs for intent detection (arXiv 2608.20371) independent
https://arxiv.org/abs/2608.20371
[57]Hamming: Voice AI latency, what's fast and what's slow (Jan 12, 2026) vendor
https://hamming.ai/blog/voice-ai-latency-whats-fast-whats-slow-how-to-fix-it
[58]Gan and Sun: RAG-MCP, mitigating prompt bloat in tool selection (arXiv 2505.03275) independent
https://arxiv.org/abs/2505.03275
[59]Anthropic Engineering: Advanced tool use (Nov 24, 2025) vendor
https://www.anthropic.com/engineering/advanced-tool-use
[60]Fei et al., MCP-Zero: proactive toolchain construction (arXiv 2506.01056) independent
https://arxiv.org/abs/2506.01056
[61]Arcade: Testing tool search with 4,000 tools (Dec 3, 2025) vendor
https://arcade.dev/blog/anthropic-tool-search-4000-tools-test
[62]Borgeaud et al. (DeepMind), RETRO: Improving language models by retrieving from trillions of tokens independent
https://arxiv.org/abs/2112.04426
[63]Izacard et al. (Meta), Atlas: Few-shot learning with retrieval augmented language models independent
https://arxiv.org/abs/2208.03299
[64]Allen-Zhu and Li, Physics of Language Models 3.3: Knowledge Capacity Scaling Laws (ICLR 2025) independent
https://arxiv.org/abs/2404.05405
[65]Kandpal et al., Large Language Models Struggle to Learn Long-Tail Knowledge (ICML 2023) independent
https://arxiv.org/abs/2211.08411
[66]Singh, Bastos, Mulang: Position, avoid overstretching LLMs for every enterprise task (arXiv 2605.09365) independent
https://arxiv.org/abs/2605.09365
[67]OpenAI Help Center: HIPAA-eligible products and functionality (Oct 2026) vendor
https://help.openai.com/en/articles/20001069-hipaa-eligible-products-and-functionality
[68]AWS: Amazon Nova 2 Sonic service card vendor
https://docs.aws.amazon.com/ai/responsible-ai/nova-2-sonic/overview.html
[69]Anthropic: Covered models under a BAA vendor
https://support.claude.com/en/articles/15455031-covered-models-under-a-business-associate-agreement-baa
[70]PMC: Stanford Secure GPT deployment (2025) independent
https://pmc.ncbi.nlm.nih.gov/articles/PMC11833461/
[71]navyaai: Self-host LLM vs API cost (Sep 2026) independent
https://www.navyaai.com/guides/self-host-llm-vs-api-cost
[72]OfficeChai: Share of closed models has fallen on Vercel data (Sep 2026) independent
https://officechai.com/ai/share-of-closed-models-has-fallen-from-around-70-to-21-in-the-last-3-months-vercel-data/
[73]Menlo Ventures: 2025, the state of generative AI in the enterprise vendor
https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/
[74]CX Today: Pindrop and NICE CXone integration independent
https://www.cxtoday.com/contact-center/pindrop-nice-integration-deepfake-detection-cxone/
[75]Biometric Update: Zoom expands Pindrop deepfake detection (Mar 2026) independent
https://www.biometricupdate.com/202603/zoom-expands-pindrop-deepfake-detection-to-customer-service
[76]PCI SSC: Protecting telephone-based payment card data (information supplement, Nov 2018) independent
https://listings.pcisecuritystandards.org/documents/Protecting_Telephone_Based_Payment_Card_Data_v3-0_nov_2018.pdf
[77]Wiley Rein: FCC TCPA restrictions cover AI-generated voices independent
https://www.wiley.law/alert-FCC-Extends-Regulatory-Reach-Over-AI-Announces-TCPA-Restrictions-Cover-AI-Generated-Voices-in-Outbound-Calls
[78]Justia: Recording phone calls and conversations, 50-state survey independent
https://www.justia.com/50-state-surveys/recording-phone-calls-and-conversations/
[79]Conferbot: AI chatbot disclosure laws guide vendor
https://www.conferbot.com/blog/ai-chatbot-disclosure-laws-compliance-guide
[80]Wisconsin Legislative Council: US state AI laws overview (Aug 13, 2026) independent
https://docs.legis.wisconsin.gov/misc/lc/study/2026/3042/090_august_13_2026_meeting_10_00_a_m_room_411_south_state_capitol/003_us_state_ai_laws_an_overview_final
[81]InfoQ: TypeSafe AI releases Jev (Oct 1, 2026) independent
https://www.infoq.com/news/2026/10/typesafe-ai-jev-released/
[82]beri.net: Jev calibration, decomposition and shadow eval independent
https://www.beri.net/article/typesafe-jev-typed-decision-model-calibration-decomposition-shadow-eval
[83]Guillermo Rauch on X: open models share of AI Gateway tokens (Sep 2026) vendor
https://x.com/rauchg/status/2101186741042663579
[84]Vercel changelog: AI Voice Elements (Jan 14, 2026) vendor
https://vercel.com/changelog/ai-voice-elements
[85]Netguru: Why most chatbot implementations fail (Jul 2026) vendor
https://www.netguru.com/blog/why-most-chatbot-implementations-fail
[86]ByteByteGo: How OpenAI delivers low-latency voice AI (Jul 1, 2026) independent
https://blog.bytebytego.com/p/how-openai-delivers-low-latency-voice

Discussion

Questions and comments

Corrections, counterpoints and field experience are welcome. Comments are public and stored as GitHub issues on this report's repository through utterances, which is free and open source, with no ads or tracking. Sign in with GitHub to post.

Comments appear here when the page runs scripts. You can also read and post them on GitHub.

Read comments on GitHub Post on GitHub

About

About this report

The report

A vendor-neutral view of conversational voice agents for leaders in regulated contact centers, published as a monthly edition. Every figure links to its source, and vendor-reported numbers are labeled as such.

Corrections and source suggestions are welcome as issues on GitHub, which also keeps past editions.

The author

Madhav Srinivasan

Engineering leader who builds AI agents and platforms.

Written in a personal capacity; the views are my own, not any employer's. No commercial relationship with any vendor named, and no sponsorship.

Back to sbmadhav.com Connect on LinkedIn