Voice AI Agent Platform Evaluations
The G2 voice AI agent platform evaluation measures how well each platform resolves a caller's request end to end. Bluejay deploys the same agent on every platform and places the same 90 simulated calls, five times each, across healthcare, legal and customer support; every call is graded on the tool calls made and the database state left behind.
01Leaderboard
Read moreRead less
A high score means a platform can host a tool-using agent that hears callers correctly, answers without lag and leaves the records it touches in the right state. Hover a measurement to isolate its share of each score; click a column to sort.
81.4 | 85.6% | 4.7% | 37.1% | 2.15 s | 4.47 s | |
80.9 | 81.1% | 3.7% | 38.4% | 1.15 s | 3.32 s | |
80.4 | 89.1% | 5.7% | 32.9% | 3.30 s | 6.80 s | |
77.0 | 86.0% | 11.4% | 40.2% | 3.55 s | 6.50 s | |
65.0 | 63.8% | 6.9% | 48.0% | 1.90 s | 3.20 s | |
47.1 | 43.3% | 4.8% | 37.9% | 2.25 s | 5.01 s |
The front desk of a multi-office dermatology practice. Callers book and move visits, check insurance, ask about cosmetic treatments and pay bills.
88.7% | 133 / 150 | 77.0 | |
87.3% | 131 / 150 | 80.4 | |
86.7% | 130 / 150 | 81.4 | |
79.3% | 119 / 150 | 80.9 | |
66.7% | 100 / 150 | 65.0 | |
40.7% | 61 / 150 | 47.1 |
Intake at a personal-injury law firm. Callers want to know if the firm can take their case, give their story, book an evaluation, or check on an existing matter.
82.7% | 124 / 150 | 80.4 | |
80.0% | 120 / 150 | 81.4 | |
78.7% | 118 / 150 | 80.9 | |
71.3% | 107 / 150 | 77.0 | |
56.7% | 85 / 150 | 65.0 | |
34.0% | 51 / 150 | 47.1 |
Phone support for a national electronics retailer. Callers track deliveries, return products, book repairs, manage memberships and ask for price matches.
98.0% | 147 / 150 | 77.0 | |
97.3% | 146 / 150 | 80.4 | |
90.0% | 135 / 150 | 81.4 | |
85.3% | 128 / 150 | 80.9 | |
68.0% | 102 / 150 | 65.0 | |
55.3% | 83 / 150 | 47.1 |
02Pareto frontiers
A platform is on the Pareto frontier when no other platform is both faster (or more accurate) and more successful at tasks. The stepped line traces the frontier and the shaded area beneath it is the dominated region. Quadrants split each chart at the field medians instead. Hover a platform for its values; click it for the platform profile.
ElevenLabs Agents, Retell and LiveKit Agents make up the Pareto frontier for task success and latency: no other platform is both faster and more successful at tasks than any of them.
Retell is both faster than the field median and above the median task success.
ElevenLabs Agents, Retell, LiveKit Agents and Pipecat make up the Pareto frontier for task success and latency: no other platform is both faster and more successful at tasks than any of them.
Retell is both faster than the field median and above the median task success.
ElevenLabs Agents, Retell and LiveKit Agents make up the Pareto frontier for task success and latency: no other platform is both faster and more successful at tasks than any of them.
Retell and ElevenLabs Agents are both faster than the field median and above the median task success.
ElevenLabs Agents, Retell, LiveKit Agents and Pipecat make up the Pareto frontier for task success and latency: no other platform is both faster and more successful at tasks than any of them.
Retell is both faster than the field median and above the median task success.
ElevenLabs Agents, Retell and LiveKit Agents make up the Pareto frontier for task success and word error rate: no other platform is both more accurate and more successful at tasks than any of them.
Retell is both more accurate than the field median and above the median task success.
ElevenLabs Agents, Retell, LiveKit Agents and Pipecat make up the Pareto frontier for task success and word error rate: no other platform is both more accurate and more successful at tasks than any of them.
Retell is both more accurate than the field median and above the median task success.
ElevenLabs Agents, Retell and LiveKit Agents make up the Pareto frontier for task success and word error rate: no other platform is both more accurate and more successful at tasks than any of them.
Retell and ElevenLabs Agents are both more accurate than the field median and above the median task success.
ElevenLabs Agents, Retell, LiveKit Agents and Pipecat make up the Pareto frontier for task success and word error rate: no other platform is both more accurate and more successful at tasks than any of them.
Retell is both more accurate than the field median and above the median task success.
03Reliability
Each task runs five times. pass@1 is the share of single calls that passed: what a demo shows. pass^5 is the share of tasks that passed on all five runs: what a caller can count on in production. A platform that wins demos but loses tasks on a re-run has a wide gap between the two.
LiveKit Agents has the widest gap between demo and production: 89.1% of its single calls pass, but only 47.2% of tasks pass on all five runs, 41.9 points lower. The number under each platform is that gap.