Bland
Voice agent platform that runs its own stack end to end, reached over the WebSocket in its official SDK.
Score breakdown
Task success is worth up to 80 points, word error rate and latency up to 10 each.
Testing implementation
One of the calls Bland took, end to end: the simulated caller, the agent as deployed on Bland, the tool server it called and the grade. Callers, tools and grading are identical on every platform; only the platform and its model stack change.
Caller
T4-H1 · Cancel Plus after one save offer
“Cancel my Kestrel Plus membership today.”
- full name
- Selina Cortez
- phone
- 541-555-0171
- postal code
- 97035
- card last4
- 1156
Bland agent
Customer support front desk
- STT
- Configured by the provider
- LLM
- Configured by the provider
- TTS
- Configured by the provider
- Transport
- WebSocket
- Prompt and tools
- Shared customer support spec, 25 tools
Tool server
Bluejay customer support systems
- Tools
- 25 HTTP tools
- State
- Seeded database per call
- Log
- Every request, keyed to the call
- Same build
- Every platform
Grade
Pass
- ✓
identify_customerfull_name=Selina Cortez, phone=541-555-0171 - ✓
verify_identitypostal_code=97035, card_last4=1156 - ✓
get_customer_summary - ✓
quote_membership_cancellation - ✓
confirm_membership_cancellationproration_acknowledged=True
Tool-call diffempty
Database-state diffmatch
Task success by industry
Share of calls where the agent completed the caller's request. Click an industry to see what its callers ask for.
Word error rate
How well the platform heard the caller: the share of the caller's words its transcript got wrong. Lower is better. Variants replay a task with background noise or a degraded line.
Latency
How fast the agent answered: time from the caller's last word to the agent's first audio, over every turn. Full marks inside the shaded band; no points in the hatched end.
Findings
Setup review
The authors' assessment of building, deploying and running the agent on Bland: interface, developer experience, customization, going live, MCP, documentation, pricing and languages. Bluejay's voice AI engineers score it during setup from documented capabilities and hands-on use. Expand a category for what we check and how each item is scored.
UIHow well someone can build, test and review an agent in the console without writing code.6.8 / 10
How it is scored. Six weighted line items, each graded full, half or none: time to first agent, build without code, test in the browser, call review, errors you can see, team workspace.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Time to first agent | 2.5 | half* 1.25 | not checked |
| Build without code | 2.0 | full* 2 | New Agent builder in dashboard (DECISION.md) |
| Test in the browser | 1.5 | full* 1.5 | "Talk in browser" dashboard test, no public API for it (DECISION.md) |
| Call review | 1.5 | half 0.75 | Per-call timeline exists (v1/pathway_calls?v=2) but /v1/calls list has end_at null (DECISION.md) |
| Errors you can see | 1.5 | half 0.75 | Fire-and-forget warning only in the call timeline; untagged tools fail silently per its own docs (DECISION.md) |
| Team workspace | 1.0 | half* 0.5 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
DevXWhether a developer can build, run and read back the whole agent from code.4.2 / 10
How it is scored. Six weighted line items: agent-as-code, call control, data out, API correctness, SDK quality, scoped keys and auth.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Agent-as-code | 2.5 | half 1.25 | v2 Agents API works (versions, promote) but legacy/v2 split; prompt read back as prefix (ts_date, DECISION.md) |
| Call control | 1.5 | half 0.75 | New Agents reachable only by phone; WebSocket session API refuses v2 Agents (DECISION.md) |
| Data out | 2.5 | half 1.25 | /v1/calls list end_at null; slow call-id matching by caller number + start time (DECISION.md) |
| API correctness | 1.5 | none 0 | Custom tools fire-and-forget by default; responses ignored until response_data set; free-text reason codes (DECISION.md) |
| SDK quality | 1.0 | half 0.5 | Published @blandsdk/client differs from its docs (DECISION.md) |
| Scoped keys and auth | 1.0 | half* 0.5 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
Platform customizabilityHow far you can shape the agent: its models, conversation behaviour, prompt and tools.3.8 / 10
How it is scored. Seven weighted line items, including interruption handling measured on the platform's defaults during end-to-end validation.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Model choice | 1.5 | half* 0.75 | LLM is Bland built-in |
| Conversation behaviour | 1.0 | half 0.5 | Injects wall-clock now over the prompt's date (P19); end_call can't speak a goodbye (DECISION.md, ts_date) |
| Interruption handling | 3.0 | half* 1.5 | Not measured |
| Prompt and context | 1.0 | half 0.5 | Prompt fits but Bland overrides the date (ts_date P19) |
| Custom tools | 2.5 | none 0 | Drops ids from prior tool replies, fills fields from call variables, fire-and-forget default, tool loops capped at 3 (DECISION.md) |
| Speech controls | 0.5 | half* 0.25 | not checked |
| Agent handoff | 0.5 | half* 0.25 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
Go liveEverything needed to take the agent to production and keep it running.6.8 / 10
How it is scored. Seven weighted line items, including runtime under load: the break rate at 20 concurrent calls against the 2-call baseline.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Telephony and numbers | 2.0 | full 2 | Bought and attached Bland numbers; phone is its standard channel (DECISION.md) |
| Release management | 1.5 | full 1.5 | Versions saved, published, promoted, pinned (DECISION.md, ts_date) |
| Runtime under load | 2.5 | half 1.25 | Concurrency had to go 2->4->10 (concurrency_change.jsonl) |
| Integrations at the edge | 1.0 | half* 0.5 | not checked |
| Ops and incident response | 1.5 | half* 0.75 | not checked |
| Compliance and trust | 1.0 | half* 0.5 | not checked |
| Support and escalation | 0.5 | half* 0.25 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
MCPWhether the agent can use tools over MCP, and whether coding agents can build and run the platform through MCP.6.5 / 10
How it is scored. Four weighted line items: MCP tools on a live call, connection controls, a platform MCP server, and how much of the API that server covers.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| MCP tools on a live call | 3.5 | half* 1.75 | not checked |
| MCP connection controls | 1.5 | half* 0.75 | not checked |
| Platform MCP server | 3.0 | full* 3 | Official MCP server with OAuth + tool reference (docs.bland.ai/integrations/mcp) |
| What that server can do | 2.0 | half* 1 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
DocsWhether the docs are accurate, complete and usable by people and by coding agents.5.2 / 10
How it is scored. Eight weighted line items, from reference accuracy and payload schemas to change management and machine-readability.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Reference accuracy | 2.0 | none 0 | Web-agent SDK docs differ from published SDK; legacy vs v2 unclear (DECISION.md) |
| Payload and event schemas | 1.5 | half 0.75 | Fire-and-forget response handling found in timeline, not in tool docs (DECISION.md) |
| Concepts and lifecycle | 1.0 | half 0.5 | Migration guide clear on Agents vs pathways (DECISION.md) |
| Telephony and go-live guides | 1.5 | full 1.5 | Attach numbers to an agent documented end to end (DECISION.md) |
| Errors, limits and gotchas | 1.0 | half 0.5 | Documents untagged tools failing silently (DECISION.md) |
| Examples and quickstarts | 1.0 | half* 0.5 | not checked |
| Change management | 1.0 | half 0.5 | Dated migration deadline (15 Nov 2026) published (DECISION.md) |
| Machine-readability | 1.0 | full 1 | llms.txt used during setup (DECISION.md) |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
Pricing clarityWhether a buyer can predict what the platform will cost and see what it is costing.3.5 / 10
How it is scored. Six weighted line items, including cost per hour: our estimate from public pricing against the real bill for the scored calls.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Public price | 2.0 | half* 1 | Only third-party price lists found (~$0.11-0.14/min + plan fee); official page not fetched |
| Every line item named | 1.5 | none* 0 | No official itemized lines found |
| Cost per hour matches the estimate | 3.0 | half* 1.5 | Bill not reconciled |
| Usage and spend in the product | 1.5 | half* 0.75 | not checked |
| Spend controls | 1.5 | none* 0 | Not found |
| Free start | 0.5 | half* 0.25 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
Language supportWhether the agent can take a call in Spanish, French, Mandarin Chinese, Hindi or Arabic, understand it and reply in that language.7 / 10
How it is scored. One translated probe task per language, two runs each. Full when the agent understood and replied in the language, half when it understood but replied in English.
| What we checked | Weight | Grade | Note |
|---|---|---|---|
| Spanish | 2.0 | full* 2 | Graded by configuration; live probes not run |
| French | 2.0 | full* 2 | Graded by configuration; live probes not run |
| Mandarin Chinese | 2.0 | half* 1 | not checked |
| Hindi | 2.0 | half* 1 | not checked |
| Arabic | 2.0 | half* 1 | not checked |
Initial grades from the 28–29 Oct 2026 run and public docs; * = judged from docs, not yet verified on a call.
Run details
Model configuration
- STT
- Configured by the provider
- LLM
- Configured by the provider
- TTS
- Configured by the provider
- Transport
- WebSocket
Run
- Scored calls
- 630
- Runs per task
- 5
- Run dates
- 21–22 Oct 2026
- Concurrency held
- up to 10 calls
- Turns measured
- 3,577 turns