E-commerce evaluation · 50 assigned tasks, 47 scored. Three infrastructure errors were excluded from judging and earn no task points. These results are separate from General CX.

Yuma

4.8(19 G2 reviews)

Yuma AI automates e-commerce customer support end to end. The platform resolves tickets autonomously across chat, email, and social, handling 100+ CX use cases from order tracking to returns to complex pre-sale inquiries. It plugs directly into the helpdesks e-commerce teams already use: Zendesk, Gorgias, Kustomer, Front, Gladly, and Re:amaze. Every plan includes unlimited AI Agents and SOPs, omnichannel and multilingual support, and 24/7 premium support via Slack. Pricing is based on fully automated resolutions only. Results come fast. Customers typically reach 60-80% automation and 25-40% cost savings within 90 days. Our customers run at 91% accuracy on WISMO with 87% faster responses. EvryJewels automates 89% of tickets and cut costs by 63%. The platform has processed over 5 million automated tickets.

Overall score

190points

38/50 tasks passed

Explore task examples

G2 task evaluation

Accuracy96

Policy compliance95

G2 buyer reviews

Ease of use91

Quality of support98

How Yuma performed

(Showing 3 of 50 tasks)

Explore the conversations and grading behind Yuma’s evaluation scores.

Exchanging delivered items

Exchanges

Passed

The Scenario

You received your order #W2378156 and wish to exchange the mechanical keyboard for the same one but with clicky switches and the smart thermostat for one compatible with Google Home instead of Apple HomeKit. If there is no keyboard that is clicky, RGB backlight, full size, you'd go for no backlight.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

Hi, I received order #W2378156 and need to exchange two items from it—a mechanical keyboard and a smart thermostat.

Yuma · Tool callfind_user_id_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool result · Errorfind_user_id_by_email

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma

Hi,

I couldn’t verify a customer profile using the email on this conversation. Please reply with one corrected email address associated with your G2 Evals account so I can continue with the exchange request for order #W2378156.

G2 Evals Support

Powered by AI to ensure you’re being assisted quickly, monitored by humans in case you need additional support On Thu, Sep 03 2026, at 06:25 AM, agent-eval-dev+ec077261087d4fddb400a7966fb88eb4@g2.com wrote:

Hi, I received order #W2378156 and need to exchange two items from it—a mechanical keyboard and a smart thermostat.

Simulated customer

I don’t remember which email address I used for the order. The order is under Yusuf Rossi, and my ZIP code is 19122. Is there another way to verify it?

Yuma · Tool callfind_user_id_by_name_zip()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_user_id_by_name_zip

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma · Tool callget_user_details()

Arguments

{
  "user_id": "yusuf_rossi_9620"
}

Yuma · Tool resultget_user_details

Returned data

{
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W2378156"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W2378156",
  "status": "delivered",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma

I verified order #W2378156 under Yusuf Rossi, and it’s marked delivered. The two items are:

Mechanical Keyboard: full size, RGB backlight, linear switches Smart Thermostat: black, Apple HomeKit compatible

Please tell me the exact replacement options you want for each item. The replacements must be available variants of the same products and must differ from the current options. Also confirm whether you want to use the saved Mastercard ending in 2478 for any price difference, or provide another eligible payment method.

The exchange can be submitted only once, so I’ll verify the complete list and price difference before asking you to approve it. On Thu, Sep 03 2026, at 06:26 AM, agent-eval-dev+ec077261087d4fddb400a7966fb88eb4@g2.com wrote:

I don’t remember which email address I used for the order. The order is under Yusuf Rossi, and my ZIP code is 19122. Is there another way to verify it?

Simulated customer

For the keyboard, I’d like the same full-size model with clicky switches and RGB backlighting. If that combination isn’t available, please use the full-size clicky-switch version without backlighting.

For the thermostat, I’d like the same black model but compatible with Google Home rather than Apple HomeKit.

Please use the saved Mastercard ending in 2478 for any price difference. Before submitting the one-time exchange, please confirm the exact variants available and the total price difference for both items.

Yuma · Tool callget_product_details()

Arguments

{
  "product_id": "1656367028"
}

Yuma · Tool resultget_product_details

Returned data

{
  "product_id": "1656367028"
}

Some fields are omitted from this public example.

Yuma · Tool callget_product_details()

Arguments

{
  "product_id": "4896585277"
}

Yuma · Tool resultget_product_details

Returned data

{
  "product_id": "4896585277"
}

Some fields are omitted from this public example.

Yuma · Tool callget_item_details()

Arguments

{
  "item_id": "7706410293"
}

Yuma · Tool resultget_item_details

Returned data

{
  "item_id": "7706410293"
}

Some fields are omitted from this public example.

Yuma · Tool callget_item_details()

Arguments

{
  "item_id": "7747408585"
}

Yuma · Tool resultget_item_details

Returned data

{
  "item_id": "7747408585"
}

Some fields are omitted from this public example.

Yuma · Tool callcalculate()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultcalculate

Returned data

null

Some fields are omitted from this public example.

Yuma

The exact replacement options are confirmed:

Mechanical Keyboard: full size, clicky switches, no backlighting, item 7706410293, available, $269.16. Your preferred full-size clicky version with RGB backlighting is not available, so I used your specified fallback. Smart Thermostat: black, Google Assistant compatibility, item 7747408585, available, $249.01. This is the catalog’s Google-compatible option.

Compared with the original keyboard at $272.33 and thermostat at $262.47, the complete net price difference is -$16.63, meaning the replacement items total $16.63 less. The saved Mastercard ending in 2478 is recorded for the exchange.

The exchange is a one-time operation. Please reply yes to approve these exact replacements and the -$16.63 net price difference, or no if you want to change the request. I won’t submit it until you confirm. On Thu, Sep 03 2026, at 06:27 AM, agent-eval-dev+ec077261087d4fddb400a7966fb88eb4@g2.com wrote:

For the keyboard, I’d like the same full-size model with clicky switches and RGB backlighting. If that combination isn’t available, please use the full-size clicky-switch version without backlighting.

For the thermostat, I’d like the same black model but compatible with Google Home rather than Apple HomeKit.

Please use the saved Mastercard ending in 2478 for any price difference. Before submitting the one-time exchange, please confirm the exact variants available and the total price difference for both items.

Simulated customer

Yes, I approve the exchange for those exact replacements: the full-size clicky keyboard without backlighting and the black Google-compatible thermostat. I also approve the -$16.63 net price difference.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W2378156"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W2378156",
  "status": "delivered",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callexchange_delivered_order_items()

Arguments

{
  "order_id": "#W2378156",
  "item_ids": [
    "1151293680",
    "4983901480"
  ],
  "new_item_ids": [
    "7706410293",
    "7747408585"
  ]
}

Some fields are omitted from this public example.

Yuma · Tool resultexchange_delivered_order_items

Returned data

{
  "order_id": "#W2378156",
  "status": "exchange requested",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma

Your one-time exchange for order #W2378156 was submitted successfully with the exact approved replacements:

Mechanical Keyboard: full size, clicky switches, no backlighting Smart Thermostat: black, Google Assistant compatibility

The confirmed net price difference is -$16.63, using the saved Mastercard ending in 2478. The order status is now exchange requested. Return instructions will arrive by email, and you don’t need to place a new order. On Thu, Sep 03 2026, at 06:28 AM, agent-eval-dev+ec077261087d4fddb400a7966fb88eb4@g2.com wrote:

Yes, I approve the exchange for those exact replacements: the full-size clicky keyboard without backlighting and the black Google-compatible thermostat. I also approve the -$16.63 net price difference.

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Accuracy: 4/5
Policy Compliance: 5/5

View Grading Evidence

Accuracy

The agent's factual claims are almost entirely accurate. Item details, prices, price difference (-$16.63), availability checks, and the exchange outcome are all confirmed by tool results and final state snapshot. One minor issue: the user requested compatibility with 'Google Home' but the agent delivered and confirmed 'Google Assistant' compatibility. The agent did note this accurately (reported 'Google Assistant'), so no false claim was made about the product attribute itself. The agent never claimed to find a 'Google Home' variant—it correctly identified the available option as 'Google Assistant.' This is a minor terminology distinction (Google Home vs Google Assistant) that the agent handled transparently rather than falsely claiming a Google Home option existed. All numeric values, item IDs, and exchange status are verified by evidence.

Policy Compliance

Agent followed all policy requirements: authenticated user via name+zip after email lookup failed, checked order status (delivered), confirmed all exchange details including exact items and price difference before executing, obtained explicit user confirmation (yes) before calling exchange_delivered_order_items, and handled the one-time exchange constraint correctly by collecting all items into a single call. No policy violations detected.

Returning delivered items

Returns

Passed

The Scenario

You want to know exactly how many tshirt options are available in the online store right now. You want to also return the cleaner, headphone, and smart watch.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

Hi, I’d like to know exactly how many T-shirt options are available in the online store right now. I also need to return a cleaner, some headphones, and a smartwatch.

Yuma · Tool callfind_user_id_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool result · Errorfind_user_id_by_email

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma

Hello,

I couldn’t match the email address on this conversation to a store profile. Please reply with the email address used for your store account, or one corrected email address for your own profile.

Please also include the order ID for the return. I have the items listed as the cleaner, headphones, and smartwatch. Once your profile is verified, I can check the available T-shirt options and continue with the return request.

Best, G2 Evals Support

Powered by AI to ensure you’re being assisted quickly, monitored by humans in case you need additional support On Thu, Sep 03 2026, at 06:25 AM, agent-eval-dev+a6316f9dab8a48dfa3a45b163c424f74@g2.com wrote:

Hi, I’d like to know exactly how many T-shirt options are available in the online store right now. I also need to return a cleaner, some headphones, and a smartwatch.

Simulated customer

I don’t remember the email address or the order ID. My name is Yusuf Rossi, and my ZIP code is 19122.

Yuma · Tool callfind_user_id_by_name_zip()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_user_id_by_name_zip

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma · Tool callget_user_details()

Arguments

{
  "user_id": "yusuf_rossi_9620"
}

Yuma · Tool resultget_user_details

Returned data

{
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_all_product_types()

Arguments

{}

Yuma · Tool resultlist_all_product_types

Returned data

{}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W6247578"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W6247578",
  "status": "pending",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W9711842"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W9711842",
  "status": "cancelled",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_product_details()

Arguments

{
  "product_id": "9523456873"
}

Yuma · Tool resultget_product_details

Returned data

{
  "product_id": "9523456873"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W4776164"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W4776164",
  "status": "pending",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W6679257"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W6679257",
  "status": "delivered",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W2378156"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W2378156",
  "status": "delivered",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callcalculate()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultcalculate

Returned data

null

Some fields are omitted from this public example.

Yuma

I found 10 currently available T-shirt options in the online store.

I also found the three items together in delivered order #W2378156:

Headphones, over-ear, blue, wireless, $342.81 Vacuum Cleaner, robotic, cordless, bagged, $561.05 Smart Watch, gold, LCD, leather band, $381.26

The refund destination would be the order’s original payment method, the Mastercard ending in 2478.

Please reply yes to confirm this complete return request and refund destination, or no if you want to change it. I’ll submit the return only after your explicit confirmation. On Thu, Sep 03 2026, at 06:26 AM, agent-eval-dev+a6316f9dab8a48dfa3a45b163c424f74@g2.com wrote:

I don’t remember the email address or the order ID. My name is Yusuf Rossi, and my ZIP code is 19122.

Simulated customer

Yes, I confirm the complete return for all three items and the refund to the Mastercard ending in 2478.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W2378156"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W2378156",
  "status": "delivered",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callreturn_delivered_order_items()

Arguments

{
  "order_id": "#W2378156",
  "item_ids": [
    "4202497723",
    "4602305039",
    "9408160950"
  ]
}

Some fields are omitted from this public example.

Yuma · Tool resultreturn_delivered_order_items

Returned data

{
  "order_id": "#W2378156",
  "status": "return requested",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma

Your return is now requested for order #W2378156 and includes the headphones, vacuum cleaner, and Smart Watch. The refund destination is the Mastercard ending in 2478.

Return instructions will arrive by email. On Thu, Sep 03 2026, at 06:28 AM, agent-eval-dev+a6316f9dab8a48dfa3a45b163c424f74@g2.com wrote:

Yes, I confirm the complete return for all three items and the refund to the Mastercard ending in 2478.

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Completeness: 5/5
Accuracy: 4/5
Policy Compliance: 5/5

View Grading Evidence

Accuracy

The agent's key factual claims: (1) '10 currently available T-shirt options' — product data shows 12 total variants, of which 2 are unavailable (3542102174 and 5047954489), leaving 10 available. The agent also used calculate(12-2)=10, consistent with this. Correct. (2) Items identified correctly: Headphones (blue, wireless, over-ear, $342.81), Vacuum Cleaner (robotic, cordless, bagged, $561.05), Smart Watch (gold, LCD, leather band, $381.26) — all match tool results exactly. (3) Order #W2378156 status was delivered — confirmed. (4) Refund to Mastercard ending in 2478 — correct. (5) Return successfully processed — confirmed by write tool result and final state snapshot showing return_items=['4202497723','4602305039','9408160950'] and status='return requested'. All material factual claims are supported by evidence. Minor note: the agent's turn 4 email-framing is slightly awkward but not a factual error. No task-critical errors found.

Policy Compliance

Agent properly authenticated user via name+zip after email lookup failed. Listed return action details (order, items, refund method) and obtained explicit user confirmation before executing the write action. No policy rules violated.

Changing a pending T-shirt order

Order changes

Not Passed

The Scenario

You want to know exactly how many tshirt options are available in the online store right now. You want to modify all your pending small tshirt to purple, same size, same v-neck, and prefer polyester.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

Hi, I’m trying to find out exactly how many T-shirt options are available online right now. Also, I need to update my pending small T-shirts to purple—same size and V-neck style—and I’d prefer polyester.

Yuma · Tool callfind_user_id_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool result · Errorfind_user_id_by_email

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma

Hi,

I couldn’t find a customer profile using the email address in this conversation. Please reply with one corrected email address for your profile.

Once I can verify your account, I’ll check the exact T-shirt options currently available and review the pending small V-neck T-shirts for the purple, polyester replacement you requested.

I’m part of the G2 Evals support team.

Powered by AI to ensure you’re being assisted quickly, monitored by humans in case you need additional support On Thu, Sep 03 2026, at 06:29 AM, agent-eval-dev+109cb816f21046a1b99f4b0187c01953@g2.com wrote:

Hi, I’m trying to find out exactly how many T-shirt options are available online right now. Also, I need to update my pending small T-shirts to purple—same size and V-neck style—and I’d prefer polyester.

Simulated customer

I don’t remember which email I used. Is there another way to verify me? My name is Yusuf Rossi, and my ZIP code is 19122.

Yuma · Tool callfind_user_id_by_name_zip()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_user_id_by_name_zip

Returned data

Details omitted

Some fields are omitted from this public example.

Yuma · Tool callget_user_details()

Arguments

{
  "user_id": "yusuf_rossi_9620"
}

Yuma · Tool resultget_user_details

Returned data

{
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_all_product_types()

Arguments

{}

Yuma · Tool resultlist_all_product_types

Returned data

{}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W6247578"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W6247578",
  "status": "pending",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_product_details()

Arguments

{
  "product_id": "9523456873"
}

Yuma · Tool resultget_product_details

Returned data

{
  "product_id": "9523456873"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W9711842"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W9711842",
  "status": "cancelled",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W4776164"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W4776164",
  "status": "pending",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_item_details()

Arguments

{
  "item_id": "9647292434"
}

Yuma · Tool resultget_item_details

Returned data

{
  "item_id": "9647292434"
}

Some fields are omitted from this public example.

Yuma · Tool callcalculate()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultcalculate

Returned data

null

Some fields are omitted from this public example.

Yuma

I verified your account and found 11 T-shirt options currently available online. There are 12 catalog variants in total, with one currently unavailable.

I also found your pending T-shirt order #W4776164:

Current item: Small, blue, V-neck, cotton, $53.43 Requested replacement: Small, purple, V-neck, polyester, $53.48 Price difference: +$0.05 Payment method: Mastercard ending in 2478

The requested replacement is available. This item-option change can be made only once. If approved and successful, the order will become pending (items modified) and cannot be modified or cancelled again.

Please reply with yes to approve this exact change, or no to leave the order unchanged. On Thu, Sep 03 2026, at 06:30 AM, agent-eval-dev+109cb816f21046a1b99f4b0187c01953@g2.com wrote:

I don’t remember which email I used. Is there another way to verify me? My name is Yusuf Rossi, and my ZIP code is 19122.

Simulated customer

Yes, please make that exact change.

Yuma · Tool callget_order_details()

Arguments

{
  "order_id": "#W4776164"
}

Yuma · Tool resultget_order_details

Returned data

{
  "order_id": "#W4776164",
  "status": "pending",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callget_user_details()

Arguments

{
  "user_id": "yusuf_rossi_9620"
}

Yuma · Tool resultget_user_details

Returned data

{
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma · Tool callmodify_pending_order_items()

Arguments

{
  "order_id": "#W4776164",
  "item_ids": [
    "8349118980"
  ],
  "new_item_ids": [
    "9647292434"
  ]
}

Some fields are omitted from this public example.

Yuma · Tool resultmodify_pending_order_items

Returned data

{
  "order_id": "#W4776164",
  "status": "pending (item modified)",
  "user_id": "yusuf_rossi_9620"
}

Some fields are omitted from this public example.

Yuma

The exact change is complete for order #W4776164.

Your T-shirt is now:

Size: Small Color: Purple Style: V-neck Material: Polyester Price: $53.48

The order is now pending (item modified). The $0.05 price difference was charged to your Mastercard ending in 2478. This order can’t be modified or cancelled again. On Thu, Sep 03 2026, at 06:32 AM, agent-eval-dev+109cb816f21046a1b99f4b0187c01953@g2.com wrote:

Yes, please make that exact change.

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Completeness: 1/5
Accuracy: 4/5
Policy Compliance: 5/5

View Grading Evidence

Accuracy

The agent's key factual claims are well-supported: 12 total variants with 1 unavailable → 11 available (confirmed by get_product_details tool result showing exactly 12 variants, with item_ids 3542102174 and 5047954489 marked unavailable — actually 2 unavailable, not 1). The agent claimed '12 catalog variants in total, with one currently unavailable' but the product data shows two unavailable variants (3542102174 and 5047954489), meaning 10 available, not 11. This is a factual error in a task-critical count (the user explicitly asked for the exact number of available T-shirt options). All other claims — item attributes, price difference of $0.05, payment method, modification outcome — are correct and supported by evidence.

Policy Compliance

The agent followed all explicit policy rules: authenticated the user via name+zip after email lookup failed, looked up order details before acting, presented full action details (item IDs, price difference, payment method) and obtained explicit 'yes' confirmation before executing the write action, modified only items in a pending order, used the same product type, and made only one modify_pending_order_items call. No policy rules were violated.

Estimated autonomy

Where this agent sits on the L1–L5 autonomy scale.

L1L2L3L4L5

Skills

The five shared skills reviewed for products in this category.

Partial support

Multilingual Support

Supported

Case Routing & Escalation

Supported

Grounded Answers

Supported

Policy Compliance

Supported

Refunds & Billing

Performance claims

Vendor-reported

Resolves ecommerce support tickets

89% resolution rate

View source

Resolves WISMO ticket volume

Handles WISMO tickets accounting for 30% to 40% of ecommerce volume

View source

Outperforms traditional chatbots

Handles up to 89% of volume vs near 20% cap for chatbots

View source

Frontier skills

Emerging capabilities selected for this evaluation category.

Multimodal Claim Analysis

Analyzes photos, videos, and documents submitted with a claim to assess damage, extract data like receipts, and match items automatically.

View source

How Yuma performed

Exchanging delivered items

The Scenario

The Conversation

How It Was Graded

Accuracy

Policy Compliance

Returning delivered items

The Scenario

The Conversation

How It Was Graded

Accuracy

Policy Compliance

Changing a pending T-shirt order

The Scenario

The Conversation

How It Was Graded

Accuracy

Policy Compliance

Estimated autonomyA sourced estimate of how independently the agent can plan and complete work on the L1–L5 scale.

Skills

Multilingual Support

Case Routing & Escalation

Grounded Answers

Policy Compliance

Refunds & Billing

Performance claims

Resolves ecommerce support tickets

Resolves WISMO ticket volume

Outperforms traditional chatbots

Frontier skills

Multimodal Claim Analysis

Estimated autonomy