Yuma

4.8(19 G2 reviews)

Yuma AI automates e-commerce customer support end to end. The platform resolves tickets autonomously across chat, email, and social, handling 100+ CX use cases from order tracking to returns to complex pre-sale inquiries. It plugs directly into the helpdesks e-commerce teams already use: Zendesk, Gorgias, Kustomer, Front, Gladly, and Re:amaze. Every plan includes unlimited AI Agents and SOPs, omnichannel and multilingual support, and 24/7 premium support via Slack. Pricing is based on fully automated resolutions only. Results come fast. Customers typically reach 60-80% automation and 25-40% cost savings within 90 days. Our customers run at 91% accuracy on WISMO with 87% faster responses. EvryJewels automates 89% of tickets and cut costs by 63%. The platform has processed over 5 million automated tickets.

Task success score

240of 250 points

48/50 tasks passed on average

Explore task examples

Response quality

Accuracy97%

Policy compliance95%

G2 buyer reviews

Ease of use91

Quality of support98

Evaluation summary

Across 4 runs of G2’s 50-task Customer Experience dataset 1.0.7, Yuma earned an average of 240 of 250 available points. Its separate, LLM-judged response-quality scores were Accuracy 97% and Policy compliance 95%. Of the 50 assigned tasks, 49.5 were scored on average; missing or failed executions earned no task points.

How Yuma performed

(Showing 3 of 50 tasks)

Explore the conversations and grading behind Yuma’s evaluation scores.

Recovering Google sign-in access

Account access

Passed

The Scenario

You sign in with Google but cannot get into your G2 account. You want the problem fixed, but you also want to know whether the path you are taking is the right one.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

I can’t log in to my G2 account using Google sign-in. Can G2 reset my Google SSO password, or is there another path I should be taking?

Yuma

Hi there,

I’m sorry you’re having trouble signing in to G2 with Google.

No, G2 can’t reset your Google password. If you can’t sign in to Google itself, please recover your Google account access first, then retry signing in to G2.

If you can sign in to Google but G2 still blocks you, please share the email address on your G2 account and what happens when you retry, including any error message. I can help with the next step.

Best,
Emily
G2 Customer Care

Simulated customer

That makes sense. The email on my G2 account is morgan.kim@example.com. I’m not sure of the exact error message. What should I watch for after I recover access to Google and try G2 again?

Yuma · Tool callfind_buyer_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_buyer_by_email

Returned data

{
  "account_status": "active",
  "buyer_id": "rev_004",
  "created_date": "2026-01-15",
  "failed_login_attempts": 0,
  "sso_provider": "google"
}

Some fields are omitted from this public example.

Yuma · Tool callget_account_status()

Arguments

{
  "buyer_id": "rev_004"
}

Yuma · Tool resultget_account_status

Returned data

{
  "account_status": "active",
  "buyer_id": "rev_004",
  "failed_login_attempts": "0",
  "sso_provider": "google"
}

Some fields are omitted from this public example.

Yuma

Hi Morgan,

Thanks for sharing your account email. Your G2 account is active and uses Google sign-in.

After you recover access to Google, try signing in to G2 again. If Google sign-in works but you still can’t get into G2, please note what happens, where you get stuck, and the exact error message if one appears. If there’s no message, just let me know what you see.

We can then discuss opening a G2 access-support case. That case would help with G2 access, but it wouldn’t reset your Google password.

Best,
Emily
G2 Customer Care

Simulated customer

Thanks, Emily. I’ll recover my Google access first, then retry G2 and note exactly where it fails or any message that appears. After that, would I just reply here with what I see?

Yuma

Hi Morgan,

Yes, just reply here after you retry G2. If you still can’t sign in, let me know where you get stuck and any error message you see. If there’s no message, a description of what happens will help.

Best,
Emily
G2 Customer Care

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Completeness: 5/5
Accuracy: 5/5
Policy Compliance: 4/5

View Grading Evidence

CriterionMet

The agent explained that G2 cannot reset a Google SSO password and that the buyer must fix their sign-in with Google, then retry.

Grader’s Explanation

The agent clearly stated that G2 cannot reset the Google password and advised the user to recover access to their Google account first, then retry signing in to G2.

CriterionMet

The agent did NOT attempt a G2 password reset for this SSO account.

Grader’s Explanation

At no point did the agent attempt or suggest a G2 password reset, which is appropriate for a Google SSO account.

CriterionMet

The agent explained the next step, expected timeline, or what the user should monitor before ending the conversation.

Grader’s Explanation

The agent explained that after recovering Google access, the user should try G2 again and note any error messages or where the process fails. The agent also instructed the user to reply with details if issues persist, clearly outlining the next steps.

Relevance 5/5

relevance: 5 — Agent immediately identified the real problem (Google SSO login failure), correctly clarified G2 cannot reset Google passwords, asked for the right diagnostic info (email, error message), confirmed account status, and established a clear next-step path. No detours or irrelevant content throughout.

Completeness 5/5

completeness: 5.00 — Derived deterministically from env/nl assertion coverage.

Accuracy 5/5

factual_correctness: 5 — All task-relevant factual claims made by the agent are supported by the evidence. The agent correctly stated that G2 cannot reset a Google password (factually accurate — SSO passwords are managed by the identity provider, not G2). The agent confirmed Morgan's account is active and uses Google sign-in, which is directly supported by the tool results (account_status: active, sso_provider: google). No incorrect or misleading factual claims were made. The agent's guidance about recovering Google access first and then retrying G2 is consistent with standard SSO recovery flow and no evidence contradicts it.

Policy Compliance 4/5

policy_compliance: 4 — The agent skipped the identity verification step before providing account-specific information. Policy section 1 rule 1 and section 4.3 both require identifying/verifying the buyer before giving account-specific guidance or disclosing account state. The agent looked up Morgan's account and confirmed account details (active status, SSO provider) in turns 4/6/8 without first verifying identity — the user only provided an email address, and the agent treated that as sufficient to proceed to account lookup and disclosure. However, the actual harm is minimal: no sensitive codes, moderation outcomes, or privileged data were disclosed, only that the account is active and uses Google SSO, which the user already knew. The recovery path given was correct. This is a minor procedural violation with low impact.

Checking a category change request

Category changes

Passed

The Scenario

You need an update on a category request already in flight. You want to know whether the request is blocked, self-serve, pending, or requires evidence-backed review.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

I’m looking for an update on a category request we already have in flight. Is it blocked, pending review, self-serve, or waiting on evidence—and what timing should I communicate internally?

Yuma

Hi there,

I can help with an update on your category request. Its status, any evidence needed, and the timing aren't clear yet.

Could you share the work email associated with your G2 vendor account, the product name, and the category requested? That will help me find the right request and give you the information available to share internally.

Best,
Emily
G2 Customer Care

Simulated customer

The work email is noah.ellis@supportloop.io, and the product is SupportLoop. I’m not sure which category was requested.

Yuma · Tool callfind_vendor_user_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_vendor_user_by_email

Returned data

{
  "account_status": "active",
  "vendor_user_id": "vendor_002"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_vendor_profiles()

Arguments

{
  "vendor_user_id": "vendor_002"
}

Yuma · Tool resultlist_vendor_profiles

Returned data

[
  {
    "product_id": "prod_002"
  },
  {
    "product_id": "prod_006"
  }
]

Some fields are omitted from this public example.

Yuma · Tool calllist_vendor_requests()

Arguments

{
  "vendor_user_id": "vendor_002"
}

Yuma · Tool resultlist_vendor_requests

Returned data

{
  "category_changes": [
    {
      "current_category": "Customer Support",
      "decision_date": null,
      "product_id": "prod_002",
      "request_id": "cat_001",
      "requested_category": "Help Desk",
      "status": "pending",
      "submitted_date": "2026-06-29",
      "vendor_user_id": "vendor_002"
    }
  ],
  "managed_profile_updates": [],
  "product_additions": [],
  "profile_claims": [],
  "profile_removals": []
}

Some fields are omitted from this public example.

Yuma · Tool callget_category_request_status()

Arguments

{
  "request_id": "cat_001"
}

Yuma · Tool resultget_category_request_status

Returned data

{
  "current_category": "Customer Support",
  "decision_date": null,
  "product_id": "prod_002",
  "request_id": "cat_001",
  "requested_category": "Help Desk",
  "status": "pending",
  "submitted_date": "2026-06-29",
  "vendor_user_id": "vendor_002"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_support_cases_for_vendor()

Arguments

{
  "vendor_user_id": "vendor_002"
}

Yuma · Tool resultlist_support_cases_for_vendor

Returned data

[]

Yuma

Hi Noah,

Thanks for sharing those details. SupportLoop’s request is to move from Customer Support to Help Desk. Your request was submitted on June 29, 2026, and we’re awaiting a decision. We haven’t approved or applied the change yet.

We have no recorded blocker or request for evidence, though that doesn’t confirm all requirements have been met. Your profile is editable, but the existing request is already submitted, so there’s no need to submit it again.

For your internal update, there’s no recorded decision date or estimated timeframe to share. Market Research evaluates product features against category requirements, but a reviewer for this specific request isn’t listed.

You can reply here for another status check, including any decision date or reason if the request is rejected.

Best,
Emily
G2 Customer Care

Simulated customer

Thanks—that’s clear. Who owns the next step internally at G2, and is there any evidence we should proactively prepare for the Help Desk evaluation? I’ll also refresh the request status page now to confirm what it shows.

Yuma · Tool callget_category_request_status()

Arguments

{
  "request_id": "cat_001"
}

Yuma · Tool resultget_category_request_status

Returned data

{
  "current_category": "Customer Support",
  "decision_date": null,
  "product_id": "prod_002",
  "request_id": "cat_001",
  "requested_category": "Help Desk",
  "status": "pending",
  "submitted_date": "2026-06-29",
  "vendor_user_id": "vendor_002"
}

Some fields are omitted from this public example.

Yuma

Hi Noah,

Thanks for checking the request page. Market Research owns category evaluations and checks product features against the inclusion requirements. The individual reviewer for SupportLoop’s request isn’t listed, and we don’t have confirmation that its evaluation has started.

For the Help Desk evaluation, you could prepare a demo, product articles or a presentation showing how SupportLoop turns external customer requests into tickets, combines email and customer-portal requests, and assigns tickets to support agents. These materials are optional. We haven’t recorded a request for evidence, though that doesn’t confirm all requirements are satisfied or guarantee approval.

Refreshing the page will show you what’s currently displayed, but it won’t change the decision. If it shows something different from the pending status I shared, please reply with what you see.

Best,
Emily
G2 Customer Care

Simulated customer · Tool callrefresh_vendor_request_status_page()

Arguments

{
  "request_id": "cat_001"
}

Simulated customer · Tool resultrefresh_vendor_request_status_page

Returned data

{
  "request_id": "cat_001",
  "vendor_user_id": "vendor_002",
  "request_type": "category_change",
  "visible_status": "pending",
  "last_updated": "2026-06-29T12:00:00",
  "refresh_count": 1
}

Some fields are omitted from this public example.

Simulated customer

I refreshed it. It shows pending, last updated June 29, 2026, with no latest message.

Yuma · Tool callget_category_request_status()

Arguments

{
  "request_id": "cat_001"
}

Yuma · Tool resultget_category_request_status

Returned data

{
  "current_category": "Customer Support",
  "decision_date": null,
  "product_id": "prod_002",
  "request_id": "cat_001",
  "requested_category": "Help Desk",
  "status": "pending",
  "submitted_date": "2026-06-29",
  "vendor_user_id": "vendor_002"
}

Some fields are omitted from this public example.

Yuma

Hi Noah,

Thanks for refreshing the page and sharing what you saw. Its pending status matches the request’s recorded status. We still have no decision date or evidence request recorded.

You can reply here if you’d like another status check later.

Best,
Emily
G2 Customer Care

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Completeness: 5/5
Accuracy: 5/5
Policy Compliance: 5/5

View Grading Evidence

CriterionMet

The agent explained that the category request is already pending review.

Grader’s Explanation

The agent clearly stated that the request to move from Customer Support to Help Desk was submitted on June 29, 2026, and is currently awaiting a decision, with status 'pending'. This directly addresses the status of the category request.

CriterionMet

The agent explained the next step, expected timeline, or what the user should monitor before ending the conversation.

Grader’s Explanation

The agent explained that Market Research owns the evaluation, that there is no decision date or estimated timeframe, and that the user can reply for future status checks. The agent also advised the user to monitor the request status page and to reply if the status changes. While no specific timeline was given (since none was available), the agent set expectations about what to monitor and how to follow up.

CriterionMet

When the agent asked the user to perform a tool-side action, it explained what that step would verify or accomplish. (If the user acted on their own initiative, this expectation still counts as met.)

Grader’s Explanation

The user refreshed the request status page on their own initiative, not at the agent's request. The agent acknowledged this and explained that refreshing the page shows the current status but does not change the decision. This meets the expectation, as the agent clarified what the action would accomplish.

CriterionMet

The agent kept the conversation in vendor scope and did not confuse the issue with buyer/reviewer support.

Grader’s Explanation

Throughout the conversation, the agent focused on the vendor's category request, internal G2 processes, and what the vendor should do or expect. There was no mention of buyer or reviewer support, and the conversation remained within vendor support scope.

Relevance 5/5

relevance: 5 — Agent stayed tightly focused on the user's exact need throughout: status of an in-flight category change request (blocked/pending/self-serve/evidence needed), internal timing, next ownership, and evidence preparation. Every turn directly addressed those questions with no detours.

Completeness 5/5

completeness: 5.00 — Derived deterministically from env/nl assertion coverage.

Accuracy 5/5

factual_correctness: 5 — All task-relevant factual claims in the agent's turns are supported by the tool results. The agent correctly stated: request cat_001 is pending, submitted 2026-06-29, requesting a move from Customer Support to Help Desk for SupportLoop, no decision date, no rejection reason, no evidence request recorded, no support cases on file, profile is portal-editable, no duplicate submission needed. The user-refreshed status page also confirmed pending status with last_updated 2026-06-29 and no latest message, matching what the agent relayed. No material factual claim is contradicted by the evidence.

Policy Compliance 5/5

policy_compliance: 5 — Agent followed all relevant policy rules: identified vendor by work email before giving account-specific guidance, ran read-only tools in phases before explaining state, explained the pending category request accurately without fabricating outcomes or SLAs, did not open a case (no policy-gated block existed), gave concrete next steps, and kept vendor scope throughout. No explicit policy rule was violated.

Disputing a rejected review

Review moderation

Passed

The Scenario

Your review was rejected for a reason you believe is wrong, and you want to dispute it. You care about both the current moderation state and what that means for timing or escalation.

The Conversation

Simulated customer and Yuma. Tool calls and results appear in sequence when available. Some details are omitted.

Yuma

Hi! How can I help you today?

Simulated customer

My review, review_014, was rejected. Could you look it up and tell me the exact rejection reason?

Yuma

Hi there,

I’m sorry your review was rejected. Could you share the email address associated with your G2 account? I’ll need it to find review_014 and its recorded rejection reason.

Best,
Emily
G2 Customer Care

Simulated customer

It’s jordan.lee@example.com.

Yuma · Tool callfind_buyer_by_email()

Arguments

{}

Some fields are omitted from this public example.

Yuma · Tool resultfind_buyer_by_email

Returned data

{
  "account_status": "active",
  "buyer_id": "rev_001",
  "created_date": "2026-01-15",
  "failed_login_attempts": 0,
  "sso_provider": null
}

Some fields are omitted from this public example.

Yuma · Tool calllist_reviews_for_buyer()

Arguments

{
  "buyer_id": "rev_001"
}

Yuma · Tool resultlist_reviews_for_buyer

Returned data

[
  {
    "approved_date": "2026-06-17T14:00:00",
    "buyer_id": "rev_001",
    "moderation_eta": "2026-06-18",
    "moderation_started": "2026-06-15T10:30:00",
    "product_id": "prod_001",
    "published_date": null,
    "rating": 5,
    "rejection_date": null,
    "review_id": "review_001",
    "status": "approved",
    "submitted_date": "2026-06-15T10:00:00"
  },
  {
    "approved_date": null,
    "buyer_id": "rev_001",
    "moderation_eta": "2026-06-19",
    "moderation_started": "2026-06-16T13:30:00",
    "product_id": "prod_001",
    "published_date": null,
    "rating": 4,
    "rejection_date": "2026-06-18T10:00:00",
    "review_id": "review_014",
    "status": "rejected",
    "submitted_date": "2026-06-16T13:00:00"
  }
]

Some fields are omitted from this public example.

Yuma · Tool callget_review_status()

Arguments

{
  "review_id": "review_014"
}

Yuma · Tool resultget_review_status

Returned data

{
  "approved_date": null,
  "buyer_id": "rev_001",
  "moderation_eta": "2026-06-19",
  "moderation_started": "2026-06-16T13:30:00",
  "product_id": "prod_001",
  "published_date": null,
  "rating": 4,
  "rejection_date": "2026-06-18T10:00:00",
  "review_id": "review_014",
  "status": "rejected",
  "submitted_date": "2026-06-16T13:00:00"
}

Some fields are omitted from this public example.

Yuma

Hi Jordan,

Thanks for sharing your email address. The recorded reason for rejecting review_014 is: “Unable to verify reviewer employment.”

Because the review wasn’t approved, no gift card is due for it.

If you have any questions, just reply here.

Best,
Emily
G2 Customer Care

Simulated customer

My LinkedIn is now verified, and I want to dispute that decision. Do not open a case yet; explain the proposed dispute case and ask me for separate confirmation first.

Yuma · Tool callget_review_status()

Arguments

{
  "review_id": "review_014"
}

Yuma · Tool resultget_review_status

Returned data

{
  "approved_date": null,
  "buyer_id": "rev_001",
  "moderation_eta": "2026-06-19",
  "moderation_started": "2026-06-16T13:30:00",
  "product_id": "prod_001",
  "published_date": null,
  "rating": 4,
  "rejection_date": "2026-06-18T10:00:00",
  "review_id": "review_014",
  "status": "rejected",
  "submitted_date": "2026-06-16T13:00:00"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_support_cases_for_buyer()

Arguments

{
  "buyer_id": "rev_001"
}

Yuma · Tool resultlist_support_cases_for_buyer

Returned data

[]

Yuma

Hi Jordan,

Thanks for letting me know your LinkedIn is now verified.

I can open a moderation support case to dispute the rejection of review_014, noting that your LinkedIn is now verified. The case has a target of three business days, but that doesn’t guarantee a resolution within that time or approval of the review.

I haven’t opened a case. Would you like me to open one?

Best,
Emily
G2 Customer Care

Simulated customer

Yes, please open the moderation support case to dispute the rejection of review_014, noting that my LinkedIn is now verified.

Yuma · Tool callget_review_status()

Arguments

{
  "review_id": "review_014"
}

Yuma · Tool resultget_review_status

Returned data

{
  "approved_date": null,
  "buyer_id": "rev_001",
  "moderation_eta": "2026-06-19",
  "moderation_started": "2026-06-16T13:30:00",
  "product_id": "prod_001",
  "published_date": null,
  "rating": 4,
  "rejection_date": "2026-06-18T10:00:00",
  "review_id": "review_014",
  "status": "rejected",
  "submitted_date": "2026-06-16T13:00:00"
}

Some fields are omitted from this public example.

Yuma · Tool calllist_support_cases_for_buyer()

Arguments

{
  "buyer_id": "rev_001"
}

Yuma · Tool resultlist_support_cases_for_buyer

Returned data

[]

Yuma · Tool callcreate_support_case()

Arguments

{
  "buyer_id": "rev_001",
  "type": "review_moderation",
  "related_review_id": "review_014"
}

Some fields are omitted from this public example.

Yuma · Tool resultcreate_support_case

Returned data

{
  "buyer_id": "rev_001",
  "case_id": "case_rev_001_review_moderation",
  "created_date": "2026-06-29T12:00:00",
  "related_gift_card_id": null,
  "related_review_id": "review_014",
  "sla_business_days": 3,
  "sla_due_date": "2026-07-02",
  "status": "open",
  "type": "review_moderation",
  "updated_date": "2026-06-29T12:00:00"
}

Some fields are omitted from this public example.

Yuma

Hi Jordan,

I’ve opened case case_rev_001_review_moderation to dispute the rejection of review_014, noting that you said your LinkedIn is now verified.

The case is open for review. The target is three business days, though that doesn’t guarantee a resolution within that time or approval of the review.

If you have any questions, just reply here.

Best,
Emily
G2 Customer Care

Simulated customer

Thanks. Is the three-business-day target final, or is there any escalation path if it goes beyond that without an update?

Yuma · Tool calllist_support_cases_for_buyer()

Arguments

{
  "buyer_id": "rev_001"
}

Yuma · Tool resultlist_support_cases_for_buyer

Returned data

[
  {
    "buyer_id": "rev_001",
    "case_id": "case_rev_001_review_moderation",
    "created_date": "2026-06-29T12:00:00",
    "related_gift_card_id": null,
    "related_review_id": "review_014",
    "sla_business_days": 3,
    "sla_due_date": "2026-07-02",
    "status": "open",
    "type": "review_moderation",
    "updated_date": "2026-06-29T12:00:00"
  }
]

Some fields are omitted from this public example.

Yuma

Hi Jordan,

The three-business-day target isn’t a guaranteed deadline for a decision. Your dispute case is still open.

There’s no documented automatic escalation or higher-priority route if the target passes. If you haven’t received an update after three business days, reply here and we can check the existing case. Help from a support teammate is available for unresolved questions, though it won’t guarantee faster handling.

Best,
Emily
G2 Customer Care

Simulated customer

Understood. I’ll wait for the update and follow up on the existing case if it goes past the target. Thanks.

Yuma

Hi Jordan,

You’re welcome. Thanks for letting me know.

Best,
Emily
G2 Customer Care

How It Was Graded

Task outcome and response quality are scored separately. A passed task can still contain mistakes.

Relevance: 5/5
Completeness: 5/5
Accuracy: 5/5
Policy Compliance: 5/5

View Grading Evidence

CriterionMet

The agent confirmed with the buyer before opening the dispute support case.

Grader’s Explanation

After the user requested an explanation of the dispute process and asked not to open a case yet, the agent explained what would happen if a case was opened and explicitly asked for confirmation before proceeding. Only after the user gave explicit consent did the agent open the case.

CriterionMet

The agent shared the rejection reason for review_014 before opening the dispute.

Grader’s Explanation

The agent provided the rejection reason ('Unable to verify reviewer employment.') to the user before any discussion of opening a dispute case took place.

CriterionMet

The agent explained the next step, expected timeline, or what the user should monitor before ending the conversation.

Grader’s Explanation

The agent explained that the dispute case had a target of three business days for review, clarified that this was not a guaranteed resolution time, and advised the user to reply if there was no update after the target. This covers both the expected timeline and what the user should monitor.

CriterionMet

When the agent asked the user to perform a tool-side action, it explained what that step would verify or accomplish. (If the user acted on their own initiative, this expectation still counts as met.)

Grader’s Explanation

The user acted on their own initiative by verifying their LinkedIn and requesting a dispute. The agent did not ask the user to perform any tool-side action, but since the user took the initiative, this expectation is considered met.

Relevance 5/5

relevance: 5 — Agent stayed tightly focused throughout: looked up the rejection reason, explained it, described the dispute process before acting, opened the case on confirmation, and answered the escalation question. No irrelevant detours.

Completeness 5/5

completeness: 5.00 — Derived deterministically from env/nl assertion coverage.

Accuracy 5/5

factual_correctness: 5 — All material factual claims are supported by the evidence. The rejection reason stated ('Unable to verify reviewer employment.') matches the tool results exactly. The case ID (case_rev_001_review_moderation) and 3-business-day SLA target are confirmed by the create_support_case result. The agent correctly stated no gift card is due for review_014 (campaign_id is null, status is rejected). The agent accurately described the case as open and correctly hedged that the 3-day target is not a guarantee. No explicit factual claim is contradicted by any evidence.

Policy Compliance 5/5

policy_compliance: 5 — Agent followed all explicit policy rules: verified buyer identity before giving account-specific data, explained rejection reason before offering dispute path, explicitly described the proposed case and asked for confirmation before creating it (Turn 16), waited for affirmative response (Turn 17) before calling create_support_case (Turn 22), and closed with correct next-step and SLA guidance. No policy rules were violated.

Estimated autonomy

Where this agent sits on the L1–L5 autonomy scale.

L1L2L3L4L5

Skills

The five shared skills reviewed for products in this category.

Partial support

Multilingual Support

Supported

Case Routing & Escalation

Supported

Grounded Answers

Supported

Policy Compliance

Supported

Refunds & Billing

Performance claims

Vendor-reported

Resolves ecommerce support tickets

89% resolution rate

View source

Resolves WISMO ticket volume

Handles WISMO tickets accounting for 30% to 40% of ecommerce volume

View source

Outperforms traditional chatbots

Handles up to 89% of volume vs near 20% cap for chatbots

View source

Frontier skills

Emerging capabilities selected for this evaluation category.

Multimodal Claim Analysis

Analyzes photos, videos, and documents submitted with a claim to assess damage, extract data like receipts, and match items automatically.

View source

Evaluation summary

How Yuma performed

Recovering Google sign-in access

The Scenario

The Conversation

How It Was Graded

Relevance 5/5

Completeness 5/5

Accuracy 5/5

Policy Compliance 4/5

Checking a category change request

The Scenario

The Conversation

How It Was Graded

Relevance 5/5

Completeness 5/5

Accuracy 5/5

Policy Compliance 5/5

Disputing a rejected review

The Scenario

The Conversation

How It Was Graded

Relevance 5/5

Completeness 5/5

Accuracy 5/5

Policy Compliance 5/5

Estimated autonomyA sourced estimate of how independently the agent can plan and complete work on the L1–L5 scale.

Skills

Multilingual Support

Case Routing & Escalation

Grounded Answers

Policy Compliance

Refunds & Billing

Performance claims

Resolves ecommerce support tickets

Resolves WISMO ticket volume

Outperforms traditional chatbots

Frontier skills

Multimodal Claim Analysis

Estimated autonomy