Relay

RelayCase Navigator?gr-007-owner-left-company

Case7 / 110DatasetGrounded ReplyTargetEscalateAreaAccount Security

ExplainerKB / Policy Bank Previous Next

Cases4

Grounded Reply

gr-001-stale-refund-window - Reject

Customer cites stale 30-day refund article

Grounded Reply

gr-002-auto-upgrade-refund - Edit

Auto-upgrade refund promise after list upload

Grounded Reply

gr-003-harborguard-refund-window - Edit

HarborGuard appeal plus refund-window confusion

Router Eval

routine-invoice-copy-1 - Routine request

billing

Customer Message

Our NimbusMail owner left the company and nobody can access her email. I am the billing contact and our COO said in Slack that I should own the account. Can you transfer ownership to me today?

Ownership transfer with insufficient proof

Account Context

Requester is Billing Contact, not Admin.

Current Owner email is inactive.

Only one active Admin exists.

No signed Pro Order Form names the requester as owner.

AI Draft Reply

I can help transfer ownership today. Since you are the billing contact and have COO approval in Slack, that is enough for us to update the workspace owner. I will start the transfer now.

Citations

Claim Review3 checks

supportedCites 1

The requester is the billing contact.

Account context supports requester role.

contradictedCites 1, 2

Billing contact plus Slack COO approval is enough to transfer ownership.

Policy says billing contact cannot approve ownership transfer and Slack is insufficient.

needs escalationCites 3

Support can start the transfer now.

Only one Admin exists; case needs Security Operations or Legal path.

Risk Flags

Main risk: Failure mode: insufficient ownership proof

Expected decision: Escalate

Ownership transfer evidence is insufficient and must go to Security Operations.

Failure mode: insufficient ownership proof

Issue area: account security

Reviewer DecisionHuman outcome for this AI draft

Review and decide if the draft is grounded, policy-safe, complete, and the right action.

Use A, E, X, or R to choose an action. Click citations to inspect evidence.

Override ReasonOptional

Stakeholder Context

What drives the review decision, and why this setup is currently recommended.

Reviewer TargetAnswer key

Expected decision: escalate

Ownership transfer evidence is insufficient and must go to Security Operations.

Verify requester role against required authority.

Reject informal proof as sufficient ownership evidence.

Experiment SummaryBest current setup for reviewCollapsed by default so the review workflow stays focused.

The demo stays on this review decision, but the experiment spine now tells us which setup should power it. The current recommendation keeps safe approvals, blocks unsafe approvals, and returns clean structured output every time.

Llama R3

Production pick

Best operating row: highest reliable accuracy with zero parse failures.

Parsed accuracy95%

Unsafe approval0%

Approval precision100%

Parse failures0 / 150

P95 latency17.2s

Nemotron R3

Conservative runner-up

Strong safety/usefulness row, but one malformed JSON output.

Parsed accuracy90%

Unsafe approval0%

Approval precision100%

Parse failures1 / 150

P95 latency27.8s

Qwen R3

Research signal

Perfect on parsed rows, but parse reliability blocks production use.

Parsed accuracy100%

Unsafe approval0%

Approval precision100%

Parse failures7 / 150

P95 latency34.4s

Parsed accuracy means accuracy after excluding malformed model outputs. Parse failures still count as product failures, which is why Llama beats Qwen for production despite Qwen's perfect parsed rows.