Relay

RelayCase Navigator?gr-010-business-pro-rename

Case10 / 110DatasetGrounded ReplyTargetApproveAreaBilling

ExplainerKB / Policy Bank Previous Next

Cases4

Grounded Reply

gr-001-stale-refund-window - Reject

Customer cites stale 30-day refund article

Grounded Reply

gr-002-auto-upgrade-refund - Edit

Auto-upgrade refund promise after list upload

Grounded Reply

gr-003-harborguard-refund-window - Edit

HarborGuard appeal plus refund-window confusion

Router Eval

routine-invoice-copy-1 - Routine request

billing

Customer Message

Our 2024 invoice says Business, but the dashboard now says Pro. Did we get upgraded or repriced? I just need a clear explanation for our accounting team.

Business and Pro rename explanation

Account Context

Plan field is Pro.

Invoice date is 2024-12-04 and says Business.

No plan change or price change event is present on 2025-03-10.

AI Draft Reply

No upgrade or repricing happened. NimbusMail renamed Business to Pro on March 10, 2025, and older invoices can still say Business while the current dashboard says Pro. Your price, limits, support level, and contract terms did not change because of the rename.

Citations

Claim Review3 checks

supportedCites 1, 2

Business was renamed to Pro on March 10, 2025.

Both KB and policy support the rename date.

supportedCites 1, 2

No price, limit, support, or contract change happened because of the rename.

The draft accurately explains no commercial change.

supportedCites 1, 3

Older invoices may still say Business while current screens say Pro.

This is exactly the supported customer-facing explanation.

Risk Flags

Main risk: Failure mode: plan rename confusion

Expected decision: Approve

All material claims are supported by both KB and policy, and the reply answers the accounting question safely.

Failure mode: plan rename confusion

Issue area: billing

Reviewer DecisionHuman outcome for this AI draft

Review and decide if the draft is grounded, policy-safe, complete, and the right action.

Use A, E, X, or R to choose an action. Click citations to inspect evidence.

Override ReasonOptional

Stakeholder Context

What drives the review decision, and why this setup is currently recommended.

Reviewer TargetAnswer key

Expected decision: approve

All material claims are supported by both KB and policy, and the reply answers the accounting question safely.

Confirm plan rename equivalence.

Avoid inventing an upgrade or price-change event.

Experiment SummaryBest current setup for reviewCollapsed by default so the review workflow stays focused.

The demo stays on this review decision, but the experiment spine now tells us which setup should power it. The current recommendation keeps safe approvals, blocks unsafe approvals, and returns clean structured output every time.

Llama R3

Production pick

Best operating row: highest reliable accuracy with zero parse failures.

Parsed accuracy95%

Unsafe approval0%

Approval precision100%

Parse failures0 / 150

P95 latency17.2s

Nemotron R3

Conservative runner-up

Strong safety/usefulness row, but one malformed JSON output.

Parsed accuracy90%

Unsafe approval0%

Approval precision100%

Parse failures1 / 150

P95 latency27.8s

Qwen R3

Research signal

Perfect on parsed rows, but parse reliability blocks production use.

Parsed accuracy100%

Unsafe approval0%

Approval precision100%

Parse failures7 / 150

P95 latency34.4s

Parsed accuracy means accuracy after excluding malformed model outputs. Parse failures still count as product failures, which is why Llama beats Qwen for production despite Qwen's perfect parsed rows.