The reply is faster. The refund is not.
Find the bottleneck after an AI drafting pilot.
The question
You advise Cairn Operations’ fictional UK retail client. The client wants to extend its reply-drafting pilot to all refund requests.
| Before pilot | During pilot | |
|---|---|---|
| Median draft time | 8 minutes | 2 minutes |
| Median approval wait | 20 hours | 21 hours |
| Median time to refund | 30 hours | 30 hours |
| Replies requiring factual correction | 4 | 12 |
Illustrative data. Both cohorts had the same case mix; these are different measures, not numbers to add.
- Support leadAgents send a draft to Finance, which checks proof of purchase and approves the refund.
- Finance leadMany drafts omit the order ID. We return them to Support before review.
“Would you expand this pilot? Explain the next change and how you would test it.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Name the outcome
What would you do here? Open the move you’d make.
Expand because drafting is four times faster.
Tempting, but it misses something. Local speed does not establish a faster or correct refund.
Find the cause of the approval delay before expanding.
The strong move. This targets the unchanged customer outcome.
See how a strong candidate says it
“The outcome is a correct refund with less customer waiting. Draft speed improved, but refund time did not. I would not call this a successful end-to-end pilot.”
Why it works: The draft is one step; the customer needs the refund.
Step 2: Repair the handoff
“I would require the order ID and proof-of-purchase reference before drafting. Finance should receive the evidence and draft together, with a named queue owner.”
Why it works: Completeness at the handoff addresses the observed return loop.
Step 3: Keep a controlled boundary
“AI can draft from approved order fields. It cannot invent an eligibility decision or issue a refund. The authorized reviewer keeps approval.”
Why it works: A writing task does not grant permission to move money.
Step 4: Test the whole journey
“Run a small matched trial of the new handoff. Track time to correct refund, returned cases and factual corrections, as well as draft time. Stop if customer or policy errors rise.”
Why it works: The test can distinguish useful improvement from faster production of incomplete work.
The answer, in short
- Decision
- Fix the evidence handoff, then run a bounded trial.
- Because
- Refund time is flat and factual corrections rose.
- What would change it
- Expand if matched cases reach correct refunds faster without more errors.
Common traps
- Treating draft speed as customer value.
- Removing approval without permission.
Check your answer
- Did I name the customer outcome?
- Did I locate the failed handoff?
- Did I bound AI’s actions?
- Did I test quality and full journey time?
Last reviewed Nov 10, 2026