The film, in writing.
Imagine a client asking for a refund 45 days after delivery. The AI finds the standard 30-day policy and drafts a refusal. Maya catches it: this client has a separate agreement allowing 90 days, filed under the client’s old name. She knows where to find it and why it takes precedence.
Finding the document gets the team only part of the way. Someone has to connect it to the right client, keep it current and decide who can approve the reply. That is the work around the model: where it can act, which information applies, how the answer is checked and who takes responsibility.
I’d keep two tests from this correction. One checks that a client with a special agreement gets the right answer. The other checks that a client without one is still subject to the standard policy. Approving every refund would only replace one mistake with another. Keeping the request, the expected answer and the reason gives the team something it can check when a new model arrives.
A question to take to work
When someone on your team corrects an AI answer, what happens to the lesson?
What to keep in mind. A correction doesn’t train the model by itself. Someone still has to agree on the right answer, fix the information or workflow, and keep the tests current. Some of Maya’s judgment will continue to need a conversation.