How to Measure AI Support Quality
To measure AI support quality honestly, track five things together: verified resolution rate (the primary KPI), reopen or recontact rate (the lie detector), CSAT segmented to AI-handled tickets, escalation quality, and cost per verified resolution. No single number is trustworthy in isolation, because each one can be gamed by loosening its definition. Quality lives in the combination.
The honest metrics checklist
- Verified resolution rate: confirmed by survey, confirmation, or no recontact. The headline number.
- Reopen / recontact rate: ideally under 5 percent, watched over a 72 hour window.
- Segmented CSAT: satisfaction on AI-handled tickets, reported separately from human-handled ones.
- Escalation quality: whether the tickets the AI passed up were the right ones, with full context.
- Cost per verified resolution: the true unit economics, not cost per deflection.
Metrics to distrust on their own
Deflection rate, containment rate, and raw "resolution" figures that blend deflection into the count are the numbers most often inflated. Treat any of them as a red flag when they appear without a reopen rate beside them. If a dashboard shows you only the flattering number, assume the unflattering one is worse.
How Resolve instruments it
Resolve derives auto-resolution rate, deflection, CSAT, SLA attainment, and top issues from real tickets, and shows auto-resolution and deflection side by side rather than merging them. Reopen rate is the north-star honesty metric. Because CSAT is collected on both AI-resolved and human-resolved tickets, you can compare the two directly instead of assuming the bot is fine.