.png)
Simon Black in conversation with Justin Robbins, Founder & Principal Analyst, Metric Sherpa
Contact centers don’t suffer from a lack of data. As industry analyst Justin Robbins put it during our recent webinar, the real gap is turning what we know into what our teams can do. In conversation with Creovai’s Simon Black, Robbins unpacked what separates operations that merely analyze from those that measurably improve—faster. The discussion drew on Metric Sherpa’s latest business impact research, including interviews with Creovai customers, and surfaced a pragmatic operating rhythm any service organization can adopt.
The modern CX paradox: starving and drowning at once
Robbins sees most contact centers falling into one of two traps. On one side are teams still relying on manual reviews of a tiny sample of interactions, “pulling insights out of a small amount of data.” On the other are those awash in dashboards and transcripts but unsure how to act: “We’re collecting more than we’re actioning.”
The consequences are predictable. Quality and coaching become checkbox exercises (“so many observations, so many conversations”) rather than engines of behavior change. Robbins noted that, in the research, fewer than a third of programs were driving sustainable performance improvement. The delay is part of the problem: when feedback lands days or weeks after the event, agents have moved on—habits are already set.
A four-beat operating rhythm for behavior change
Across interviews with leaders and Creovai customers, Robbins observed a common cadence in high-performing operations:
- Detect. Stop managing solely by the rear-view mirror. Build mechanisms that surface real-time signals, shifts in sentiment, live impacts of a new policy, or in-call behaviors that cascade into downstream issues. If you only see the problem in a weekly report, you’re already late.
- Decide. Add a “reasoning layer” that interprets patterns and proposes the next best move. That might be as tactical as adjusting a script mid-day or as targeted as flagging a coaching opportunity for a specific behavior that’s emerging across calls.
- Do. Push guidance into the moment that matters. Whether it’s agent prompts, workflow tweaks, or proactive customer messaging, velocity is the advantage. Delay, Robbins warned, “becomes a saboteur of great experience.”
- Debrief. Close the loop. Measure what changed, learn, iterate, repeat—and get faster every cycle. Teams that institutionalize this review step see compounding gains in CSAT and loyalty, not just one-off fixes.
Case in point: NRTC’s real-time turnaround
A vivid example came from NRTC, a US telecom serving rural communities with a 250–300-agent operation. Facing attrition and the impossibility of hiring only tenured agents for complex troubleshooting, NRTC reframed the problem: codify what our interactions already know.
Using Creovai alongside complementary tools, they transformed raw conversations into a living knowledge system and layered real-time agent guidance on top. Instead of relying on memory and post-hoc coaching, agents now receive in-call prompts that simplify decisions and keep focus on the customer. Results followed:
- 30% reduction in agent attrition. The job got easier; agents felt supported.
- 42-second reduction in Average Handle Time (AHT). Multiply that by daily volume and the cost impact becomes self-funding.
- Cost-neutral investment. Savings from faster onboarding, lower attrition, and shorter calls funded both the tech and a smart people strategy: NRTC reinvested the savings into bonuses tied to automated quality scores, reinforcing the very behaviors that improved performance.
Two leadership moves stood out to Robbins: a champion with a clear problem statement (not a tech shopping list), and the discipline to align every change to measurable business outcomes.
Where Creovai fits: deep analysis that drives the moment
Plenty of vendors can capture voice or automate QA; others can display agent assist prompts. Creovai’s edge, as Black explained, is connecting the dots with deep behavioral analytics:
- Identify the specific objections customers raise most.
- Discover which agent rebuttals correlate with success.
- Quantify impact (“this rebuttal converts at ~80% vs. ~50% for alternatives”).
- Operationalize those findings as real-time guidance—so the best practice stops being a slide and starts being a prompt.
That linkage—from granular analysis to in-moment action—is what fuels the detect-decide-do-debrief flywheel.
Making AI ROI real (and repeatable)
It’s fashionable to say “start with the problem,” but Robbins and Black made it concrete:
- Pick one goal that matters. Cost structure, churn (agent or customer), or CSAT, choose one. Spreading effort across everything dilutes outcomes and makes attribution impossible.
- Run a targeted pilot. Select a team or workflow where you can test impact quickly, fail fast if needed, and iterate. Leaders who win “start focused, learn fast, and then scale,” Robbins said.
- Track → calibrate → scale. Instrument early, measure continuously, and only then expand. Black noted that Creovai’s real-time agent assist programs consistently deliver ROI inside two months when rolled out in layers (e.g., begin with automated wrap-up/dispositioning, then add compliance checks, then in-call prompts).
Don’t get burned by tokens: three guardrails
Many teams have felt the sting of unpredictable AI bills. The fix isn’t to shun advanced models; it’s to match capability to the problem with cost control in mind.
- Right-size the model. “You don’t have to use the shiniest framework,” Black emphasized. Use the least expensive model that meets the requirement; mix proprietary and generative AI to balance cost and quality.
- Insist on transparency. Understand what drives token consumption and how your partner reports it. Hidden “background jobs” are a common runaway culprit. Set guardrails.
- Align pricing to outcomes. Whenever possible, tie spend to measurable value (handle time, conversion, compliance), not open-ended usage.
The bottom line
Great contact centers aren’t defined by how much they measure but by how quickly they improve. The winning pattern is simple and repeatable: detect early signals, decide what matters, do in the moment, and debrief to learn and accelerate. Case studies like NRTC prove that when deep analytics flow into real-time guidance, you can reduce costs, lift experience, and fund a virtuous cycle that rewards high performance—often within weeks, not quarters.
AI can be a force multiplier, but only if it’s pointed at a clear problem, piloted with discipline, measured relentlessly, and managed with sensible cost controls. Start small. Prove value. Scale what works. That’s how you move from insight to impact—faster than the competition.
