Klarna’s AI assistant is a widely discussed production customer-service agent deployment, but its launch metrics measure conversation volume and speed — not resolution quality — and the 2025 reintroduction of human agents illustrates why deflection-first metrics are insufficient for agent teams.
This is a desk-research teardown of what was actually shipped, what each published number can and cannot prove, and what builders should instrument differently.
What Klarna Actually Shipped
In late February 2024, Klarna announced an AI assistant powered by OpenAI that had been live globally for one month, according to the vendor’s self-reported Klarna Press Release.
The deployment facts, all self-reported by the vendor in that release:
- Builder: powered by OpenAI, with quotes in the release from Klarna CEO Sebastian Siemiatkowski and OpenAI COO Brad Lightcap
- Tenure at announcement: live globally for one month
- Scope: available in 23 markets, 24/7, in more than 35 languages
- Task coverage: refunds, returns, payment issues, cancellations, disputes, and invoice inaccuracies
- Fallback: a live agent option was retained alongside the assistant
That scoping matters for builders. This was not an open-ended agent. It was a bounded customer-service agent handling high-volume, structured errands — the exact workload where deflection looks impressive first and resolution quality becomes visible later.
The Launch Metrics, as Published
All figures below are vendor self-reported figures from the Klarna Press Release. None were independently audited in the release. We summarize each claim as published so we can audit it in the next section:
- 2.3 million conversations handled by the AI assistant in its first month — self-reported by Klarna.
- Two-thirds of Klarna’s customer service chats handled by the assistant — self-reported by Klarna.
- Doing the equivalent work of 700 full-time agents — self-reported workload-equivalence claim by Klarna, methodology unpublished.
- On par with human agents on customer satisfaction score (CSAT) — self-reported by Klarna.
- 25% drop in repeat inquiries due to more accurate errand resolution — self-reported by Klarna.
- Customers resolve errands in under 2 minutes versus 11 minutes previously — self-reported by Klarna.
- Estimated to drive a USD 40 million profit improvement in 2024 — explicitly framed by Klarna as an estimate/projection, self-reported.
- Available in 23 markets, 24/7, in more than 35 languages — self-reported deployment scope from Klarna.
Those eight claims became the industry headline. Only two of them — CSAT parity and repeat inquiries — even attempt to measure quality. The rest measure volume, speed, scope, or projected money.
Metric Audit — What Each Number Proves and Doesn’t
Here is the core audit. Each row separates the narrow claim the metric actually supports from the broader inference teams often draw from it.
| Metric (as published) | Type | Source & attribution | What it proves | What it doesn’t prove |
|---|---|---|---|---|
| 2.3 million conversations in first month, self-reported | Volume | Vendor self-reported via Klarna Press Release | That the assistant served traffic at production scale in a 30-day window | That those conversations were correctly resolved, or that customers didn’t need further help |
| Two-thirds of customer service chats, self-reported | Volume / deflection share | Vendor self-reported via Klarna Press Release | Routing behavior: a large share of incoming chats went through the AI first | That the AI resolved those chats without escalation; deflection is not resolution |
| Equivalent work of 700 full-time agents, self-reported | Volume / workload equivalence | Vendor self-reported via Klarna Press Release, methodology unpublished | That Klarna models the handled volume as comparable to a large human team | Headcount replaceability, cost savings realized, or equal output quality |
| On par with human agents on CSAT, self-reported | Quality proxy | Vendor self-reported via Klarna Press Release | Parity on a single satisfaction metric in that launch window, as measured by Klarna | Superiority, durability over time, or correctness on complex or disputed cases |
| 25% drop in repeat inquiries, self-reported | Quality proxy | Vendor self-reported via Klarna Press Release | A correlational improvement in repeat-contact rate during the period | Causation — that the assistant alone caused the drop — or that first-contact resolution improved for all intent types |
| Under 2 minutes vs 11 minutes previously, self-reported | Speed | Vendor self-reported via Klarna Press Release | Much lower time-to-resolution as measured by Klarna | That faster answers were correct answers; speed without accuracy is just faster failure |
| USD 40 million profit improvement in 2024, self-reported estimate | Financial projection | Vendor self-reported estimate via Klarna Press Release | Klarna’s internal expectation of profit impact at launch | Realized savings, net of rework, escalation cost, or later reinvestment in human support |
| 23 markets / 24/7 / 35+ languages, self-reported | Deployment scope | Vendor self-reported via Klarna Press Release | Breadth of deployment: geography, availability, and language coverage | Depth of capability in any single market or language, or consistent quality across them |
Six of the eight claims are volume, speed, scope, or projection. Only two touch quality, and both are vendor-measured without published methodology.
The Derived Arithmetic (Original Artifact)
We can derive one useful context figure from Klarna’s self-reported numbers. This figure is derived, not published — it appears nowhere in the vendor release.
Inputs (both self-reported by Klarna):
- A = 2.3 million AI-handled conversations in first month
- B = two-thirds share of customer service chats (2/3 ≈ 0.667)
Computation: Total chats in period = A ÷ B = 2.3M ÷ (2/3) = 2.3M × 1.5 = 3.45M
Result (derived): ~3.45 million total customer-service chats in that first-month period.
What this derived ~3.45M lets us infer: the sheer scale of the funnel the assistant sat in front of. A two-thirds routing share at that volume means even a small miss rate creates a large absolute number of escalations or repeat contacts.
What it does not let us infer: resolution rate, escalation rate, cost per resolved errand, or whether total demand grew or shrank. You cannot get quality from volume arithmetic.
The 2025 Reversal, As Reported
Over a year after the 700-agent claim, Klarna was again recruiting humans for customer service, as reported by the press in May 2025 — not announced as a vendor primary study.
- Klarna again recruiting humans for customer service, over a year after the equivalent-work-of-700-agents claim, per press reporting in Customer Experience Dive.
- The CEO framed the move as a shift in priority toward high-quality human support, not a reversal on AI, per press reporting in Tech.co.
- The stated principle going forward was that customers must always be able to reach a human, per press reporting in Outsource Accelerator.
- Separately, the company had paused hiring for more than a year and reduced its workforce — described here only qualitatively with no headcount figure cited — per Fast Company.
To be clear on attribution: this reversal narrative comes from independent press reporting, not from a Klarna primary technical postmortem. There is no vendor-published root-cause analysis to cite.
The Gap Between Deflection and Resolution
The 2025 reporting is best read as a measurement lesson, not a model failure story.
Sources attribute the shift back toward humans to quality and customer experience — a priority on high-quality human support and guaranteed access to a human — and no single technical fault of the assistant should be claimed. The published record contains no evidence for one bug, one model regression, or one missing capability causing the reversal, so we do not assert one.
What the sequence illustrates for builders:
- Conversation count went up on the AI channel.
- Time-per-errand went down.
- CSAT was reported at parity at launch.
- Humans were reintroduced, with press attributing it to a priority on quality.
That combination is only contradictory if you assume deflection equals resolution. It doesn’t. An assistant can handle two-thirds of chats, resolve errands in under 2 minutes versus 11 minutes previously (both self-reported), and still leave a quality gap that only shows up in escalations, rework, complex disputes, or customers who simply want a person for money issues.
Volume metrics answer: did the agent talk to customers? Resolution metrics answer: did the customer’s problem stay solved without extra work? Klarna’s launch announcement, as published, was almost entirely the first kind.
Engineering Takeaways for Agent Builders
If you ship a customer-service agent, take Klarna’s scope discipline but not its headline metrics.
1. Instrument resolution quality as a first-class metric. Track resolved-without-recontact within 7 and 14 days per intent (refunds, disputes, invoice inaccuracies behave very differently). Track reopen rate and escalation-to-human rate. Segment by language and market — 35+ languages means nothing if three of them escalate at 3x.
2. Treat deflection as a secondary signal. Log this flow for every conversation:
- Incoming chat → routed to AI vs. human
- AI attempts → self-contained resolution vs. handoff requested vs. handoff forced by confidence threshold
- Post-handoff → human resolves in one touch vs. asks customer to repeat context
- Post-resolution → no recontact in 14 days vs. repeat inquiry
Deflection is step 1. Resolution quality lives in steps 3 and 4.
3. Keep a human path in the loop by design. Klarna retained a live-agent option even at launch, and later reporting emphasized customers must always be able to reach a human. Build it that way from day one: explicit escape hatch, full transcript + extracted entities passed to the human, no re-authentication, and a handoff SLA. Measure handoff quality — time-to-human, context preserved, and CSAT specifically for handed-off conversations.
4. Treat vendor self-reported launch numbers as directional only. CSAT parity without methodology, a 25% drop in repeat inquiries without a control, and a USD 40 million profit improvement labeled as an estimate are signals to investigate, not benchmarks to promise. Re-run them on your own traffic with your own audit.
The builder mantra from this case: measure resolution quality and handoff rates, not only deflection.
A Deflection-vs-Resolution Measurement Checklist (Second Artifact)
Before you trust any agent deployment’s numbers — yours or a vendor’s — answer these:
- Resolution defined? What counts as resolved: conversation ended, CSAT submitted, or no recontact in N days with no escalation?
- Handoff rate tracked? What % of AI conversations escalate to a human, by intent, language, and confidence band?
- Repeat contact joined? Can you link the AI conversation ID to a repeat inquiry within 14 days for the same customer and errand?
- Quality segmented? Are CSAT, resolution, and escalation broken out by refunds vs. disputes vs. invoice errors — not just averaged?
- Speed paired with correctness? Is every latency win (<2 min vs. 11 min style) paired with a correctness check on the same cohort?
- Human baseline current? Is AI CSAT compared to a concurrent human control, not a stale pre-launch average?
- Financials netted? Does any profit-improvement estimate subtract escalation handling, rework, and QA costs?
- Human path guaranteed? Can a customer always reach a human in under your SLA, with full context carried over?
If you can’t check the first four, you don’t have a resolution dashboard. You have a deflection dashboard.
FAQ
Was Klarna’s AI assistant actually handling customer service better than humans?
Based only on the published record, we can’t say it was better. Klarna self-reported that CSAT was on par with human agents, that repeat inquiries dropped 25%, and that resolution time fell from 11 minutes to under 2 minutes — all via the Klarna Press Release. Those are vendor self-reported, methodology unpublished, and parity is not superiority. No independent benchmark in the published record establishes better-than-human resolution quality.
Why did Klarna bring human agents back in 2025?
Press reporting attributes it to a priority shift toward high-quality human support and customer experience, not to a single technical fault. Per Tech.co, the CEO framed it as a priority shift, not a reversal on AI, and per Outsource Accelerator, the principle was that customers must always be able to reach a human. The move was reported by Customer Experience Dive over a year after the 700-agent claim.
Which of Klarna’s published metrics should agent teams actually track?
Track the quality proxies, but rebuild them rigorously: CSAT with a concurrent human control and segmentation by intent, and repeat-inquiry rate with identity-joined recontact windows. Treat the volume metrics (2.3M conversations, two-thirds share, 700-agent equivalent, all self-reported) and the speed metric (<2 min vs 11 min, self-reported) as capacity signals, not success criteria. Treat the USD 40 million figure as what it was labeled — an estimate.
What should builders measure instead of deflection rate?
Measure resolution quality and handoff rates as primary: first-contact resolution without recontact, escalation rate by intent and language, context-preservation on handoff, time-to-human when escalation happens, and CSAT for AI-resolved vs. handed-off vs. human-only cohorts. Deflection tells you where conversations went. Those tell you whether problems stayed solved.
Is this review based on hands-on testing of the assistant?
No. This review is based on the vendor’s published press release and independent press reporting; no hands-on testing was performed.
The Bottom Line
Klarna shipped a genuinely large-scale production agent — 2.3 million conversations at a two-thirds chat share across 23 markets and 35+ languages in a month, all self-reported — but headlined the wrong thing: how much the agent talked and how fast, not how often problems stayed solved. When Klarna reinvested in human agents a year later in the name of quality, it did not disprove the scale; it exposed the metrics gap. Builders should copy the deployment ambition and invert the metrics: resolution quality and handoff rates first, deflection second.
How This Guide Was Built
This review is based on the vendor’s published press release and independent press reporting; no hands-on testing was performed. Launch figures and scope come from the vendor self-reported Klarna Press Release. Reversal coverage comes from press reporting in Customer Experience Dive, Tech.co, Outsource Accelerator, and Fast Company. The ~3.45M total-chat figure is derived (2.3M ÷ 2/3) and labeled as such.
📖 Related Reads
- ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
- CodeIntel Log — code quality, debugging, and software engineering benchmarks
Cross-links automatically generated from NiteAgent.
← Back to all posts
