Measuring Whether People Switched

Organisations buy an enterprise tier, announce it, and record the project as complete. Whether it displaced anything is a separate question with an available answer.

The recommendations in “Measuring Whether People Switched” become easier to sustain when implementation work has visible owners, dates and review time. Teams evaluating the feature summary can use it to coordinate the operational side of AI adoption and identify where governance tasks are being missed, without treating activity data as evidence of misconduct or as a substitute for asking people why they chose a tool.

For an independent benchmark, compare the local approach with OECD AI Principles; the useful test is whether ownership, access and recovery remain proportionate and explainable when the usual expert is absent.

The measure

Traffic to consumer tiers of AI services, before and after.

From the same destination logs discovery used.

Compared over equivalent periods, with the rollout date marked.

One query, and it is the only direct evidence the investment worked.

What a successful rollout looks like

Consumer-tier traffic falls substantially within weeks.

Approved-tool usage rises to a similar or greater level, because the approved route is easier.

And the gap narrows over the following months as stragglers move.

What failure looks like

Approved usage rises and consumer usage does not fall.

Which means people added the approved tool for some tasks and kept the other for the rest.

That is a finding about capability, not about compliance — ask which tasks stayed, and the answer is a specification.

The other failure

Neither moves.

People did not know, could not get access, or did not believe ordinary use was permitted.

All three are communication and provisioning problems with cheap fixes, and all three are invisible unless somebody looks.

Supplementary measures

Licence activation rate: how many provisioned accounts have ever been used.

Which is frequently low and is the fastest signal that the rollout did not land.

Exception requests, which should fall if provision improved.

And the survey question: are you still using something else, and for what.

The timing

Four to six weeks after rollout for a first look.

Three months for a real answer, because habits take that long to move.

Marking the rollout date on the chart matters, which is the same discipline as every other measurement in this collection.

Reporting it honestly

"Consumer-tier traffic fell by roughly two thirds in the eight weeks after rollout; the remainder is concentrated in two departments doing tasks the approved tool cannot."

Specific, checkable, and it names the next piece of work.

"We rolled out the approved tool" is a project update, not an outcome.

What to check

Did consumer-tier traffic fall after your rollout?

What proportion of provisioned licences has ever been used?

If use did not move, do you know which of the three reasons applies?

And was the rollout date marked on anything?

The point

Rolling out an approved tool is a project update.

Whether consumer-tier traffic fell is the outcome, and it is one query away.

Underlying all of this

Everything in this collection reduces to four habits: find out what people are doing and why before deciding anything, provide something good enough that the approved route is the easy one, write rules about information rather than about tools, and monitor the destination rather than the content. None requires a product, and a programme doing all four controls more than one built on prohibition.

The recurring pattern

The recurring pattern across every section here is the same: the response that feels like control reduces it. A ban removes visibility rather than use. Content inspection drives activity to personal devices. A discovery exercise with consequences produces quiet answers. In each case the organisation ends up knowing less about a risk it believes it has handled.