Skip to content
Sections
All notes

All notes · Foundations

The Risks, Sorted by How Real They Are

The published risk lists mix severe, moderate and theoretical. Sorting them is what makes a proportionate response possible.

Foundations · Analysis

Vendor material lists every conceivable risk at equal weight, which makes everything urgent and therefore nothing actionable. Here they are ordered.

The recommendations in “The Risks, Sorted by How Real They Are” become easier to sustain when implementation work has visible owners, dates and review time. Teams evaluating billable hours tracker can use it to coordinate the operational side of AI adoption and identify where governance tasks are being missed, without treating activity data as evidence of misconduct or as a substitute for asking people why they chose a tool.

For an independent benchmark, compare the local approach with NIST AI Risk Management Framework; the useful test is whether ownership, access and recovery remain proportionate and explainable when the usual expert is absent.

Serious and common

Confidential data entered into a service whose terms permit retention or training. The core concern and the one worth building controls around.

Regulated data — health, financial, personal — going somewhere with no contract covering it. The legal exposure is real and jurisdiction-specific.

Output used without verification where accuracy matters: figures, legal positions, medical or safety content.

Serious and less common

Automations running unattended against an unassessed service, which turns a one-off into a continuous flow.

Credentials or keys pasted into a prompt, which happens and is worse than most people assume.

Code of unclear provenance entering a product, which has its own note and is primarily a licensing question.

Moderate

Loss of record-keeping: work done in a tool the organisation cannot see or retain.

Inconsistent output quality across a team using different tools.

Supplier concentration, where a service becomes load-bearing with no agreement behind it.

Overstated in most discussions

Model memorisation of your specific input resurfacing in somebody else's output. Theoretically possible, extremely unlikely for a single input, and it dominates the conversation.

instruction smuggling as a threat to ordinary office use. Real for agents with tool access, largely not for a person drafting an email.

Competitors learning your secrets through a shared model. A compelling story with little supporting evidence.

Why the ordering matters

A programme that treats all of these as equal produces a ban.

A ban produces invisible use, which removes your ability to control the three serious items at the top.

Sorting is what allows a rule that distinguishes pasting a contract from drafting an email, which is the only kind of rule that gets followed.

Doing your own sorting

Take the list above and mark each: has this happened here, could it, what would it cost.

Most organisations find two or three that genuinely apply and several that do not.

Write down the ones you are accepting, with the reason, so they stop reappearing in every discussion.

What to check

Which of the serious items has actually occurred in your organisation?

Does your policy distinguish them from the moderate ones?

Has anybody written down which risks you are accepting?

And how much of your internal discussion is about the overstated category?

The point

Sorting the risks is what allows a rule that distinguishes pasting a contract from drafting an email, and that is the only kind of rule people follow..

Underlying all of this

Everything in this collection reduces to four habits: find out what people are doing and why before deciding anything, provide something good enough that the approved route is the easy one, write rules about information rather than about tools, and monitor the destination rather than the content. None requires a product, and a programme doing all four controls more than one built on prohibition.

The recurring pattern

The recurring pattern across every section here is the same: the response that feels like control reduces it. A ban removes visibility rather than use. Content inspection drives activity to personal devices. A discovery exercise with consequences produces quiet answers. In each case the organisation ends up knowing less about a risk it believes it has handled.