Phishing simulations that teach instead of punish

A phishing simulation that produces a click rate and a list of names is a measurement exercise dressed as a training one. It tells the organisation something it already suspected, embarrasses a group of people who were doing their jobs quickly, and changes very little about what happens during a real campaign.

We run these programmes differently. The purpose is to build a reporting reflex and to test whether the controls behind the person hold up, which makes the design decisions look quite different from the ones a low click rate would optimise for.

Report rate is the metric that matters

Click rate measures a moment of attention. Report rate measures a behaviour the organisation can build on, because a reported message starts an investigation and a clicked one that nobody mentions is a hole in the timeline. We track how many people reported, how quickly the first report arrived, and what the security team did with it. A campaign where a fair number of people clicked and the first report landed within ninety seconds is a healthier result than one where nobody clicked and nobody said anything, because the second organisation has no signal at all when the real message arrives.

Design the lure to teach one lesson

Every campaign should have a single lesson attached to it, chosen before the template is written. That constraint rules out most of the lures that generate impressive numbers and teach nothing.

• Pick a pretext that resembles something staff genuinely receive, so the lesson transfers to their real inbox rather than to an artificial scenario.
• Vary difficulty deliberately across the year and say so in the debrief, because an obvious template and a well researched one measure different things.
• Avoid lures that trade on personal anxiety, such as fake bonuses, disciplinary notices or redundancy news; the anger outlasts anything learned.
• Keep the teaching moment short, immediate and specific to the cues in that message, rather than a generic module assigned a fortnight later.

Never attach a result to a person's record

The moment a simulation result can affect a performance review, the programme stops producing useful data. People warn each other, which is good behaviour that happens to ruin the measurement, and anyone who does click has every incentive to say nothing. That is precisely the outcome to avoid, because the cost of a real compromise is measured in the hours between the click and the report. We agree in writing before a campaign runs that reporting is aggregate, that individual data is kept only as long as the training loop needs it, and that repeat clickers get a conversation rather than a sanction.

The person who clicked and told you within a minute has given you the most useful thing anyone in the organisation can give you.

Test the controls, not only the people

A campaign is also an end to end test of a technical chain that rarely gets exercised. Did the message pass authentication checks at the gateway, and was the link rewritten. Did the endpoint agent see the download. Did the identity provider apply conditional access when the credentials were replayed from somewhere else. Credential phishing now runs through reverse proxy kits that sit between the user and the genuine login page, so a one time code from an authenticator app is captured and used within seconds. Phishing resistant factors bound to the origin, meaning security keys and platform passkeys built on WebAuthn, are the control that actually breaks that chain, and a simulation is a reasonable way to make the case for funding them.

Rules of engagement come first

ATT&CK describes phishing as an initial access technique, and simulating it means sending real messages to real colleagues, which makes the governance as important as the tradecraft. We agree scope, exclusions, retention period and escalation path with security, human resources and legal before anything is sent, and we make sure a named group knows the campaign is running so a genuine incident during the window is not waved away as the exercise. Sending results to a leadership meeting before the affected teams have seen them is the fastest way to lose the goodwill the whole programme depends on.

Run this way, a simulation programme is less dramatic and considerably more useful. The numbers move slowly, the time to first report drops, the security team gets earlier signal, and the conversation shifts from who clicked to how quickly we knew. We would rather deliver a campaign that leaves a room full of people willing to tell us what happened than one that wins an argument in a slide deck.