What side-channels taught me about assurance

Systems leak information in ways their designers never intended. So do compliance programmes. The gap between what a control claims and what it actually emits is where real assurance lives, and finding it requires a different kind of measurement.

What side channels are

In 1996, Paul Kocher published a paper demonstrating that the time taken to perform RSA private-key operations varied measurably with the value of the key. The cryptographic protocol was correct. The mathematics were sound. The specification did not include timing as an information channel. None of that mattered: the implementation leaked the key anyway, through a pathway nobody had designed and nobody had defended.

This is a side channel: an unintended information pathway created by the physical or operational characteristics of a system's behaviour, distinct from and outside the designed interface. Power consumption, electromagnetic emissions, cache access patterns, timing variations. None of these appear in a system's specification, and none are addressed by evaluations that test only the specified interface.

Differential power analysis, developed by Kocher and colleagues three years later, showed that even small statistical correlations between power consumption and secret data were sufficient to recover cryptographic keys from hardware that was, by every formal standard, secure. The designed interface was fine. The side channel was the real attack surface.

The security boundary was drawn around the designed interface. The attack went around it.

The compliance programme has side channels too

A control is a designed interface. It specifies a process, a set of inputs, a set of outputs, and a claim about what it achieves. An annual access review claims that inappropriate access is identified and removed. A penetration test claims that critical vulnerabilities are found and remediated. A vendor questionnaire claims that third-party risk is assessed and managed.

These claims describe what the control is supposed to do. They say nothing about what the control actually emits.

Consider an access review. Reviewers receive a list of accounts and are asked to confirm or revoke each one. The process is documented. The manager approval is recorded. The audit trail is retained. The designed interface is clean.

What does the control actually emit? In most organisations: reviewers confirm access because revoking it requires a separate process, a separate ticket, a separate team, and a follow-up that may never happen. Reviewers lack the context to understand what half the accounts are. Reviews are completed in bulk, minutes before the deadline. The process has run for three years without a single account being removed as a result.

The side channel says: the organisation does not know who has access to what, and has no mechanism for finding out. The designed interface says something different entirely.

Measuring the emission

Testing the designed interface asks: does the control exist, is it documented, is there evidence it ran? Testing the emission asks: what changed in the environment as a result of this control operating?

For the access review, the emission questions are straightforward. How many accounts have been removed in the past three cycles? What is the mean time from review completion to deprovisioning? When did the review population last expand to cover a new system that came into scope? Has the reviewer pool changed, and if so, how was continuity maintained?

These are not hostile questions. If the control is working, the answers are available and positive. If the control is not working, the questions reveal that fact quickly and precisely, far more quickly than repeating the designed interface test would.

The measurement approach generalises. For a penetration test: what findings from the previous assessment remain open? What is the mean time to remediation for critical findings across the last three cycles? Has the scope changed, and if so, what drove that change? For a vulnerability management programme: what is the current mean time to patch critical CVEs, and how does that compare to the commitment in the policy?

Why this gap exists

The gap between designed interface and actual emission is not usually the result of bad faith. It is the result of optimising for auditability rather than effectiveness.

Audit evidence is point-in-time and binary: the control either ran or it did not, the document either exists or it does not. Programmes designed primarily for audit evidence optimise for those binary tests. They produce documentation that passes the test. They do not necessarily produce controls that work.

This is not a criticism of audit as a function. Audit is a check on whether the designed interface is in place, and that check is necessary. The insufficient part is treating it as sufficient. Programmes that conflate the two gradually accumulate controls without accumulating assurance. The control list grows. The risk picture does not improve. The first indicator is usually an incident the control was supposed to prevent.

Building for emission

Three things change when you design controls to emit measurable outcomes rather than auditable evidence.

The first is that you define metrics before deployment rather than after. What would successful operation of this control look like in the environment? What would change? What numbers would move? These questions are easier to answer at design time than after the control has been running for two years and the team has reorganised twice.

The second is that control reviews become evidence reviews rather than process reviews. The question is not "did this run?" but "what did this find, and what happened as a result?" A review that cannot answer those questions is a review of the designed interface. It is not evidence of assurance.

The third is that you can distinguish between controls that are primarily about accountability and controls that are primarily about risk reduction. Both types are legitimate. An audit trail, a signed-off decision, a documented approval: the "emission" of these controls is the record, and a clean record is the intended output. But for controls whose purpose is to reduce real-world risk, the designed interface is rarely sufficient evidence of effectiveness. Knowing the difference, and being able to show which of the two you have, is the work.

When assessing third parties

The designed interface versus emission distinction matters most when assessing organisations you cannot directly observe. A vendor's security questionnaire response describes their designed interface. What their programme actually emits is harder to see, but not invisible.

Penetration test reports and red team findings describe what was discovered when someone looked for the gap between designed interface and actual behaviour. Bug bounty disclosure histories reveal how the organisation responds when that gap is found by an outsider. Regulatory inspection outcomes, where available, are a direct test of emission under external pressure. Incident disclosure history, both what happened and how quickly it was contained and communicated, says more about a programme's operational reality than any questionnaire.

Asking "do you have an access review process?" tests the designed interface. Asking "what did your last access review find, and what changed as a result?" tests the emission. The second question is harder to answer convincingly if the emission is poor, and that difficulty is itself informative.

A note on what this is not

Side-channel thinking applied to assurance is not an argument against controls, against documentation, or against audit. It is an argument against treating those things as sufficient.

Controls that exist, are documented, and are audited are better than controls that do not exist. Documentation creates accountability. Audit creates external verification. These are real goods.

The point is simply that they are not the same good as a programme that actually reduces risk. Knowing the difference, and being able to demonstrate which of the two you have built, is what separates assurance from theatre.