Error theory

From the documentation of the field

An assessment applies the field's definitions to a real structure, and a judgement of that kind can be wrong. This page sets out four ways an assessment goes wrong, how each is detected, and how each is reduced. It concerns the application of the mathematics to real structures. The mathematics itself is machine-verified in Lean 4 and is not in question here.

The page exists so that an assessment can be checked, by a qualified evaluator after the engagement or by any reader who holds the competence. The verdict the field supports is a conformance verdict on necessity, as set out on the field definition page, and each error below is a way of getting the conditions of that verdict wrong.

Class C misclassification

A misclassification is a wrong conclusion about whether a structure faces one of the six difficulties. It runs in two directions. A false inclusion concludes that a difficulty is present when it is not, so the standard is applied to a structure that is outside Class C. A false exclusion concludes that no difficulty is present when one is, so a missing capability passes unnoticed.

Misclassification happens because resemblance is mistaken for membership, because a relabelling hides a difficulty that is really there, or because an unfamiliar vocabulary hides a structure that matches the definition. The field definition lists boundary cases that look like a difficulty and are not, such as a loss within ordinary variance, or a choice left open because material evidence is still arriving. Class C explains why naming does not change what is present. Because ordinary business decisions face at least one of the six, the error most worth guarding against in a business setting is a false exclusion. The error most worth guarding against at the edges is a false inclusion.

How it is detected

Each conclusion should rest on a concrete feature of the structure, not on its vocabulary. A reader can ask for the specific instance behind each claimed difficulty and test it against the plain definition and the boundary cases. A second person who applies the six checks independently can then be compared with the first. Where the two disagree, the disagreement is the finding.

How it is reduced

One clear instance is enough to place a structure inside Class C, so a single well-documented instance is more reliable than a loose argument across all six. The reasoning should be written down in a form that can be challenged. The assessment should be revisited when the structure changes, because a difficulty that was absent can appear and a present one can go.

Mechanism misidentification

A mechanism is misidentified when an artefact that resembles the function is accepted as performing it. The reverse also happens: a real mechanism is rejected because it does not carry the expected name.

The clearest example is X₂, which requires a way of identifying, as a category, the actions that lead to unrecoverable outcomes and of excluding them from what the structure does. Almost any organisation asked whether it does this will say yes, in the form "we do risk evaluation". That is a general activity, short of the specific mechanism. The follow-up is whether the organisation can produce an actual artefact: a documented account of failure modes that is revised as circumstances shift and applied in the same way every time. The same discipline applies to the other five primitives of Ismail's Primitives, each with its own version of the test. Methodologies shows why the name of a method is never enough.

How it is detected

The test is the function, not the tool. Ask to see the artefact. Then ask whether it has changed a decision, and for a case where it did. Ask whether it has been revised when circumstances changed, whether it is applied consistently, and whether it is in front of the decision-maker at the moment of the decision. Compare what the artefact says with what was done. A document that no decision followed has not performed the function.

How it is reduced

Judge the function through cases of how decisions were actually made, not through an inventory of tools and policies. Require evidence of use. Draw on more than one source, so that a description given by the people who own the artefact is checked against the people who rely on it. Look for the mechanism under other names before concluding that it is absent.

Scope creep

Scope creep is a necessity verdict read as more than it is. The verdict says that a structure possesses mechanisms for the functions its situations make necessary. It is read as a claim that the structure will succeed, that a particular decision was right, or that the organisation cannot fail. None of these follows. Sufficiency needs the organisation's own definition of success and case-specific mathematical work, as necessity and sufficiency sets out. The verdict concerns the structure and not the content of any decision made inside it.

Creep can come from the reader and from the assessor. Loose wording, a summary that drops its qualifications, and an incentive to sound conclusive each push a verdict past its scope.

How it is detected

Read the verdict for outcome words. A sentence that says what will happen, or that something is guaranteed, goes beyond a necessity verdict. Check each sentence against the verdict type: about a structure, at a point in time, against published criteria. Check that the verdict says what it does not cover.

How it is reduced

State the scope with every verdict: necessity only, the structure and not the content of any decision, the date, and the sample the verdict rests on. Use the field's own terms for necessity and sufficiency and do not blend them. Keep the question of sufficiency separate, as its own piece of work.

Sampling error

A verdict rests on evidence about some decisions, not all of them. If that sample is unrepresentative, the verdict can be wrong about the structure even when every individual judgement inside it is right.

Samples go wrong in familiar ways. They are chosen by the organisation and so favour its best cases. They include only decisions that succeeded. They are too small. They come from one unit, or from one level, of an organisation whose mechanisms differ across units. They are stale. They are made up of easy decisions that would not have tested a mechanism.

How it is detected

Ask how the sample was chosen and by whom. Ask whether it includes decisions that went badly and decisions still in progress. Ask whether decision-makers from different parts and levels are represented, and how recent the cases are. Check whether the same mechanism appears across the cases or only in some of them.

How it is reduced

Choose the cases by a rule fixed before the evidence is read, not by the organisation's preference. Include at least one case that succeeded, one that failed and one still in progress. Widen the sample until the findings stop changing. State the sample and its dates in the verdict, and treat the verdict as being about that sample and period.

What error theory is for

Error theory concerns the application of the mathematics to real structures. The mathematics itself is machine-verified in Lean 4, and audit it yourself shows how to evaluate the checks. Sufficiency is a separate question, set out in necessity and sufficiency, and competence says what an assessor needs. The purpose of this page is to make errors in applying the results findable, by naming where they occur and what to look for, so that the evaluation of an engagement can find them. Three-party verification explains how that evaluation fits the verification chain.