How Far Does the Evidence Reach? 

An institution hires five hundred people to repair a capacity shortage.

The hiring is real. The records are accurate. The new employees are necessary.

Has capacity been restored?

The hiring records cannot say. They establish that hiring happened. Capacity is a different condition, reached further along a dependency chain: hiring, training, qualification, deployment, and performance under real demand. The pathway between them can be real while the records still describe only the first condition.

Wherever institutions measure their own progress, the same pair repeats. A policy is published, and the claim is that behavior changed. Components pass their tests, and the claim is that the system works. Funds are distributed, and the claim is that capability exists. In each pair, the evidence is real and the claim is downstream of it.

What the evidence establishes. A hiring record establishes how many people were hired. A training log establishes who completed training. A component test establishes how that component performed under the test's conditions. This is the evidence's home ground, and on its home ground the evidence can be strong. Strength and location are different properties: a record can be beyond dispute about the condition it measures and silent about everything downstream of it.

What the evidence makes more likely. If nearly everyone who completes a mature training program goes on to qualify, another hundred completions are informative about qualification. Ignoring that would throw away real information. Expectation of this kind is legitimate, and its strength depends on how well the connecting links are known: how stable the relationship has been, and whether the conditions that produced it still apply.

What the evidence is being asked to certify. Trouble begins when evidence from one point in the chain is presented as establishing a condition further along it, with nothing shown about the links between them. Hiring offered as proof of capacity. Training completion offered as proof of changed practice. Component tests offered as proof the system performs. The evidence can be accurate, the people reporting it honest, and the connection between the two conditions real. The claim has traveled farther than the evidence did.

Actions can be documented the day they happen. Outcomes take time. An institution repairing itself can show hiring, training, and new procedures before anyone can observe whether the repaired condition performs under real demand. The pressure to demonstrate progress arrives early, and the only evidence available early is evidence of intervention. So intervention evidence gets presented, and the temptation is to present it with the standing of outcome evidence.

The intervention may be exactly right. Its execution may be excellent. Neither makes the outcome arrive sooner.

Where does the evidence stop and the claim begin?

Find the condition the evidence directly establishes. Find the condition the claim asserts. If they are the same condition, that question is settled, and the only remaining issue is how well the evidence establishes it. If the claim is downstream, look at what connects them, because a separate question applies at each connecting link: what shows that this condition reliably produces the next one, under the conditions operating here and now?

Sometimes the answer is strong, because the pathway is mature and measured. Sometimes the relationship was established in an earlier era and has never met current conditions. Sometimes the upstream number is simply the one that was easiest to produce and announce.

The question takes nothing away from good evidence. It gives the evidence its proper standing.

A good measure should be allowed to say everything it knows.

It should not be asked to certify a condition it did not measure.