EN
Three different test stations are mounted on one base and connected through blue lines.

Not observed, not applicable, and insufficient evidence do not mean the same thing

What does a space without a conclusion actually mean? It may indicate an absence of observation, an expectation outside the role, or insufficient evidence. Distinguishing these states prevents us from assigning a gap that the data does not demonstrate.

PRYSMAP5 min read

A blank space in an assessment seems easy to interpret. Information is missing, and someone needs to complete the record. But that absence may represent very different situations, with different consequences for the person and for the decision that will use the result.

One expectation may be relevant but still awaiting observation. Another may fall outside the role being assessed. There may also be a record whose quality is insufficient to support the intended conclusion.

If the system treats all three cases as equivalent, it turns a limitation of the data into a statement about performance. The assessment acquires a certainty that no one has demonstrated.

First review applicability

Before asking what evidence is missing, clarify whether the expectation actually belongs to the work. That relationship depends on the role, level, responsibilities, and decision the assessment is intended to support.

A person at an entry level may participate in a contingency without having the authority to approve a decision reserved for another level. Coordination during the contingency may be relevant. Final approval is outside their scope.

Confusing these cases distorts the result. A situation that has not yet occurred needs an opportunity for observation. A responsibility that does not belong to the role should be excluded with an explicit rationale.

After reviewing the scope, a second question arises: was there an appropriate situation in which to observe the expectation? If so, we still need to determine whether the information gathered is sufficient for the intended use.

The order matters because applicability, observation, and sufficiency describe different problems.

Three states that should not become the same number

State What it tells us Valid action Interpretation to avoid
Not observed The expectation applies, but an appropriate observation is still missing. Identify a suitable situation or source and leave the judgment open. Turning the absence of observation into poor performance.
Not applicable The situation falls outside the assessed purpose, role, level, or context. Document the exclusion criterion and remove the case from the relevant calculation. Using it to conceal an expectation that does belong to the work.
Insufficient evidence Information exists, but its quality, recency, context, or consistency does not support the intended conclusion. Specify what is missing and which evidence would resolve it. Presenting a partial observation as a confirmed gap.
A metal disk, ring, and triangle are arranged as different samples on a laboratory bench.

The difference can appear within a single assessment. A real contingency has not yet occurred even though the expectation is valid for the role: it remains pending observation. Authorization for a higher-level decision belongs to another scope and should be excluded. An isolated comment about an intervention in a critical situation provides information, but it may be insufficient for a promotion decision.

The person being assessed is the same. What changes is the relationship among the expectation, the available evidence, and the use of the result.

The decision defines how much evidence is needed

An observation may be useful for guiding a development conversation and still be insufficient for deciding on a change in level. The record does not change; the demands of the decision do.

The AERA, APA, and NCME standards connect validity with specific interpretations and defined uses. An assessment does not acquire validity in the abstract: it needs evidence that supports the specific conclusion someone intends to use. AERA, APA, and NCME: Standards for Educational and Psychological Testing.

This precision prevents two common mistakes. The first is treating an incomplete observation as proof of poor performance. The second occurs when an exclusion is used to conceal an expectation that does belong to the work.

What must survive when the data reaches a dashboard

The distinction loses its value if it disappears when results are consolidated. A form may correctly record that a situation was not observed yet still send a zero to a report. Another tool may remove an excluded expectation without explaining why it no longer participates in the calculation.

In both cases, the person reviewing the result receives less information than needed. A pending data point may look like a gap. An exclusion may make comparisons artificially favorable. A partial observation may end up influencing a promotion as if it represented sufficient evidence.

A defensible architecture retains the affected expectation, the reason, the date, and the next action. It also distinguishes which elements contribute to a result, which remain pending, and which were excluded on a sound basis.

The person being assessed needs to understand that difference and provide context when appropriate. A leader must also recognize when the system is describing observed performance and when it is only displaying a question that remains open.

The quality of an assessment depends in part on keeping that boundary visible. A blank space may represent a pending observation, a relevant exclusion, or insufficient evidence. None of those states authorizes us to invent a conclusion. Nor should it hide that the person may not have had a comparable opportunity to produce evidence.

To extend this reading, see Confidence is not competence, which develops a complementary dimension of the problem.