When measuring more does not mean deciding better

The problem begins when one system combines signals created for different purposes. Some help choose among alternatives. Others explain the context, open a line of inquiry, or show what happened afterward.

Volume neither resolves that hierarchy nor destroys it by definition. A complex decision may require many signals; a simple one may depend on a single signal. What matters is the relationship between the available information and the use that warrants our attention.

The starting point already contains a decision

Many measurement systems grow from availability. A platform can capture a data point, the data seems relevant, and it finds its way into a report. New questions, internal requests, and technical capabilities emerge later. The collection grows without necessarily changing its architecture.

The reverse path starts with a different definition: what choice, review, or correction needs support? That question narrows the purpose without demanding an immediate answer. It also reveals whether the organization is trying to decide, understand a situation more fully, or reduce uncertainty before acting.

The difference has practical consequences. If the objective is to make a development assignment, role expectations, performance evidence, and available alternatives matter. If the objective is to explore why a signal changed, contextual data will be needed—but that data does not yet justify an intervention. Both uses are legitimate. Combining them can make diagnostic information look like decisive evidence.

The OECD performance budgeting framework (opens in a new tab) addresses a different domain but presents a relevant tension. It recommends limiting objectives and indicators to avoid overload while warning that a single measure may be insufficient. Selection should favor information that is relevant and useful for decision-making. Applied cautiously to talent, the principle is not to measure less. It is to justify why each measurement remains visible.

Four functions that often share the same dashboard

A signal can perform at least four functions within a system:

• Decide: help choose, prioritize, maintain, or correct an alternative.
• Interpret: provide context for understanding a primary signal and its limits.
• Explore: support investigation of an unresolved uncertainty.
• Review: show what happened afterward and support learning from a decision.

These functions can coexist, but they are not equivalent. An exploratory signal is not useless merely because it does not yet trigger a response. Contextual data should not dominate a decision simply because it is easy to update. The problem arises when the design makes these functions indistinguishable.

In a skills system, a primary signal might show the distance from a defined expectation. Other signals could help examine the quality of the evidence, the context in which the skill was applied, or the stability of the result. A completed activity may help track an intervention. By itself, it does not prove development. Presenting these signals as equivalent confuses monitoring with observed change.

Rebuild measurement around its use

Designing from the decision means working through six dependencies. First, specify what must be resolved. Second, identify who has the authority to resolve it. Without an accountable decision-maker, the information may start a conversation, but there is no decision path yet.

Next comes the required signal. Its definition must correspond to the conclusion being sought. In a skills assessment, a score indicates a difference from a reference point. It does not automatically explain the cause, priority, stability, or appropriate response. Those inferences require additional evidence and explicit rules.

The fourth dependency is a material change. A variation may be visible in a report without altering any of the alternatives. Materiality depends on the expectation, measurement uncertainty, possible consequences, and the cost of being wrong.

The fifth dependency is an available response. If a result does not affect a choice, priority, investigation, or review, its function needs clarification. This does not necessarily mean removing it. The result may belong at another level of the system or serve another audience.

The final dependency is review. Every consequential decision rests on assumptions: what change we expected, over what period, and what evidence would allow us to recognize it. Without this layer, the system records the starting point but cannot reveal whether the interpretation proved sound.

The complete sequence looks like this:

decision → accountable owner → signal → material change → response → review

This is not a formula for automating judgment. It is a way to locate where the connection between measurement and use breaks down.

The audience changes a signal’s value

The same data can serve different functions depending on who receives it. Individuals need to understand what evidence supports their result and what they can do with it. A manager needs to distinguish a confirmed gap from an area with insufficient observation. Human Resources may need to identify patterns, coverage risks, or inconsistencies in the system.

An identical visualization does not produce a shared interpretation. Each audience has different responsibilities, alternatives, and needs for detail. When a report tries to serve everyone without setting priorities, it tends to add information and postpone selection until the moment of decision.

This hierarchy also guards against a common assumption: that whatever can be quantified deserves more attention. A precise number may rest on weak evidence. A well-situated qualitative observation may explain a limitation the indicator cannot capture.

Keep, demote, or retire

Reviewing a measurement system is not a search for the shortest possible dashboard. It is an exercise in deciding where each signal belongs.

Some signals should remain primary because they change real alternatives. Others should move to a diagnostic layer, available for interpretation or investigation. Some may be useful only for a period while a hypothesis is tested or a transition observed. There may also be legacy indicators whose function no one can reconstruct.

The most useful review does not begin by counting indicators. It begins by asking what function each one performs, who needs to see it, and what would happen if it changed materially. When those answers are unavailable, the organization does not necessarily have a data problem. It has an incomplete relationship between information and decision.

Measurement designed around use leaves room to decide, interpret, explore, and review. Improvement does not come from making lower volume an end in itself. It comes from recognizing the work each signal must do and moving anything unrelated to the current decision out of the foreground.