
Assessing the work people already do with AI
Assessing AI-assisted work requires observing decisions, not just deliverables. The assessment must justify which assistance it retains and which responsibility the evidence supports assigning.
Allowing AI in an assessment and then removing it to check whether the person “really knows” may seem like a reasonable precaution. Sometimes it will be. But the two situations do not necessarily answer the same question. One observes how someone works with the resources available; the other, what they can do without one of them. To interpret the difference, we need to know which of those capabilities the role requires, under what circumstances and for which decisions.
If everyday work includes AI assistance, using it can be part of a relevant performance sample. It may also be necessary to check what happens when the tool is unavailable or produces an answer that should not be trusted. What we should avoid is automatically ranking the two assessments, as though removing technology always brought us closer to a truer capability, while working with it showed only how well we can conceal a gap.
The SIOP principles for personnel selection (opens in a new tab), published in 2018, connect content-based evidence to an adequate sample of the work and its demands. They were not written as an AI assessment protocol. They nevertheless retain a useful requirement for designing one: an assessment’s difficulty does not substitute for a justification of what it allows us to observe. Before deciding which tool to remove, we must be able to explain which part of the work would be better represented by doing so.
Assistance also changes what we can conclude
An improvement achieved with AI is a result worth recognizing, even if we cannot yet attribute every part of it. In the experiment by Dell’Acqua and his colleagues with Procter & Gamble professionals (opens in a new tab), individuals working with AI achieved performance comparable to that of teams without it on the innovation challenges studied. It would be strange to assess that work while ignoring a contribution that made it possible. It would also be excessive to conclude that each participant had thereby acquired the expertise of another specialty, or would respond the same way to any problem.
The difficulty arises when a good deliverable is used to assign a broader responsibility. Preparing a recommendation, checking its foundations and deciding whether to apply it may all come together in a convincing document. If the role needs all three contributions, assessing only the presentation of that document leaves part of the expectation unobserved. A subsequent explanation helps reconstruct the reasoning, but need not be sufficient: it may be correct for the case already solved without showing what the person would do if a decisive condition changed.
The uneven nature of assistance matters here. The experiment with Boston Consulting Group consultants published in 2026 (opens in a new tab) found improvements with GPT-4 on selected tasks within its capabilities, but lower accuracy on a problem outside that frontier. This is evidence about those tasks and that technology, not a map of the limits of current models. For assessment, it poses a concrete difficulty: a sample concentrated on cases where assistance works well may leave unobserved how someone responds when it stops working.
Observing a decision, not just its explanation
An assessment without AI would have a different place when continuity of work requires acting without it. That situation would have to be represented: which resources would remain available, which decisions could not wait and what assistance it would be legitimate to request. This does not mean manually reproducing the entire automated process. Knowing how to stop it, preserve essential information or transfer the case to someone with the authority to resolve it may be enough. Demanding absolute independence where the work requires coordination would also distort the sample.
The conclusion must retain its conditions
Comparing results requires knowing what assistance was available. Working with a familiar tool is not equivalent to working with one just introduced, or with a configuration offering different kinds of support. The instructions, resources and familiarization time need to relate to what is being assessed. Recording those conditions helps interpret differences; it neither automatically eliminates comparison problems nor requires every private interaction to be retained as evidence.
The result should support a more precise statement than “knows how to use AI.” A person was able to produce an adequate solution with certain assistance and, in addition, review their decision when given contradictory information. That provides a different basis from having obtained only the first solution. Neither observation, by itself, warrants a conclusion about how they will act in any domain, whether they can train others or whether they are ready to take on responsibility with greater consequences.
The article on developing judgment under automation asked how that capability is renewed. Here, the assessor’s obligation is more immediate: not to attribute it before gathering relevant evidence. If the organization expects someone to supervise assisted work, that expectation must appear in the situation it observes and in the basis for the decision it makes afterward. Otherwise, it may demand a responsibility the assessment never examined.
An assessment does not have to penalize the use of a tool to be rigorous. It has to let us explain what we observed in the person and why that is enough for the responsibility we are going to assign. We should not approve someone to answer for work when all we have established is that they can obtain an answer.





