There is something seductive about a number.
A score. A ranking. A rating out of ten. The moment evaluation gets quantified, it feels clean. Defensible. Like the conversation about who deserves what has been moved out of the realm of opinion and into the realm of fact.
This feeling is almost always wrong.
Every rubric was written by someone. Every criterion reflects a decision about what matters and what doesn’t. Every benchmark was calibrated against a standard that someone, at some point, decided was the standard. The number at the end of the process carries all of that history inside it, quietly, invisibly, wearing the costume of neutrality.
The myth of the neutral evaluator is not a fringe concern. It sits at the center of how institutions justify their decisions, how organizations defend their hiring, how schools measure intelligence, how governments allocate resources. And the most dangerous version of the myth is not the one that claims to be objective. It’s the one that genuinely believes it.
Objectivity as a Social Construction With a Paper Trail
The idea that evaluation can be fully separated from the evaluator is a relatively recent invention, and a culturally specific one. It emerged from Enlightenment-era faith in reason as a universal solvent; the belief that if you stripped away enough subjectivity, you would arrive at something true and transferable. The scientific method, the standardized test, the structured interview, the performance review rubric: all of them are descendants of this idea.
What the history of these tools reveals is not their neutrality but their authorship. The early standardized intelligence tests developed in the United States in the early twentieth century were designed explicitly with particular populations in mind. The criteria used to evaluate fitness for military service, for immigration, for educational advancement, were not plucked from the air. They reflected the assumptions, anxieties, and hierarchies of the people who built them. The scores looked objective. The design was anything but.
This is not ancient history dressed up as a lesson. It is a structural feature of how evaluation systems are built. A rubric requires someone to decide which qualities are worth measuring. A rating scale requires someone to define what a five looks like versus a three. A benchmark requires someone to choose which performance counts as the reference point. At every stage, a human judgment is embedded in what will later be presented as a human-independent result.
The philosopher Sandra Harding called this “strong objectivity”: the idea that truly rigorous inquiry requires examining the values that shaped the inquiry itself, not just the data it produced. Most evaluation systems skip that step entirely. They begin after the value-laden decisions have already been made, and they treat the structure they inherited as neutral ground.
It isn’t.
The Hiring Process as a Case Study in Structured Subjectivity
Nowhere is the myth of the neutral evaluator more consequential, or more consistently misunderstood, than in hiring.
The modern hiring process has built an impressive architecture of apparent objectivity. Job descriptions with precise competency lists. Structured interviews with scored questions. Assessment centers with standardized tasks. Applicant tracking systems that filter resumes before a human ever sees them. The machinery of meritocracy, assembled and operational.
And yet the outcomes of this machinery consistently reflect the biases of the people who built it. Audit studies, where identical resumes are submitted with names that signal different racial or gender identities, show persistent gaps in callback rates that the formal structure of the process does nothing to eliminate. The algorithm that screens resumes was trained on historical hiring data, which reflects historical hiring preferences, which reflected the biases of previous hiring managers. The structured interview questions were designed by people who had a mental model of the ideal candidate, a model shaped by who previously succeeded in that role, which reflects who was previously given the chance to succeed in that role.
This is not a failure of the system. It is the system working exactly as designed; it is just that the design was never as neutral as advertised.
The psychologist Nathan Kuncel, whose research spans decades of personnel selection, has consistently found that human judgment in hiring is both overconfident and systematically skewed. People believe they can read a candidate in an interview. They cannot, not reliably. But the confidence with which they believe it is itself a product of the myth: the idea that experienced evaluators develop accurate intuition, rather than refined bias.
The most telling detail is this: organizations that add structure to their hiring processes often do so because they believe structure removes subjectivity. In practice, it often just formalizes it.
When the Rubric Becomes the Reality
There is a particular moment in the life of an evaluation system when something quietly catastrophic happens: the measure becomes the thing being measured.
The economist Charles Goodhart observed that when a measure becomes a target, it ceases to be a good measure. What he identified as an economic principle has metastasized across every domain where assessment systems operate. Schools optimize for test scores rather than learning. Employees optimize for metrics rather than performance. Researchers optimize for publication counts rather than contribution. The rubric, designed to capture something real, gradually replaces the real thing it was supposed to capture.
This substitution is invisible from inside the system. People filling out the rubric believe they are measuring performance. They are measuring performance-as-defined-by-the-rubric, which is a different object entirely. The gap between the two is where bias lives most comfortably, because no one is looking there. They’re looking at the scores.
The cultural theorist Marilyn Strathern, building on Goodhart’s work, noted that audit cultures, organizations and institutions built around measurement and accountability, tend to produce the behaviors that satisfy the audit rather than the underlying purposes the audit was designed to serve. The assessment apparatus becomes self-referential. It measures its own assumptions.
This is how institutions can produce clean numbers that mask profound dysfunction. The performance review scores are consistently high while morale collapses. The standardized test results improve while actual comprehension does not. The diversity metrics tick upward while the culture remains unchanged. The system is working. What it is working toward is the question nobody thinks to ask.
The Most Biased Systems Are the Ones That Don’t Know They Are
There is a meaningful difference between a system that acknowledges its subjectivity and one that does not.
A hiring manager who says “this is my judgment, and my judgment has limits” creates at least the possibility of accountability. A hiring process that says “our scoring system is objective” closes that possibility down entirely. The claim of objectivity is not just inaccurate. It is actively protective of whatever biases the system contains, because it pre-answers any challenge with a credential.
This is the specific danger of the neutral evaluator myth: it converts value-laden decisions into apparently technical ones, and technical decisions are much harder to contest. You can argue with a person’s opinion. It is far more difficult to argue with a score, a ranking, or an algorithm’s output, because the machinery of the process has already absorbed the contestable parts and spat out something that looks like fact.
Safiya Umoja Noble’s examination of algorithmic bias demonstrated precisely this: that automated systems encode the assumptions of their designers in a format that obscures those assumptions from view. The algorithm doesn’t have opinions. Except that it was trained on data produced by people who did. And those opinions are now legible only to those who know to look for them.
The institutions most resistant to examining their evaluation systems are almost always the ones most invested in the myth of their own objectivity. The belief that you’ve solved the subjectivity problem is, functionally, a belief that you no longer need to look for it.
What “Structured” Actually Protects
There is a version of the structured evaluation argument that is worth taking seriously, not because structure eliminates bias but because, used carefully, it can make bias more visible and therefore more contestable.
The value of a rubric is not that it removes the evaluator’s judgment. It is that it makes that judgment legible. A structured process forces explicit articulation of what is being valued and why. It creates a record. It enables comparison across evaluators. And it provides a surface against which inconsistency can be identified: if the same behavior is scored differently depending on who is performing it, that gap is now visible in a way it would not be if evaluation happened entirely inside someone’s head.
This is a limited and conditional benefit. It requires that someone is actually looking at the gaps. It requires that the organization treats inconsistency as a problem to investigate rather than noise to ignore. And it requires a willingness to trace the inconsistency back to its source, which means being willing to examine the assumptions embedded in the rubric itself.
Most organizations are not willing to do this. The rubric is the answer. The rubric is not supposed to be a question.
But the evaluation systems that earn genuine trust, not the performance of trust, are the ones built by people who understand that every criterion is a choice, every benchmark is an argument, and every score is a compressed version of someone’s values. The question is not whether values are present. They are always present. The question is whether anyone is honest enough to say whose.
