“Fully many a flower is born to blush unseen” Thomas Gray
Metrics provide tangible proof of performance while merit provides a moral sense of deserving one’s success. Both elements underpin much of our modern society, but both metrics and merit can be flawed ideals, as two books point out - The Tyranny of Metrics by historian Jerry Muller and The Tyranny of Merit by philosopher Michael Sandel.
Metrics and merit might both seem to embody de facto good. Yet, as both authors explain, they can be wielded in ways that are cruel and oppressive. Both metrics and merit are also enmeshed in the socio-technical system that increasingly uses data-driven AI systems. We optimize metrics that can result in inequalities that are tearing apart our social fabric. Related to this failure is the way in which meritocratic values have led the winners in this system - those with the best metrics - to look down on those who don’t seem to measure up.
Two Tyrants
In The Tyranny of Metrics, Muller’s central thesis that we have over used quantification and have inappropriately applied it to areas that are not quantifiable. In doing so, we have either used inappropriate proxies as measurement or have taken an extremely reductionist approach. In either case, we’ve misapplied quantification. This is especially true of complex social problems and has been exacerbated by technologies which promise to use data-driven approaches to solve these problems.
Sandel’s central point in The Tyranny of Merit is that while we are attracted to the idea of being masters of our own making, of a meritocratic society, that ideal is flawed. There are many factors outside of our control that go into our success. These might be circumstances that provide lucky breaks - everything from our genetic makeup to market forces that privilege certain people or skills. Sandel cites LeBron James as an example of a combination of genetics, talent, hard work and the good fortune of being born in a society that loves basketball. It’s this bigger combination of factors many of which reside outside of individual choice that lead to success. Market value (how much money we earn) is used to measure success and attributed to merit. Here, Sandel points to a similar issue as Muller, where the metric (money) is a proxy for a deeper value (merit). In this case, our market value is equated to our worth as human beings.
Undermining the common good, upholding inequality
Metrics are how we uphold the idea of merit. Measurements provide the score that is used to decide who or what is worthy. For example, earning more money, having a socially prestigious job, scoring well on an exam or having good health indicators, can be cited as demonstrable evidence or proof that a person had made good choices. Thus, this person with high scoring metrics is a person deserving of success while someone with lesser scores is not.
Yet, what can be measured and accounted for in a metric is often reductionist or flawed, as Muller notes in The Tyranny of Metrics. It doesn’t take into account many contextual factors that might have bearing on an outcome but instead get stripped away. We can see this in metrics like exam scores, which don’t account for cultural biases in the exam, for the impacts of home environments (shelter, food or safety) on students, or the lack of access to resources to help prepare for an exam. If we use the resulting metric or outcome of the test score as input data for the basis of making further predictions, as we do with AI systems, we encode and amplify these inequities.
But, as Sandel points out, the situation is even worse because we feel morally justified in doing this. Our reverence for merit leads us to see the good outcomes as going to the most deserving people. As a society that upholds merit, we believe that those with good outcomes obviously made good choices while those with poor outcomes made poor choices and are therefore less deserving people. Instead of seeing how random factors based on luck or chance play into success, the ideology of merit upholds inequality by making it morally justified.
Increasingly, the use of automation has enabled a scaling up of this type of unfair power and is enmeshed in the deep inequalities we continue to experience in society. These ideas were echoed in a recent CBC documentary, The Science of Success, in which big data and network analysis were applied in order to try and unpack the scientific evidence that drives success as measured in terms of market popularity and wealth. In a study of artists, it was found that if artists received early showings at a prestigious gallery as their entry into the network, scientists could predict their success based on this factor. This entry into the “right” network was not necessarily an indication of hard work or talent, so much as having good fortune.
How might we make systems driven by privilege more equal while also exposing the myth of meritocracy? One idea floated in this talk by Sandel is to use a lottery amongst otherwise well qualified students as a means to determine admission to Ivy League schools. However, as another panelist points out - that wouldn’t be very good for endowments.
By Katrina Ingram, CEO, Ethically Aligned AI
________
Sign up for our newsletter to have new blog posts and other updates delivered to you each month!
Ethically Aligned AI is a social enterprise aimed at helping organizations make better choices about designing and deploying technology. Find out more at ethicallyalignedai.com
© 2022 Ethically Aligned AI Inc. All right reserved.