Unskilled and Unaware: Why incompetent are confident and the best lack conviction.

If you’ve ever come across someone whose performance is grossly inadequate but they’re confident that their skills are reasonable or even excellent, you likely witnessed the Dunning-Kruger Effect in action.

Psychologists David Dunning and Justin Kruger coined the eponymous Dunning- Kruger effect, a cognitive bias whereby people who are incompetent at something cannot recognize their incompetence. Some researchers include in their definition also the opposite effect for high performers: their tendency to underestimate their skills.

The scientific study conducted in 1999 put data to thought what philosophers, scientists, poets, and writers have propounded since Socrates.

” The only true wisdom is knowing you know nothing.”

Socrates

In ‘The Descent of Man’ in 1871,   

“Ignorance more frequently begets confidence than does knowledge.”

-Charles Darwin

The poem “The Second Coming” in 1920 presented the Dunning Kruger effect with the lines.

“The best lack all conviction, while the worst are full of passionate intensity.”

-W. B. Yeats

In 1933, an essay titled “The Triumph of Stupidity” that lamented the rise of the Nazi movement in Germany.

“The fundamental cause of the trouble is that in the modern world, the stupid are cocksure while the intelligent are full of doubt.”

-Bertrand Russell

In an interview about writers with the literary journal “Arete” in 1989

“But the problem is that bad writers tend to have the self-confidence, while the good ones tend to have self-doubt.”

-Charles Bukowski

Mount Stupid and Valley of Despair

A Dunning Kruger effect curve is often attributed and is depicted with a peak, fittingly named “Mount Stupid,” signifying that the people who are the most confident about their knowledge are the ones who have little to no actual knowledge about the given topic. The drop from the peak symbolizes the fall in confidence into the valley of despair once one learns more about a topic and realizes how much one doesn’t know. This point is similar to experiencing ‘imposter syndrome’ in which an individual doubts their skills, talents, or accomplishments and has a persistent  internalized  fear of being exposed as a “fraud”. Further learning improves confidence, yet one can never match the peak of Mount Stupid again.

Water, water everywhere

One can find instances of the Dunning-Kruger Effect playing out everywhere. An article in Forbes magazine reported a study in which 32-42% of software engineers in high tech firms rated themselves as being in the top 5% of their companies. In another study cited, 68 % of faculty at the University of Nebraska ranked themselves in the top quartile for teaching ability, and over 90% rated themselves above average. In the USA, a nationwide survey cited 21% of people believing that they’ll likely become millionaires within the next 10 years. People tend to overestimate their expertise in grammar, financial knowledge, math, emotional intelligence, running medical lab tests, driving, chess, and many social and intellectual domains. 

The irony of the Dunning-Kruger Effect, as Professor Dunning notes, is that people who are unskilled in these domains suffer a dual burden.

 “Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it.”  

In the original study, the authors tested the participants on humor, grammar, and logic. They found that those scoring in the bottom quartile on tests largely overestimated their test performance and ability. Those with test scores in the 12th percentile estimated themselves to be in the 62nd percentile. 

The knowledge and intelligence required to be good at a task are often the same needed to evaluate one’s performance. So, if one lacks such knowledge and intelligence, they will remain ignorant of their true proficiency. The authors cite an example of the ability to write grammatical English. 

“The skills that enable one to construct a grammatical sentence are the same skills necessary to recognize a grammatical sentence, and thus are the same skills necessary to determine if a grammatical mistake has been made. In short, the same knowledge that underlies the ability to produce correct judgment is also the knowledge that underlies the ability to recognize correct judgment. To lack the former is to be deficient in the latter.”

In many organizations, employees are underperforming simply because they are unaware that they could be doing better or what excellent performance looks like. After training to improve their skills, the employees could differentiate their previous poor performance and their new, improved performance and actually reprehended their previous poor skills.

The Real Dunning Kruger effect curve

Like many pop-culture expressions, the truth has been stretched in the widely prevalent versions of Dunning Kruger effect findings. For instance, the Dunning Kruger effect curve discussed earlier is not the original curve that appeared in the Dunning Kruger study. The original study was aptly titled, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments” and the graph in the study showed how much the actual test score differed from the students’ perceived ability. 

The x-axis displayed the students divided into four quartiles according to how well they performed in the test. The y-axis shows at which percentile a given test score stands in relation to all the other test scores.  The students in the lowest-scoring quartile grossly overestimated their ability relative to their peers. They thought they had scored better than average, around 62nd percentile, when in fact, their performance fell in the 12th percentile.

But there is another interesting aspect in this graph. While the low-scoring students overestimated themselves, the best-scoring students underestimated their performance.   Kruger and Dunning attributed this to the false-consensus effect. That means that top-quartile participants evaluated their performance correctly, but because they performed so adeptly, they assumed the same was true of their peers.

Pot calling the kettle black

The effect is also not about stupid people but about incompetence, the absence of knowledge about a given topic. The authors explicitly believe incompetence is a matter of degree and not one of the absolutes. 

“There is no categorical bright line that separates competent individuals from incompetent ones.

Thus, when we speak of incompetent individuals, we mean people who are less competent than their peers.”

And since all of us are incompetent in one area or another, the odds are that we overestimate ourselves, too. We all, a few times, exhibit some degree of Dunning-Kruger effect.

It’s not easy to precisely guess the boundaries of our knowledge, skill, and competence. We often walk off the solid ground of what we know into thin air of things we don’t know. Knowing how competent we are and how our skills stack up against others is more than a component of self-awareness and confidence. It aids us in determining when we can forge ahead on our own decisions and instincts and when we need to seek out advice instead.

Finding your blindspots

How can we identify whether we are dithering and cruising in Dunning Kruger zone? What can we do to find out how good we are at various skills?

First, seek feedback, and consider it, even if it’s hard to hear.

Second, and more important, keep learning

The more knowledgeable we become, the fewer gaps we will have and will be less likely to incorrectly estimate our knowledge, skills, and competence.

“Real knowledge is to know the extent of one’s ignorance.”

-Confucius

References: