Book Review

Title: The Knowledge Illusion: Why We Never Think Alone
Author: Steven Sloman and Philip Fernbach
Scope: 4.5 stars
Readability: 4.5 stars
My personal rating: 5 stars
See more on my book rating system.

If you enjoy this summary, please support the author by buying the book.

Topic of Book

The role of knowledge, both personal and collective, in society.

Key Take-aways

• Individuals believe that they know far more knowledge than they actually do. Ignorance is the natural state of human beings.
• Individuals rarely understand the technologies they use every day, but their brain maintains the illusion that they do.
• Individuals rely far more on the collective knowledge than their own personal knowledge.
• Our ignorance and division of cognitive labor is critical to our survival, because it allows us to act on limited information. But it becomes dangerous when people surround themselves with others who agree with each other. This is most common in political ideologies.

Important Quotes from Book

“The human mind is not like a desktop computer, designed to hold reams of information. The mind is a flexible problem solver that evolved to extract only the most useful information to guide decisions in new situations. As a consequence, individuals store very little detailed information about the world in their heads. In that sense, people are like bees and society a beehive: Our intelligence resides not in individual brains but in the collective mind. To function, individuals rely not only on knowledge stored within our skulls but also on knowledge stored elsewhere: in our bodies, in the environment, and especially in other people. When you put it all together, human thought is incredibly impressive. But it is a product of a community, not of any individual alone.”

“Most things are complicated, even things that seem simple.”

 “Take a minute and try to explain what happens when you flush a toilet.”

“Our point is not that people are ignorant. It’s that people are more ignorant than they think they are. We all suffer, to a greater or lesser extent, from an illusion of understanding, an illusion that we understand how things work when in fact our understanding is meager.”

“We can’t possibly understand everything, and the sane among us don’t even try. We rely on abstract knowledge, vague and unanalyzed.”

“Thought is masterful at extracting only what it needs and filtering out everything else.”

 “The secret to our success is that we live in a world in which knowledge is all around us. It is in the things we make, in our bodies and workspaces, and in other people. We live in a community of knowledge.

We have access to huge amounts of knowledge that sit in other people’s heads: We have our friends and family who each have their little domains of expertise.”

“Our minds need to be designed to treat information that resides in the external environment as continuous with the information that resides in our brains. Human beings sometimes underestimate how much they don’t know, but we do remarkably well overall. That we do is one of evolution’s greatest achievements.”

“We live under the knowledge illusion because we fail to draw an accurate line between what is inside and outside our heads. And we fail because there is no sharp line. So we frequently don’t know what we don’t know.”

“The answer is that we do so by living a lie. We ignore complexity by overestimating how much we know about how things work, by living life in the belief that we know how things work even when we don’t… We tolerate complexity by failing to recognize it. That’s the illusion of understanding.”

“As soon as animals developed neurons and nervous systems, the complexity of their actions exploded and developed at a remarkable rate. This happened because the neuron is the building block of a flexible system that evolution can use to program more and more complex information-processing algorithms.”

“Being smart is all about having the ability to extract deeper, more abstract information from the flood of data that comes into our senses”

 “Remembering everything gets in the way of focusing on the deeper principles that allow us to recognize how a new situation resembles past situations and what kinds of actions will be effective.”

“the mind evolved to support our ability to act effectively.. This is critical to understanding the knowledge illusion: Storing details is often unnecessary to act effectively”

 “the purpose of thinking is to choose the most effective action given the current situation. That requires discerning the deep properties that are constant across situations. What sets humans apart is our skill at figuring out what those deep, invariant properties are.”

 “We manage to overcome the weakness and error inherent in our intuitive causal models by deliberating in step with our community. By doing so, we create an exceptionally powerful social mind.”

“Intuition gives us a simplified, coarse, and usually good enough analysis, and this gives us the illusion that we know a fair amount. But when we deliberate, we come to appreciate how complex things actually are, and this reveals to us how little we actually know.”

 “People and bees have an important property in common: We both harness the power of multiple entities working together to generate massive intelligence.

“What made all this possible was the division of cognitive labor. Each community member mastered a skill that contributed to achieving the community’s goals… spear wielding, butchery, making fire. There’s an explosive gain in efficiency and power when cognitive labor is divided.”

“These examples illustrate one of the key properties of the mind: It did not evolve in the context of individuals sitting alone solving problems. It evolved in the context of group collaboration, and our thinking evolved interdependently, to operate in conjunction with the thinking of others.”

 “The ability to share intentionality supports perhaps the most important human capability of all: the ability to store and transmit knowledge from one generation to the next. This leads to what anthropologists call cumulative culture. The transmission of knowledge enabled by our social brains via language, cooperation, and the division of labor accumulates to create a culture. It is one of the most important ingredients in the human success story.”

“People tend to remember what they have to within a particular community to best make their contribution to the division of cognitive labor. We rely on experts to remember everything else.”

 “A lot of human understanding consists simply of awareness that the knowledge is out there. Sophisticated understanding usually consists of knowing where to find it”

 “As a rule, strong feelings about issues do not emerge from deep understanding. They often emerge in the absence of understanding”

“In general, we don’t appreciate how little we know; the tiniest bit of knowledge makes us feel like experts. Once we feel like an expert, we start talking like an expert. And it turns out that the people we talk to don’t know much, either. So relative to them, we are experts. That enhances our feeling of expertise.

This is how a community of knowledge can become dangerous… When group members don’t know much but share a position, members of the group can reinforce one another’s sense of understanding, leading everyone to feel like their position is justified and their mission is clear, even when there is no real expertise to give it solid support. Everyone sees everyone else as justifying their view so that opinion rests on a mirage.”

“The social psychologist Irving Janis labeled this phenomenon groupthink.”

“What makes the problem worse is that we’re often unaware that we are inside a house of mirrors, and this insularity makes us even more ignorant. We fail to appreciate the other side’s perspective”

“The feeling that overwhelms us is “if only they understood.” If only they understood how much we care, how open we are, and how our ideas could help, they would see things our way. But here’s the rub: While it’s true that your opponents don’t understand the problem in all its subtlety and complexity, neither do you.”

 “Proponents of political positions often cast policies that most people see as consequentialist in values-based terms in order to hide their ignorance, prevent moderation of opinion, and block compromise.”

“It’s an old politician’s ploy. The secret that people who are practiced in the art of persuasion have learned over millennia is that when an attitude is based on a sacred value, consequences don’t matter.”

“We have seen that a good way to reduce people’s extremism and increase their intellectual humility is to ask them for an explanation of how a policy works. Unfortunately, the procedure does have a cost. Exposing people’s illusions can upset them… “Frequently, they no longer want to discuss the issue (and indeed, often they no longer want to talk to us).

We had hoped that shattering the illusion of understanding would make people more curious and more open to new information about the topic at hand. This is not what we have found. If anything, people are less inclined to seek new information after finding out that they were wrong”

“The upshot is that an effective group doesn’t need a lot of people with high g scores; it needs a balance of people with different skills. Whatever the task at hand, be it hunting for food, building a home, or navigating a ship, it’s going to have different components that require different skills. Performance will be best whenever you have a team that has the full panoply of skills required to do the task.”

“Everyone’s understanding—that of scientists and nonscientists alike—is dependent on what others know, so it is more important for students to understand what is known and what can be justified by others than to know the facts and the justifications themselves.”

“For humans, ignorance is inevitable: It’s our natural state. There’s too much complexity in the world for any individual to master. Ignorance can be frustrating, but the problem is not ignorance per se. It’s the trouble we get into by not recognizing it.”

“we’re not very good at knowing what we don’t know.”

If you would like to learn more about collective intelligence in history, read my book From Poverty to Progress.