Finished: May 4, 2025
Why I read this
A few years ago on a vacation to Greece I ran out of books and out of necessity purchased my first nonfiction/sociology type book with Humankind by Rutger Bregman. Before then I had read very little non-fiction in my life, and this experience opened my eyes to a whole new way to learning. I’m not sure what had clicked in my head, but the ideas presented here were engaging, even more interesting that a fiction book. I could relate to the points and I felt like I was learning a whole new vocabulary for talking about modern social issues, subjects of importance. I went on to tackle Thinking Fast and Slow, Nudge and a slew of Malcolm Gladwell books.
What I learned
Noise is defined as “undesirable variability in judgments of the same problem”, which comes across as a rather simple concept. However, as we dove deeper into the implications of that phrase the reality and omnipresence of noise in our society was astonishing. Everything from hiring decisions (which we are all already aware are flawed), to judicial sentencing (which we would expect to be relatively “fair”) are absolutely plagued by noise and inadequate decision making processes.
Before we can talk about noise we have to talk about bias. I was happy to have a reminder of the common biases we face every day and should be cognizant of when we are making decisions. I had forgotten how many powerful biases we are exposed to in almost any benign interaction. How our methods of communication can pollute our decision making processes, so I wanted to write for myself here some of those key biases as a reminded to avoid them in the future where possible.
- Base Rate Neglect – We systematically fail to ignore the general probabilities of something happening in our decision making.
- Availability bias – Recent examples or information weighs significantly more in our decision making than older information.
- Conclusion bias – Jumping to conclusions, our tendancy to go along with the first plausible conclusion that arrives in our head.
- Anchoring – We are drawn to the number first mentioned in a negotiation or decision.
- Affect heuristic – How we are currently feeling dramatically impacts our decision making processes.
- Confirmation bias – We search for information which confirms our already held beliefs.
- Excessive Coherence – Our brains want so badly to construct a coherent story we have the tendency to connect dots without the required evidence.
If we somehow manage to overcome these biases that already steer our decisions in a consistent, yet dangerous way away from the optimal choices. We are then confronted with noise, which contributes, as much, if not even more, to our decision making processes. There is Level Noise, which includes the general differences between individuals (such as a judge who is systematically more strict than another), and there is Pattern Noise, which is when someone is inconsistent with themselves based on their personal experiences (such as a judge who has a daughter being exceptionally strict on sexual harassment cases), and finally there is even Occasion Noise, where the situation of the day can affect how someone makes decisions (things as light as if they slept well the night before, or if their favorite sports team won their match over the weekend).
All of these factors add together to a significant contribution to error from the expected value of most judgements. Obviously, in the interest of reducing errors (especially for important things such as insurance premiums or prison sentences) fighting these noisy tendencies can be very important. Yet, it is an invisible battle that is little known and rarely combated. Much of the problem lies in the inability in most situations to clearly define the values of Noise, which makes it impossible to fully understand its impacts, or the value added for any effort to combat the Noise. This is a fact that is well acknowledged by the authors who instead focus their efforts on what they call Decision Hygiene. This means concentrating our efforts on the process instead of the solutions, by looking more closely at How we do things instead of what we are trying to accomplish. With that in mind I wanted to concentrate on some of these key methods on how we can make better decisions, and which can be applied to almost any situation.
Checklists: Anyone who knows me will know that I am an enormous proponent of the humble checklist. I do not believe there is any other tool that is more effective at providing bang-for-buck quality and process control. It’s a tool that is universally used and accepted by individuals (grocery lists, to do lists) and organizations (airlines, construction, health) in a way that few other things are. There is a simply reason for this. They consistently work to reduce error because our brains are not strong enough to remember every time what we need to be doing. So once again, the authors of this book (joining James Clear and his Atomic Habits, or Atul Gawunde with his Checklist Manifesto) advise that whenever we want to reduce bias and Noise in any process requiring judgement, checklists are a simple tool which can remind us of the information which is important and which is not.
Rules and Algorithms: The most polarizing idea I found from this book was the idea that simple rules and algorithms are better at making judgements than most people are. Experiment after experiment has shown that almost always in repetitive decision making (hiring decisions, bail decisions, medical choices such as whether to operate or not, even the future value estimation of wines) rules or algorithms produces better results than even the most trained experts in a field. There was a famous study (a study that was performed Princeton Professor named Orley Ashenfelter, which is also included in Thinking Fast and Slow) where a professor made a calculation based on number of sunny days, rainfall, and average temperature for the Bordeaux region of France and based on the results over growing periods and estimated the value the wines were expected to reach several years later. The prices from this calculation were on the whole better than the estimations made by wine critics. Yet, it was extremely controversial that something so inhuman could perform better than experts. In discussing this subject with my sister (a veterinarian) and her husband (an orthopedic surgeon) both of them were extremely resistive to the idea that their expert judgement could be beaten by rules of algorithms. I remember my sister even referencing a kind of Spidey Sense that allowed her to realize what was wrong with her patients since she has gone through so many surgeries. It was exactly in the face of this resistance from experts where the authors of this book brought attention to the superiority of rules. It is precisely these experts who need these rules the most to avoid the draw of the Spidey Sense. That this Spidey Sense may be confirmed because of base rate luck and will lead to overconfidence and possible disaster in future similar, yet different situations. We don’t want doctors cutting around inside of us based on a feeling. We want them following a proven procedure with the highest possible chance for success, which means following more rules, even when it feels robotic.
There are two caveats to this. The first being that experts need to be able to override rules under certain circumstances. Rules are built on probabilities and experience, and such there are certain times where obvious exceptions should be made. The example given by the authors is the probability that someone may go to the movies on any given night. A machine can calculate price of the movies, the population of the town, other attractions, or any number of variables better than a human being can and will almost certainly come up with a probability closer to the reality of the chance that say Tommy will go to the movies tonight. However, this algorithm is going to surely ignore certain facts (it can’t calculate everything). This might mean that it would not consider if Tommy had been in an accident and broken his leg on that day, in which case, almost anyone can agree that it is very unlikely that Tommy will be going to the movies on the same day of his broken leg. So rules are important, yet they should never be so solid that we cannot identify the broken leg situations and escape from rigid and nonsensical results.
The second caveat being that most people will very much dislike this approach simply because it will feel inhuman. Self-driving cars may reduce the number of road accidents as compared to human drivers (on a mile driven per accident basis), but even a single traffic death that results from an emotionless decision or error by an automated vehicle feels unacceptable. In 2022 there were just over 45,000 traffic related deaths in the United States, but these deaths were the result of human error, something which we accept, we even have the phrase that “to err is human”. However, if tomorrow we could replace all the cars in the world with self driving ones and we could reduce that number of 10-20% we should right? It could save each year in the US alone almost 10,000 lives. Yet it would never be accepted because the remaining 35,000 deaths would no longer be human accidents, but failure of a robotic system. They would feel cruel, even intentional, and would never be accepted. The only way we would ever accept this is if the cars were so much better than us that the benefits are undeniable. If self driving cars reached a point where instead of 45,000 traffic related deaths there were 45, then maybe we could accept it. So until our rules become nearly perfect we continue to use human judgement in many fields where formulas could outperform us by a significant margin simply because that margin is not great enough.
Comparison and Scales: Another less than intuitive, but ultimately clear idea was our understanding of scales and how to make better ratings of things. The idea that we use scales in a manner that is inconsistent from one person to another is clear. It drives me infinitely crazy how people will rate restaurants for example. For me a 5 star rating should be extraordinarily rare. If we are doing a comparative judgement whenever someone gives a 5 out of 5 rating for a restaurant it should mean that it is the best, or at least among, the best dining experiences of their lives. Yet on google maps the average rating for restaurants is often between 4 and 5 stars and is very rarely below 3. 3 should be the median so at least half the restaurants in the world should be below that, but it is not even close to the reality. It’s because we all use this scale differently. Some people give a 5 to any restaurant that meets their minimum criteria and only deduct stars for noticeable infractions such as slow service, cold food, or bad quality to price ratio. Some people will give a 1 star review despite excellent food because they believe a waiter was rude to them. Each person using this 1-5 scale in their own way, which makes the overall value of the scale significantly less. This is not such a big deal for restaurants, but it applies equally for much more important scales such as those used for employee evaluations or by doctors to assess pain levels (and therefore decide on treatment plans).
One method to reduce the differences between scales is to try and make scales as comparative as possible. So for our restaurant example you might do away with your stars ranking, and replace it with comparative scales of restaurants in similar categories. In my case I might use a comparative scale for pizza restaurants near my home (there are 5 or 6 different restaurants for pizza within a 10 minute walk of my apartment) where instead of ranking these places numerically and comparing the numbers, I could directly compare the restaurants and have a scale where each restaurant is directly compared to others. Was my experience at La Toscana better than Pizza Cozy? Where does Dominos rank compared to those two? On and on, and in this way my ranking would be much clearer, and much less noisy.
The problem with this method though is similar to that of the self driving cars. When applied to restaurants it is ok, but thinking about the application for employee evaluations where people are directly compared one to another can cause more problems than the accuracy of the system provides benefits. Maybe it is true that a scale of employees from best to worst would be more accurate, but the inhumanity of dedicating someone (even in a high performing team) as the worst would have its own list of shortcomings that would likely be greater than a noisy ranking system.
Beyond this there were several ideas that can be applied to my life (or anyone’s life) about improving our judgements and our methodology, yet I find that I am getting to the point here where I am trying to re-write the entire book, so I will end with a suggestion that anyone who makes important judgements that impact the lives of others on a regular basis should read this book and do their best to apply its principles.
What I didn’t like
There was very little I could criticize in this book, but if there was one place I’d like it to have gone a bit further would be how this subject affects us on a human level. That is to say, in our daily lives, where are we being impacted the most by noise. How does it impact people on dating apps, or how we select a restaurant. The examples in the book were interesting, but I feel there is a fairly small percentage of the population that deals with hiring or judiciary cases every day. If we could have added more examples that broadly apply to everyone it could have aided in making the concept more broadly applicable to everyone.
Questions I asked
Where is noise the most prevalent in my daily life?
“When the facts change, I change my mind. What do you do?” (Quote by John Maynard Keynes)
Will Noise reduction techniques add to much additional process and ultimately reduce our ability to produce?
My Favorite Quote
“Wherever there is judgement, there is noise, and more of it than you think.”
Cass Sunstein, Daniel Kahneman, Olivier Sibony
Books I liked like this one
Thinking Fast and Slow: Daniel Kahneman (for a deeper understanding of how our minds work)
Nudge : Cass R. Sunstein, Richard H. Thaler (for seemingly simple sociology that can make a large difference)
Book Reviews for Mom 