Detecting and neutralizing misinformation, a couple of book reviews, and other tidbits that crossed my desk this week: A Caterpickles miscellany*

By Shala Howell, March 6, 2021

*Why yes, I did write this title just so that I could use the word miscellany. Someone turned in her high school course requests last week, so clearly it’s time to break out the SAT words.

NOTE: This post contains affiliate links to Bookshop.org, an online bookstore that provides financial support to local, independent bookstores. At the time I wrote this post, Bookshop.org had already raised $12.2m for local bookstores. If you use the links in this post to purchase a book or two on Bookshop.org, I’ll earn a commission on your book purchase, as will your preferred independent bookshop. You can also find many of these books in the new Caterpickles Bookstore. Regardless of whether you use my links or visit the Caterpickles Bookstore, I’m glad you spent part of your day reading Caterpickles. Learn more about Affiliate Links, the Caterpickles Bookstore, and why I decided to become a Bookshop.org Affiliate.

Over the past few weeks, more than 300 people have been arrested for their role in the Capitol insurrection on January 6. Now CNN reports that communications records of some members of Congress are being investigated to determine whether or not those lawmakers wittingly or unwittingly aided the insurrectionists.

What happens next is obviously out of my hands, so I find myself returning again to the one thing that is: Figuring out how to detect misinformation on-the-fly and teaching my daughter how to do the same so that we don’t become the next Elizabeth from Knoxville, so caught up in her alternate version of reality, that she doesn’t understand why the Capital Police would mace her when all she’d done was storm the Capitol to have a little revolution.

In line with my blog post on the same topic earlier this year, I have been doing quite a bit of reading lately on the subject of detecting and neutralizing misinformation, and thought I’d share a few of the more helpful blog posts, books, and articles I’ve found thus far.

Blog Post: The SIFT Method

I was chatting with the librarian at my daughter’s middle school one day about the problem of checking sources in an unevenly reliable online world, and she pointed me to the work being done by Mike Caulfield at his online literacy blog, Hapgood.

As the director of blended and networked learning at Washington State University Vancouver, Mike Caulfield spends his days thinking about the double-edged sword of today’s open source information and online communities. On the one hand, social media, blogs, and other online information sources can help us become more effective, engaged, and informed citizens. On the other, they are a great way to spread misinformation. 

Recognizing both the perils of source verification in an online world and the folly of expecting students to do in-depth background checks on every source they use in their education, Caulfield has developed the SIFT Method, a relatively simple and straight-forward method to sort fact from fiction in an online environment. Although he developed the SIFT Method primarily for use by students, I find it a useful rubric for folks like me who want to verify information found online without spending enormous amounts of time crawling down rabbit holes. 

SIFT stands for: 

• Stop
• Investigate the source
• Find better coverage
• Trace media, quotes, and claims back to their original source 

Caulfield describes the method in detail at his blog, but essentially it boils down to this. Before you share information you find online:

1. Take a moment (Stop) to figure out who is telling you this and whether they would be in a position to know it (Investigate)
2. See whether you can find the same information from a better, more trustworthy source (Find)
3. Track down the original source of any media, quotes, and claims that catch your eye so that you can see them in their original context and verify that they weren’t warped in the retelling (Trace).

• Read Caulfield’s original blog post here: “SIFT (The Four Moves)” (Hapgood, June 19, 2019)

Book: Calling Bullshit by Carl T. Bergstrom and Jevin D. West

Book Description: Calling Bullshit

“Misinformation, disinformation, and fake news abound and it’s increasingly difficult to know what’s true. Our media environment has become hyperpartisan. Science is conducted by press release. Startup culture elevates bullshit to high art. We are fairly well equipped to spot the sort of old-school bullshit that is based in fancy rhetoric and weasel words, but most of us don’t feel qualified to challenge the avalanche of new-school bullshit presented in the language of math, science, or statistics.

“In Calling Bullshit, Professors Carl Bergstrom and Jevin West give us a set of powerful tools to cut through the most intimidating data. You don’t need a lot of technical expertise to call out problems with data. Are the numbers or results too good or too dramatic to be true? Is the claim comparing like with like? Is it confirming your personal bias? Drawing on a deep well of expertise in statistics and computational biology, Bergstrom and West exuberantly unpack examples of selection bias and muddled data visualization, distinguish between correlation and causation, and examine the susceptibility of science to modern bullshit. We have always needed people who call bullshit when necessary, whether within a circle of friends, a community of scholars, or the citizenry of a nation. Now that bullshit has evolved, we need to relearn the art of skepticism.”

Source: The book description on Bookshop.org

My Review: ☕️☕️☕️☕️☕️

The SIFT Method works great for most things, but what if tracing the original source for a claim leads you to some highly technical scientific study? Can non-scientists and non-statisticians really understand the details of that study well enough to determine whether the medical, scientific, or sociological claim based on it is likely to be true?

In Calling Bullshit, Bergstrom and West walk readers through the process of evaluating study results based on the quality of the data that goes in and the soundness of the analysis of the data that comes out. As they point out early on, there’s no need to understand the technical details of the tests run or algorithms used to parse the data to understand whether the study itself is any good.

Bergstrom and West provide lots of lovely and eye catching examples of how data can be manipulated to support false conclusions to drive their lessons home. For example, did you know that from 1999-2007 the age of the Miss American winner correlated directly with the number of people murdered by steam, hot vapors, and hot objects? The correlation falls apart when you consider data from later years, of course, because in real life, there is no actual correlation between Miss America’s age and homicide by steam. Similarly, there is no real relationship between the number of sociology degrees awarded in the U.S. and the rate of death by anticoagulant in the U.S., despite the fact that those two numbers tracked closely for a period of time between 1999 and 2009.

The examples illustrating the principle of garbage in-garbage out when it comes to the methods used to collect data for scientific studies are just as fun to read and easy to understand.

I love this book, and highly recommend it to anyone who wants to be able to analyze a scientific study like a professional without being a scientist, statistician, doctor, or Ph.D of any sort themselves.

NOTE: At the time this review was written the hardback edition of Calling Bullshit was temporarily out of stock on Bookshop.org. However, you can pre-order the paperback edition, which ships April 20, 2021, here. And of course, the e-book is always in stock.

Article: “Comparing the COVID-19 vaccines developed by Pfizer, Moderna, and Johnson & Johnson” (STAT)

You may not have time at the moment to read Bergstrom and West’s book, but that’s ok because I think I’ve found a pretty good example of their method being used out in the real world.

Vaccine access is slowly opening up in our area, and with the emergency authorization of the Johnson & Johnson vaccine last week, I found myself asking the question: If I were given a choice of COVID-19 vaccine, which would I choose?

Figuring out the answer to that required taking a closer look at how the media is reporting the relative efficacy of the three vaccines, which is how I stumbled across Helen Branswell’s February 2, 2021 article for STAT comparing the Pfizer, Moderna, and Johnson & Johnson vaccines.

As Branswell reports, when the FDA approved Pfizer and Moderna vaccines for emergency use back in December 2020, they cited the vaccines’ truly stunning 94-95% efficacy against symptomatic COVID-19 after two doses. When the single dose Johnson & Johnson vaccine was undergoing evaluation for the FDA for approval in February 2021 the media reported that it was only 66% effective against moderate to severe COVID-19 illness. At first glance, the Pfizer and Moderna vaccines are clearly superior, right? Why would anyone want to get the Johnson & Johnson vaccine when the Pfizer and Moderna ones already exist? 

This is where context really matters. As Branswell points out, the studies establishing the efficacy of the Pfizer, Moderna, and Johnson & Johnson vaccines differ in some important ways. We aren’t comparing apples to apples when we pit the efficacy rates of Johnson & Johnson’s vaccine against those of Moderna and Pfizer. (In my head, I can just see Bergstrom and West nodding their heads sagely. After all, they warned me that this might happen.)

The Pfizer and Moderna vaccine trials were held before more resistant variants were discovered in the U.K., South Africa, and Brazil. At the time Branswell wrote her article, we didn’t yet have the data to know whether Pfizer and Moderna will be equally effective against the newer variants.

In contrast, the Johnson & Johnson vaccine trials include patients infected by the potentially vaccine-resistant B.1.351 variant first detected in South Africa. When you dig deeper into the Johnson & Johnson numbers, you will find hiding behind that top-line 66% efficacy rate an 85% efficacy against severe disease and 100% efficacy against disease severe enough to cause hospitalization and death for all patients in the study, regardless of variant. Even better, that protection came after just one shot. 

Suddenly the Johnson & Johnson vaccine looks like a much more viable option. Sure it would be nice not to get symptomatic COVID-19 at all, but not having to worry about being hospitalized or dying if I do catch COVID-19 sounds pretty good to me. 

“When people ask me, ‘What can I do now that I couldn’t do before I got vaccinated?’ My answer’s pretty simple: Live. You can live. You don’t have to worry about dying. Let’s build from that.”  

– Andy Slavitt, Senior Advisor for the White House Covid-19 Response, in a 24 February 2021 interview with Dr. Bob Wachter on the podcast In the Bubble

Before I took the time to get the context on the Johnson & Johnson vaccine, I was secretly planning to hold out for the more effective Moderna or Pfizer vaccines. But now that I’ve dug into the context around that data, my position has changed. I’m back to planning to take whichever of the three vaccines is available to me first. Please and thank you, I would like to live. This is probably a good thing, since as Branswell points out, COVID-19 vaccines are still quite scarce and whichever vaccine is available at my vaccination site when my turn comes is the one I’ll get.

• Comparing the COVID-19 vaccines developed by Pfizer, Moderna, and Johnson & Johnson (StatNews)
• “Andy Slavitt from Inside the White House,” In the Bubble podcast hosted by Dr. Bob Wachter, 24 February 2021

Book: How to Talk So Kids Will Listen & Listen So Kids Will Talk by Adele Farber and Elaine Mazlish

Book Summary: How to Talk So Kids Will Listen & Listen So Kids Will Talk

“This bestselling classic by internationally acclaimed experts on communication between parents and children includes fresh insights and suggestions, as well as the author’s time-tested methods to solve common problems and build foundations for lasting relationships, including innovative ways to:

• Cope with your child’s negative feelings, such as frustration, anger, and disappointment
• Express your strong feelings without being hurtful
• Engage your child’s willing cooperation
• Set firm limits and maintain goodwill
• Use alternatives to punishment that promote self-discipline
• Understand the difference between helpful and unhelpful praise
• Resolve family conflicts peacefully

“Enthusiastically praised by parents and professionals around the world, Faber and Mazlish’s down-to-earth, respectful approach makes relationships with children of all ages less stressful and more rewarding.“

Source: The book description on Bookshop.org

My Review: ☕️☕️☕️☕️☕️

All of this work I’m doing to learn how to dismantle and neutralize misinformation isn’t worth much if I don’t know how to talk to my kid about it without getting her hackles up.

Our school district is running a workshop series for parents of tweens and teens designed to help us deal with the added stresses created by parenting in a pandemic while our primary backup parent network (teachers, librarians, school counselors, and sports coaches) are no longer available in-person. In one of the break out sessions I listened in awe as a parent described her standard reaction when her stompy, stressed out teen wails about her terrible, no-good, very bad day: “I’m so sorry you feel that way. That sounds so hard.”

I know her exact words, because I made her repeat them so I could write them down. I need scripts like this to short-circuit my own immediate emotional let-me-fix-that-for-you response.

As I was writing down her words, she mentioned that her script came straight out of one of the comic strips in the book How to Talk So Kids Will Listen & Listen So Kids Will Talk. The book was already sitting on my shelves, but I haven’t found time to read it yet.

“Oh, don’t worry about making time to read the whole thing,” she told me. “Most of what I needed from it, I learned by skimming the comics.”

Me being me, I decided to read the comics and have ended up reading the whole thing. In addition to the comics, which take the place of role-playing sessions at the therapist’s office, there are handy reminder cards summing up the lessons in each chapter. The authors recommend that you cut them out and tape them somewhere you will be likely to see them when a relevant parenting challenge comes up. (Sadly this would require defacing a book, so I have merely flagged the pages in my copy and am hoping for the best.)

Although this book is primarily about how to talk with kids without driving them back into their shells, while reading it, I realized that some of the techniques to diffuse emotional situations in Chapter 1 and 2 might also come in handy when I need to have a difficult conversation with an adult I care about but happen to disagree with.

Housekeeping

Should you choose to purchase Calling Bullshit or How to Talk So Kids Will Listen and Listen So Kids Will Talk from the Caterpickles Bookstore, you’ll notice that I’ve added a few new sections to my little virtual shop. In addition to existing recommendations for middle grade and elementary graphic novels, picture books, fiction, and nonfiction, I’ve added lists to catalog some of my favorite adult fiction and nonfiction books, as well as books that I believe have made me a better parent. Check it out.

What about you?

What have you been reading lately?

Related Links:

• Well that sure was something (Caterpickles)

This site uses Akismet to reduce spam. Learn how your comment data is processed.