Fact-checking the State of the Union
The State of the Union represents, as CNN’s Brian Stelter called it, the Super Bowl of fact-checking. And like this year’s football game, there was a lot of interest for an event that may or may not have lived up to its hype, depending on your perspective.
After watching the speech and the fact-checking that accompanied it, we had three takeaways.
1. Repetition made the job easier — but also less interesting.
Many of President Trump’s assertions in the speech were ones he’s made before, so it made fact-checking in real-time much simpler. When House Speaker Nancy Pelosi tweeted that it will “take days” to fact-check all the president’s misrepresentations, Washington Post fact-checker Salvador Rizzo countered: “Not much was new. We were done by 11:30.” The New York Times’ Amy Fiscus tweeted: “We did it in an hour.”
But the lack of new claims can also give fact-checking a recycled feel. That meant that some of the more valuable checks on Tuesday night were those that put context around a claim and gave readers a new way to think about something they might have heard before. An example wasFactcheck.org’s check of Trump’s claim that “African-American, Hispanic-American and Asian-American unemployment have all reached their lowest levels ever recorded.” The fact checker made the point that, while they have reached low levels, “the gap between white unemployment and the rate for black and Hispanic Americans has remained the same.”
1. The Super Bowl metaphor is apt.
Just as there are people who watch football once a year, there are fact-checkers who see big events as a fact-checking opportunity, even though they don’t do it as regularly as the players dedicated exclusively to fact-checking. Most of the big national news organizations fact-checked the president’s speech. Along with (Poynter-owned) PolitiFact, The Washington Post’s Fact-Checker, Factcheck.org and the Associated Press, other participants included the TV networks, the Los Angeles Times, USA Today and Politico.
Weighing in with fact checks only during big events makes sense for some news organizations — it is a time when readers are engaged, it’s a way to put context around what otherwise might be just a speech story and it can help start a stockpile for future fact-checking efforts. But when reporters who aren’t familiar with the practice produce flawed fact checks, it can give ammunition to critics of legitimate fact-checks.
3. Real-time fact-checking is still the hardest kind.
Some of the checks showed again that the art of fact-checking can be boiled down to four skills: Knowing what claims to check, avoiding subjective claims, resisting the urge to immediately assign a rating and showing your work. That all becomes harder in real time, which is where experience — and a ready stockpile — come in handy.
Given the effort needed, the question is whether it’s worth it. The answer is probably. Twenty-four hours after Trump’s appearance, The New York Times’ fact check of the speech was still one of the most popular stories on its site. According to CrowdTangle, at least five fact checks from independent fact-checkers went viral leading up to and after the speech.
The real-time challenge was also illustrated in the malfunction of FactStream, an app designed by Duke Reporters’ Lab to offer real-time updates from PolitiFact, Factcheck.org and The Washington Post Fact Checker. Bill Adair, the head of the Reporters’ Lab, wrote here about how the app crashed under an unusual surge of heavy traffic in its second State of the Union test.
- Snopes announced that it has withdrawn from its partnership with Facebook, saying the project was too labor-intensive for its remote staff. ABC News has also reportedly stopped participating and the AP is reevaluating its role. Meanwhile, debunking site Lead Storieshas joined the partnership and will us its Trendolizer platform to identify viral false stories, images and videos to flag on Facebook.
- A new report from The Knight Commission on Trust, Media & Democracy includes some recommendations aimed at Big Tech. One of the commissioners, author and former USA Today editor Joanne Lipman, outlines them here. Meanwhile, Brendan Nyhan and Patrick Ball wrote that employees at tech companies should put more internal pressure on their bosses to change anti-misinformation policies.
- Facebook and YouTube are key platforms for the amplification of antivaxxer misinformation, The Guardian reported. While YouTubesaid just a couple of weeks ago that it would start downranking conspiracy theories in its recommendation algorithms, it appears that it hasn’t really been working so far. And health misinformation is a growing problem for the platforms: new fact-checking project Health Feedback found that 7 out of 10 of the most popular health stories of 2018 included at least one piece of misinformation.
- Ivory Coast minister Alain Lobognan was imprisoned on “false news” charges after tweeting about how a state prosecutor had arrested another MP. But Lobognan said he was arrested for political reasons — not spreading false information. Ivory Coast is just the latest sub-Saharan African to imprison someone on false news charges for a tweet, albeit among the few that have imprisoned politicians.
- Columnists continue to write about U.S. Rep. Alexandria Ocasio-Cortez’s (D-N.Y.) run-in with fact-checkers — and criticize their alleged false equivalence. Sigh.
- Sen. Elizabeth Warren (D-Mass.) has launched her own fact-checking project as part of her 2020 campaign for U.S. president. The move is part of a global pattern of politicians copying fact-checkers’ format to push back on their critics.
…the future of news
- NewsGuard, a startup that rates the credibility of different websites,has come under fire for its grading system in recent weeks. But it also has an unintended positive consequence: Getting news sites to improve their transparency.
- Here’s how Chequeado is leveraging automated technology to make its fact-checking process more efficient and effective. Maybe it’s time to revisit this fact sheet published by the Reuters Institute last year on how close we are to automated fact-checking.
- We spend a lot of time in this newsletter talking about the threat that misinformation poses around the world — and how that threat is changing. But in a new study released this week, researchers found that the scope of fake news consumption and its effect on American politics has been overestimated. In a Medium post, Brendan Nyhan wrote that “Fake news consumption is concentrated among a narrow subset of Americans with the most conservative news diets.”
Each week, we analyze five of the top-performing fact checks on Facebook to see how their reach compared to the hoaxes they debunked. Here are this week’s numbers.
- PolitiFact: “No, New York abortion law doesn’t let mothers abort babies a minute before they would be born” (Fact: 31.2K engagements // Fake: 7.2K engagements)
- Factcheck.org: “Pelosi Didn’t Spend $497 Million on Renovations”(Fact: 5.5K engagements // Fake: 1.1K engagements)
- Aos Fatos: “Marina Silva did not license dams of Brumadinho and Mariana; permission came from the government of Minas Gerais” (Fact: 3.3K engagements // Fake: 36.4K engagements)
- Les Décodéurs: “No, this photo does not show Edouard Philippe sleeping in the National Assembly” (Fact: 3.3K engagements // Fake: 23.8K engagements)
- Full Fact: “Are dogs dying from eating jerky treats?” (Fact: 245 engagements // Fake: 30.3K engagements)
The Washington Post Fact Checker offered a deep dive into claims about sexual assaults against female migrants traveling to the U.S. border.
Numerous reports have attributed to Amnesty International a claim that 60 percent of migrant women are raped as they journey through Mexico. The Post gave this assertion four Pinocchios.
It also checked a claim, repeated by Trump during the State of the Union, that one in three women is sexually assaulted on the journey north, and gave that claim two Pinocchios. That assertion had been checked the previous week by PolitiFact, with a similar Half True rating.
What we liked: The Fact Checker took the reader on a wild journey through documents and studies back a quarter century to get to the bottom of this number. The piece, written by Glenn Kessler, first traces the number to a 2010 report, then a 2002 report, then a 1999 report and then, possibly, back to a 1995 study. The whole thing, he says, amounts to a “bad case of academic telephone.”
- The Agence France-Presse has expanded its fact-checking endeavors to Arabic. The AFP has also posted fact-checker jobs in Malaysia and Sri Lanka.
- A Macedonian military official was behind a series of fake news sites first identified by Lead Stories and Nieuwscheckers.
- Digiday published a guide for understanding the different ways people commit digital ad fraud.
- The New York Times’ Jack Nicas took a closer look at the numbers of fake accounts Facebook have reported in recent years andasked the question: Does the company *actually* know how many there are?
- Anticipating attempts by bad actors to suppress participation in its 2020 census count, the U.S. government is developing a plan to fight back, according to NextGov.
- U.K. Digital Minister Margot James said the government would make efforts to crack down on social media platforms that she said have “fallen short” in responding to online bullying, abuse and misinformation.
- WhatsApp is trying to battle fake news in India by tinkering with its app, bolstering local fact-checking organizations and airing national ads. The Los Angeles Times reported on those moves, saying WhatsApp’s efforts are seen as a test of parent company Facebook’s commitment to solving its fake news problem.
- Popular debunking tool InVid is seeing some features restricted for Facebook videos after the company decided to reevaluate its participation.
- Jules Darmanin, formerly of BuzzFeed France, is heading up the FactCheckEU project, which will unite European fact-checkers ahead of this year’s parliamentary elections.
- Facebook took down hundreds of accounts linked to a fake news syndicate in Indonesia.
That’s it for this week. This is the second week of our new design. Let us know what you think by emailing email@example.com!