Insights, tools and research to advance journalism

Why fact-checking is the root of journalism: 8 good questions with PolitiFact’s Angie Drobnic Holan

Angie Drobnic Holan is top editor in charge of PolitiFact, a news unit within the Tampa Bay Times organization. She sees fact-checking as a part of journalists’ mission to hold politicians accountable.

While technology has changed quickly and challenged the work of journalists, the roots of journalism have not changed and may even become more important. We talked with Holan about reader trust and how fact-checking tools like the Truth-O-Meter can help reach out to a new audience.

1. Take us back in time a little bit — how did the idea for the Truth-O-Meter come about? 

Angie Drobnic Holan

Angie Drobnic Holan

HOLAN: We started PolitiFact and our fact-checking Truth-O-Meter back in 2007. That was an interesting time for us as a news organization because our Washington bureau chief, Bill Adair, was working for the Tampa Bay Times and thinking about how to cover the 2008 presidential election in a new and different way.

He knew he wanted to do something new with more fact-checking because he had a personal belief that there wasn’t enough fact-checking in political news, and one of the ideas he had was for a database-driven website.

During 2007 when we were planning our election coverage, he pitched this idea to Tampa Bay Times editor Neil Brown and asked if he could launch this site, and the idea was approved. And it’s been very successful ever since.

The way it works is that we take individual statements from politicians, candidates, and now even sometimes talking heads, and we fact-check the statement and then we give it a rating to share its relative accuracy. And our ratings are true, mostly true, half true, mostly false, false, and then for a statement that is false and ridiculous we give it the rating pants on fire.

2. How do you decide which statements to fact-check?

One thing that we don’t do is try to balance the ratings. We don’t think about if we get a false on one side, we want to go and get a false on the other side.

We select statements to check primarily based on whether we think something either sounds wrong or just catches our attention and makes us wonder, “Hmm, could that be true?” We try to pick the things that we think the average voter would be interested in, or care about. And then, after those criteria, we follow the news, we try to fact-check whatever is being talked about in the national discourse.

So when we come in every morning, we have discussion about what’s been said, about what the issues of the day are. We have an internal spreadsheet where we collect facts that sound like we might like to check them and then we just basically look them over and say, what seems like it would be the best news story for a fact-check? And then we go forward.

We try to fact-check approximately the same number of Democrats and Republicans but we don’t keep hard-and-fast count, and one thing that we don’t do is try to balance the ratings. We don’t think about if we get a false on one side, we want to go and get a false on the other side. We do not do that. We just try to pick an interesting, diverse mix of facts to check. And then wherever the ratings come out in our editorial process, that’s how they come out.

3. Are there any red flags or especially recurring things you look for?

We can still publish our findings if something turns out to be true.

If we hear something that sounds wrong that we say, “Hmm, that sounds wrong,” we will definitely be attracted to fact-checking that. Now sometimes something will sound wrong reported, and we search it, and find that it’s actually not wrong, and we’ll still publish it anyway. We are a little different than some fact-checkers in that because we have the true rating and the mostly true rating, we can still publish our findings if something turns out to be true and sometimes it does.

In other cases it’s just facts that sound interesting or intrigued to us at some level and we’ll fact-check those to find out because you know, we are journalists and I think like a lot of journalists we are driven by curiosity and willing to find out more about the world. So some of our fact-checks are just some things that we hear and found interesting and we wanted see what the story behind them was.

4. You are working with partners across the country, how does everybody draw consistent conclusions? Can this be entirely objective?

At the end of the process we have a panel of three editors who vote on the rating … We feel like we are very transparent in our reporting and our methods.

We have a process for reporting, and the way it works is that one reporter is on a story and they work with their editor and the editor and the reporter then bring the story to a conclusion. For that process that’s very much like a traditionally reported news story; we review documents, we interview experts.

What’s different between PolitiFact and traditional news stories is, in a PolitiFact news story we go through all of the evidence we find in a pretty detailed fashion and we also in our online reports include all of our sources. So there’s a list with links to all the documents we reviewed and we name all of the experts we spoke with.

At the end of the process we have a panel of three editors who vote on the rating and sometimes there is disagreement about whether something should be rated say half true or mostly true or whether something is false or mostly false. And the editors-panel votes on this finding.

Now we’ve published more than 7,000 fact-checks and we don’t expect readers to agree with every rating. Sometimes they disagree strongly, but overall we feel like we are very transparent in our reporting and our methods, so even if someone disagrees with our rating, we hope they’ve learned some things from our report.

5. Have you seen politicians stop making false statements as a result of your fact-checking?

Most of the people we fact-check want to be right … Sometimes we will see them change what they say to try to make it more accurate.

You know, it’s interesting because I do see some politicians have changed their talking point after we fact-check it or others have fact-checked it. Other politicians just seem to repeat the same thing regardless how often it’s fact-checked, so I think it depends on the specific case.

However, I would say most of the people we fact-check want to be right and when they are found that their statements are wrong sometimes they’ll say that maybe they misused the words but their broader point was accurate. So in these cases sometimes we will see them change what they say to try to make it more accurate.

Overall I do think fact-checking has been very helpful to journalism because I think it has brought renewed attention to the importance of checking facts and not repeating claims that are easy to verify if they are not accurate, though maybe “easy” isn’t the right word. I think journalists want to be careful to not repeat claims that are not accurate. I think fact-checking gives a renewed emphasis to that and I see in traditional news stories more fact-checking since I started fact-checking back in 2007, and that is very heartening.

6. How do you measure the impact of your work?

What we say is, we are fact-checking to give voters the information they need to govern themselves.

You know, impact is an interesting question. I say often that we do not fact-check with the intention to stop politicians from exaggerating because I think that might be just part of politicians and what they do. And in fairness to the politicians, I think human beings, all of us, maybe had situations sometimes where we tended to exaggerate.

What we say is, we are fact-checking to give voters the information they need to govern themselves. So we are not really trying to change anyone’s behavior but we are fact-checking hoping that voters get the information they need so that when it’s time to make decisions about who to vote for, or where they stand on a particular issue, that they have good, factual, and accurate information to make those decisions.

7. Do you help teach other journalists about how to fact-check?

We train people who are our formal partners, who are part of our PolitiFact network. With international journalists, we tend to be more informal advisors, or have discussions with them. Another thing is, you know, we have a small staff, so our first priority is to publish our own fact-checks. So our time is limited as far as training goes but we do try to spread the word about fact-checking, share our techniques. We are always happy to talk about how we do things.

We’ve also published our reports online about how we do our fact-checking, about what our methods are, we have a document on our website called “Principles of the Truth-O-Meter” that explains in some detail how we approach fact-checking.

8. Looking ahead, do you see potential in the Truth-O-Meter expanding, either by franchising it or having teams abroad?

There is nothing that is secret about how to fact-check. I think it’s more just of an approach, an outlook, and also a commitment from the news organization that it wants to fact-check.

We have 10 state affiliates right now and we are always looking for new affiliates. We would like to have affiliates in all 50 states to fact-check the state-based elections, so I do think that there is lots of opportunity for expansion there.

We also have one international partner. We have helped an organization, PolitiFact Australia, to get launched in Australia, and they fact-checked the most recent election and we are very pleased with the results.

In addition to that, we have also advised several organizations about how to launch international fact-checking efforts, but language barriers made it more difficult to do formal partnerships. But we certainly see fact-check efforts growing around the world, and we support that because it’s independent journalism that has a focus on accuracy and and accountability. So I think it has a lot of potential to expand. I think we are really just starting right now.

Journalists we’ve talked to in other countries about what we’re doing sometimes just start sites on their own and they will say that they were influenced by PolitiFact. I would emphasise that fact-checking has its roots in traditional journalism. There is nothing that is secret about how to fact-check. I think it’s more just of an approach, an outlook, and also a commitment from the news organization that it wants to fact-check.

  • Bryan

    When did this interview take place? Holan says PolitiFact has 10 state affiliates “right now,” but with the loss of PF Ohio this year I count no more than nine. PF Ohio hasn’t published a fact check since September 2013.

    As for the international affiliate in Australia, it’s on hiatus and PolitiFact no longer links to it from its main site.

    Notice how Holan dodged this question?

    “You are working with partners across the country, how does
    everybody draw consistent conclusions? Can this be entirely objective?”

    Here’s the real answer on that one:

    http://www.politifactbias.com/2014/02/ho-hum-politifact-flubs-another-gender.html

    PolitiFact is often inconsistent, even without taking affiliates into account.

  • fst1

    Questions that Politifact ignores.3/28
    2012 Ambassador Cretz sends cable begging for more security. 4/6
    terrorists throw IED over consulate wall 4/19 Clinton sends cable to
    Cretz denies request for more security suggests pare down policy. 5/3
    2012 State Department denies request from ambassadors security team for a
    DC3. 5/22 Terrorists attack Red Cross threatened warn US of
    attacks..6/6 terrorists blow large hole
    thorough consulate gate. 6/7 Stevens asks State Department to keep MSD {
    security teams } Libya 6/7 British ambassador survives assassination
    attempt 6/10 Al Qaeda forces openly ralyl in Benghazi 6/15 Ambassador
    Lamb tells State Departments Norstrom that security team SST contract
    will not be renewed. 6’22 Stevens warns that extremist groups are
    operating openly in Benghazi as a whole. 7/6 Lamb demands that Tripoli
    not make formal request fos SST MSD SECUITY TEAMS 7/9 Stevens requires
    13 more personal because off unstable situation. not be renewed that
    Regional security officer Lamb warns State Department RISK TO US
    OFFICIALS IN Libya HIGH 8/2 Ambassador Stevens sends URGENT cable to
    Clinton requesting 12 man protective security detail body guard 8/ 5
    State department orders removal of AMBASSADOR STEVENS SECURITY TEAM !
    8/8 SECURITY TEAM DEPARTS LIBYA 8/ 16 REGIONAL SECURITY OFFICER SENDS E
    MAIL TO CLINTON WARNING OF DIRE SITUATION. 9/11 AMBASSADOR STEVENS SENDS
    CABLE TO STATE DEPARTMENT WARNING OF DIRE SITUATION IN BENGHAZI.

  • fst1

    Obama’s reaction Benghazi Angie Drobnic Holan’s work is sloppy to say the least and Journalistic malpractice to say the most . When challenged on twitter she routinely blocks comments. She claims Obama called it a terrorist attack while ignoring the Video story completely basing her questions on Romney s reaction two years after the event. In other words she makes Romney the story. It isn’t just the answers she gives that are dishonest its the questions she poses. Asking the question “Was the US flag the last flag flying ? is like saying how did you enjoy the play Mrs Lincoln ? Once again she makes Gowdy the story while absolving Obama and Hillary of any responsibility. But it doesn’t end there. She throws a bone to budget cuts not being the problem but then she goes right back to covering for Obama and Hillary.
    Did Hillary Clinton know that more security was needed? Instaed of fact checking for responsibility she goes after Rand Paul. She sets up a false narrative that Hillary isn’t directly responsible because there is no cable with her name ! She’s The Secratray Of State for G-ds sake ! Lol Here is Politifacts quote,
    “The State Department was asked repeatedly for additional U.S. security
    staff in Libya. But we see no evidence that Clinton herself was made
    aware of those requests. For example, there’s no “direct cable” that
    automatically appears on the secretary of state’s desk. Instead, all
    cables would have carried her name, and only those passed up the chain
    by staff would have reached her. She says they didn’t. Paul’s office
    didn’t provide evidence linking requests for security directly to
    Clinton. We found only evidence that requests were made of her
    department. We rate his statement Mostly False.” Are you kidding me ? Politifact is an arm of the Democrat party. It is partisan and therefore should not be considered a fact checking site.

  • Pingback: The facts are the focus | clues to the cloud

Need to Know newsletter

The smart way to start your day

Each morning we scour the web for fresh useful insights in our Need to Know newsletter. Sign up below.

Featured topics

Go deeper on…

Dive deep on everything we produce about these key topics.

Youth news literacy

Transforming a longtime youth news product to match new behaviors: 7 good questions with Channel One News

Channel One News started 23 years ago as a 10-minute daily TV news program for children. But over the years it has recognized that students and teachers are using a variety of devices and platforms to consume information. We talked to CEO CJ Kettler about what changed and what stayed the same.