Author: JF

Why fact-checking doesn’t work: Presidential address edition

On By JFLeave a comment

Ahead of what is likely to be a fact-free monologue on our public airwaves this evening from our possibly criminal president about the “border crisis,”  a lot of folks in the public eye have responded with some form of “Well, OK, but the media should fact-check everything he says!

And sure, I get that. I’ve been working in libraries and academia for decades, and citing sources and ritualizing the “checking of facts” is squarely in my wheelhouse. Various media organizations have been increasingly on the job in this space as well – especially under our current administration. There is a lot to check, and I have nothing but respect for groups like Politifact trying to play factual Whack-A-Mole with the barrage of information and misinformation floating around the digital sphere. 

The Last Stand: Science Versus Superstition (Keppler, 1899) https://www.loc.gov/item/2012647443/

And the Enlightenment happened. Is that still a thing? Empirical facts?

Yes. But.

Even if this isn’t something you particularly study (like me and a bunch of other nerds do), it’s impossible to ignore  how untethered much of our public discourse has become from a shared consensus of truth in recent years. One can blame the Internet, the 24/7 news cycle, political polarization, the decimation of local news staffs, or probably a million other things…it’s a complex phenomenon.

But regardless of the reasons – we know ideas of “truth” are more than mutable right now. Kuhn’s paradigms shift with the winds of economic and social realities. In 2019 this shifting seems faster than ever. And it is. But the work of journalism, and the way people engage with “news” has benefitted from some outstanding scholarship going back decades that has been remarkably consistent.

That scholarship pretty clearly shows that journalists “fact-checking” a powerful individual, motivated and known to lie, in an wide-open media channel is just not going to to solve much. Why not?

1. Journalists are trained and socialized to get their news from “expert” sources

Going back to the earliest ethnographic studies of journalists, we see that methods and routines are how journalists are able to lay claim to the truth. It’s getting the sources, it’s verifying the sources, it’s having the review of a editorial desk, and not skipping any of the steps. In Gaye Tuchman’s essential scholarship of investigative reporters she found that “by stressing methods — gathering supplementary evidence, presenting conflicting truth-claims, imputing facts through familiarity with police procedures, and using quotation marks, to name some techniques analyzed earlier – newsworkers produced a full-blown version of the web of facticity.” (Tuchman, 1978, p. 160). “Facts” are built on a series of well-ordered steps involving multiple informants.

Journalists traditionally work to position authority for deciding the truth outside of themselves. Even when they position themselves as arbiters of truth, such as in cases like the Politifact site, they take particular care to be transparent about their process citing every source consulted.

Mark Fishman writing in 1980 made a similar point:

“…we need to examine the journalist’s general criterion of facticity. This fundamental principle of new fact can be stated like this: something is so because somebody says it. Newsworkers take their facts from other people’s accounts. … reporters rarely sift through physical evidence or run tests on it.” (Fishman, p. 92)

Which goes to say that creating “facticity” on their own in real time is new work (historically speaking) for journalists. With the rise of data-driven reporting (Coddington 2015), more freelance writing with less editorial support, and the general waning of access journalism, this is less true (Borges-Rey, 2017), but illustrates why many of our media organizations seem so unprepared for dealing with government authorities unleashing daily barrages of misinformation.

2. People believe news based on a variety of factors; indicators of literal truth are far down on the list.

There are heaps of studies on this, but a couple that seem relevant right now.

Research illustrated that news that shows “both sides” of disagreement of an issue will often appear more credible. Performing a ritual of debate, even to the point of incivility, communicates to readers that the “best” information has won (Thorson, 2010). Heated comments sections and Sunday morning news shows are treading well-worn paths.

And there’s just so much to process. We are overwhelmed by information as a species. In order to triage all that’s coming at us, we look for people that we trust to serve as our filters. Being prescient as only Phil Agre could, writing in 2004:

“Coming up with novel political arguments requires a lot of work. Human beings are finite and nobody has the time or knowledge to invent thought-out arguments on every issue all by themselves. ….a political infrastructure is successful if it delivers the right arguments ot the right poeple at the right time. This has always been true. What is different now is simply the scale and speed with which debates collectively unfold in a society.” (Agre 2004, p. 205)

And now. On top of the amplifications of speed and scale that have been developing over a couple decades, we now have current political leaders who are also larger-than-life, and perfectly poised to inform an overwhelmed information heuristic.

“…people use political figures as a heuristic to guide evaluation of what is true or false, yet do not necessarily insist on veracity as a prerequisite for supporting political candidates” (Swire, et al 2017. p. 1)

If your favorite politician said it, it gets very little empirical scrutiny.

So, there is, in all likelihood, a very limited outcome to reporters’ fact-checking efforts — for the speech tonight or any other similar event. As others are pointing out:

So does that mean give up on fact-checking? Or the entire notion of facts?

Not at all. But facts alone are not enough. The social worlds — the professional networks, the friendship groups, our families and faith groups, the shared training among colleagues — these are all ESSENTIAL underpinnings of what makes a fact. We cannot simply throw a fact in what we perceive as an information gap. Particularly when those information gaps are buried in social worlds riddled with structural white supremacism and committed to bad faith “debates” we need new strategies.

There is difficult work ahead.

Bibliography

Agre, P. E. (2004). The practical republic: Social skills and the progress of citizenship. In A. Feenberg & D. Barney (Eds.), Community in the Digital Age (pp. 201–223). Lanham, MD: Rowman & Littlefield.

Borges-Rey, E. (2017). Towards an epistemology of data journalism in the devolved nations of the United Kingdom: Changes and continuities in materiality, performativity and reflexivity. Journalism, 1464884917693864. https://doi.org/10.1177/1464884917693864

Coddington, M. (2015). Clarifying journalism’s quantitative Turn. Digital Journalism, 3(3), 331–348. https://doi.org/10.1080/21670811.2014.976400

Fishman, M. (1980). Manufacturing the news. Austin: University of Texas Press.

Godler, Y., & Reich, Z. (2013). How journalists “realize” facts. Journalism Practice, 7(6), 674–689. https://doi.org/10.1080/17512786.2013.791067

Robinson, S. (2009). “If you had been with us”: Mainstream press and citizen journalists jockey for authority over the collective memory of Hurricane Katrina. New Media & Society, 11(5), 795–814. https://doi.org/10.1177/1461444809105353

Thorson, K., Vraga, E., & Ekdale, B. (2010). Credibility in context: How uncivil online commentary affects news credibility. Mass Communication & Society, 13(3), 289–313. https://doi.org/10.1080/15205430903225571

Swire, B., Berinksy, A. J., Lewandowsky, S., & Ecker, U. K. (2017). Processing political misinformation: Comprehending the Trump phenomenon. Royal Society Open Science. https://doi.org/10.1098/rsos.160802

Tuchman, G. (1978). Making news: A study in the construction of reality. New York: Free Press.

Understanding fake news in 2016: Before the truth gets its pants on*

On By JF4 Comments

*

I’ve been researching information credibility for around 8 years (and credibility in news and journalism for almost as long).  It was the main thread of the dissertation I just finished.

Meanwhile, in the wake of 2016 US election, figuring out how voters come to believe some (less true) news items over others is a topic a lot of people are taking way more seriously.

For example, the news about Trump’s popular vote “win” which spread like wildfire.

How do we explain this?

Are we a “post-fact” culture now?

How does misinformation spread so quickly?

Has it always been this way?

Is the decline of traditional journalism and the rise of social media as news source part of the explanation?

There is a lot of work to do to figure this out

Below is my effort at basic reading list on some good academic work discussing how people get their news, why they believe what they do, and how models from traditional news work may or may not apply to the social media world. People study this question in mass communication, but also within the disciplines of information science and human-computer interaction. This is a small slice of the research, but these are some articles and books that have been the most useful to me.

**Update 11/19/2016 – I’ve added annotations and hyperlinks to the list. Thanks for the feedback.

booktrash

Understanding credibility in “traditional” news reporting

  • Breed, W. (1955). Social control in the newsroom: A functional analysis. Social Forces, 33(4), 326–335.
    This is a foundational sociological examination of what goes on in a newsroom. The book describes the social norms that encourage reporters to write the stories they do, and how editorial pressures are negotiated. In this era there was a “taboo” against direct publisher influence on news content – the independence (in appearance at least) of reporters was a highly-valued standard. Nonetheless, the conformity of reporters to the status quo was controlled subtly by many different social pressures.
  • Gans, H. J. (1979). Deciding what’s news: A study of CBS evening news, NBC nightly news, Newsweek, and Time. Evanston, IL: Medill School of Journalism, Northwestern Univ Press.
    Similar to the Breed book in taking a social and organizational view, this study looks more specifically at the relationships between reporters and their sources – and its traditional adversarial nature. “Journalists harbor a pervasive distrust of their sources, since so many come to them with self-serving motives” (p. 130). One if its more useful points is how journalists in established beats develop relationships with well-placed sources over time and have multiple encounters over which to establish trust and build credibility. This makes for a useful contrast to 21st century journalism often limited by contingent freelancer arrangements and online interactions.
  • Hovland, C. I., & Weiss, W. (1951). The influence of source credibility on communication effectiveness. Public Opinion Quarterly, 15(4), 635–650.
    This is a classic information credibility study from two psychology researchers at Yale University. The “same” content was presented to undergraduate volunteers – one experimental group read scientific news content as if it appeared in the Soviet magazine Pravda, the other group read the content as if it appeared in Life magazine. Participants were surveyed afterward – unsurprisingly, most students viewed the Life magazine article as more trustworthy, but interestingly, retention of basic facts were the same for both groups.
  • Tuchman, G. (1978). Making news: A study in the construction of reality. New York: Free Press.
    Tuchman’s approach borrows Erving Goffman’s notion of frames and framing to show how reporters, when encountering some real-life event, will shape their description into established story frames.

How are networked and social media news sources different from newspapers?

  • Carlson, M. (2007). Blogs and journalistic authority. Journalism Studies, 8(2), 264–279. doi:10.1080/14616700601148861
    This paper is particularly interesting in that is uses 2004 US election day coverage as an illustrative case, and was one of the earlier studies to look at how blogs work alongside of established forms of media.
  • Carpenter, S. (2008). How online citizen journalism publications and online newspapers utilize the objectivity standard and rely on external sources. Journalism & Mass Communication Quarterly, 85(3), 531–548.
    Highlights the importance of routines (consistent practices for researching sources and stories) for journalists. Journalists affiliated with traditional news were more likely to adhere to normative routines and “official sources” as support for the appearance of objectivity. Citizen journalists were more likely to rely on unofficial sources and less likely to stick to routine practices.
  • Karlsson, M. (2011). The immediacy of online news, the visibility of journalistic processes and a restructuring of journalistic authority. Journalism, 12(3), 279–295.
    My favorite quote from this article: “Thus, it is difficult for journalists to refer to a ‘we get it right’ norm when the ‘we’ part is potentially compromised by users, and the ‘get it right’ part is compromised by fast inadequate news and a plethora of different voices ” (p. 280). No kidding. Karlsson suggests transparency (visibly sourcing everything) as a new norm, which, of course, then pushes a significant burden on the reader to view and evaluate all those now-visible threads.
  • Thorsen, E. (2008). Journalistic objectivity redefined? Wikinews and the neutral point of view. New Media & Society, 10(6), 935–954.
    This research is a good example of how new forms of news — in this case, Wikinews —  re-interpret existing practices and values from “old school” journalism.

The role of searching and algorithms in the trust of news

  • Jessen, J., & Jørgensen, A. H. (2011). Aggregated trustworthiness: Redefining online credibility through social validation. First Monday, 17(1).
    The fact that what people *say* about what they trust is often not exactly correlated to what they *do* when deciding what information to trust.  Proved by research! We’re much more social and less analytical then we like to let on.
  • Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM), 46(5), 604–632.
    This article from pretty early in the Internet area is prescient in predicting how quantities of links will aggregate to influence perfections of trustworthiness. Google was not yet “a thing” but the authors refer to the 1998 Brin and Page paper outlining its approach.

  • Pan, B., Hembrooke, H., Joachims, T., Lorigo, L., Gay, G., & Granka, L. (2007). In Google we trust: Users’ decisions on rank, position, and relevance. Journal of Computer-Mediated Communication, 12(3), 801–823. doi:10.1111/j.1083-6101.2007.00351.x
    This study employs eye-tracking methods to demonstrate how users concentrate their focus at the top of search results, ignoring other heuristics or relevance of the links.

How do we understand credibility, trust, and online information going forward?

 

.