A fact is a truth, that verifiable, reliable backbone of the documentarian's work. But in an era when information is filtered through the lens of belief and agenda, truth to one group may be heresy to another. Fact-checking has never been as complex or vital, whether to provide authenticity to a report or recapture a flailing public trust. Post 9/11, the documentarian has become a pivotal witness to a world in uproar, taking on many a journalistic mantle. But according to the Center for Media and Social Impact's report Dangerous Documentaries: Reducing Risk When Telling Truth to Power, documentarians often find themselves with far fewer resources than the investigative journalist. An integrated model of fact-checking is one such resource, still in development within the documentarian's workflow.
News corporations that make documentaries enlist the crucial support of an in-house fact-checking team or external consultant. In independent documentary, fact-checking often falls to the documentarian in conjunction with a producer. Filmmaker Lauren Rosenfeld Capps (Captive Radio, Rape in the Fields) notes, "You need a paper trail and primary sources to back what you are saying." Capps creates a master document that contains all data that requires fact-checking, from dates and times to quotations and legal documents. "Working in Google Spreadsheets, I add in each fact and create columns for corresponding primary sources," she explains. "Each fact is linked to its related documents in a Dropbox folder, almost like a bibliography. If it's a government document or a public record or digitized court document, you link to it online. If it's not online, you create a copy, put it in the Dropbox and refer to the page number and quote in the master document. If someone questions something, you go back to that line, see where you got that document from and how you can verify it. It's important to be meticulous, especially in more controversial investigative works. If something is factually questionable and has no primary sources to back it up, even if it's a great piece for the film, it might not be worth using."
Capps would prefer not to fact-check alone. "I like at least two eyes on a script, in addition to the producer or person writing it," she says. "Most of the time I have been able to have at least one other person involved. We either split up each fact and tackle the script jointly or have a second set of eyes on the same set of lines or facts. If I'm writing a script that’s changing, it's good to have someone else updating the fact check documents and making note of what facts still need checking. Within the [master] document, I have a column that tracks each fact to its checker's initials, whether an intern or producer. You refer back to that person if there are any questions later."
Investigative reporter, producer and independent fact-checker Jonathan Jones (Firestone & the Warlord) argues against documentarians fact-checking their own work. "It's a basic lack of understanding about what fact-checking is; it's different from reporting," he maintains. "In best-case scenarios, fact-checking should be done by someone somewhat independent of the piece, who is going to talk to the primary filmmaker or reporter and assess the credibility of sources. That's hard to do as the primary reporter. I think the biggest fear in journalism is confirmation bias and not being able to see or catch errors because you're so close to the material."
Like Capps, Jones emphasizes the importance of a methodical workflow. "If the filmmaker has organized notes and a somewhat full script, a fact-checker can relook at the material, for the larger fact-checking issues: Does it feel balanced and fair? Are we missing something? What are we particularly concerned about? Often that question will prompt you to do additional reporting. The basis of the film can be the hardest piece to nail down because you have to look at the totality of your gathered evidence."
Solo fact-checker for The New York Times Op-Docs, Lindsay Crouse has checked every Op-Docs piece since the debut of the online series. "Fact-checking at Op-Docs occurs in two different areas," explains Crouse. "One is the general nuts and bolts of any fact-checking that occurs at the Times. We consider ourselves to be the media institution of record, so everything we publish goes through that process, whether by our own reporters or by outside contributors. We want to make sure [the pieces] can effectively become part of the public record and everything is verifiable and true. That means we run formal corrections on everything. We have viewers and readers who are our eyes and ears. They write in on anything they think may not be true, whether the spelling of someone's name or something that did not, in fact, happen. My job is to first pre-empt those kinds of corrections. I vet names, ages, any objective facts, by going to the source and making sure that everything is factually true.
"The second stage," Crouse continues, "is a deeper fact-checking for transparency, and it's new ground for the Times for video, especially by external contributors. Sometimes I get on the phone with the filmmaker and ask them the who, what, when, where. Make sure no one was paid. What dates it was filmed. Is anything in the film fictional or staged? Most films come to us when they have already been shot, so I am mostly retracing the filmmaker's work. I make sure that everything presented in filmic form, in the filmmaker's depicted recording, actually occurred in that chronology, that nothing has been mashed up for the sake of telling a better story. For photos, it's making sure that the image is correct, that it is a picture of the intended person and not an example photo. I ensure transparency, in terms of not necessarily plagiarism but the veracity or originality of story. If, for example, a story has already been reported by the Times, I check that the documentarian's reporting adds to the story, rather than just repackaging something original by another reporter. Then, with the filmmaker, I double-check with experts and other sources."
The New Yorker has a team of 17 fact-checkers, headed by Peter Canby, not including three Web-checkers who fact-check videos. Says Canby, "You deconstruct a piece mentally and try to break it down into its sources. You try to figure out where everything came from, who's quoted, to whom things are attributed, what reports are mentioned, what incidents are described, who was spoken with and who wasn't spoken with. If there are long swatches of narrative with no source, you look at those with special attention because you have to figure out where all that came from. Then you ask the writer who to talk to, who not to talk to or be careful with, what sources you can use. The writer sends us notes and transcripts, which we read before making calls to sensitive subjects—say, somebody who the writer has coaxed information out of and who may get feisty and start denying things that you know he said."
News corporations are armed with tools for best practices. LexisNexis is used extensively. "We've been very successful at finding people that other people are having trouble finding," says Canby. "Nexis is almost startlingly invasive. You can find everything, including whole social security numbers and things that shouldn't be there but are. We have access to Lexis and PACER, which give us entire legal histories, and a commercial service called Leadership Directories, which gives us an amazing directory of Washington phone numbers. But it doesn't help you if you're trying to reach the White House Communications Office and nobody is willing to answer any questions. We have People Locator, which is expensive. Celebrity Service gives us names of agents for even minor celebrities. A number of years ago, I tried to work out a list of the most consulted sites. Within six months, most of those sites had been superseded by something else. Ultimately, people pass along sources by word of mouth."
For documentarians who would like to work with Frontline, creating hard-hitting investigative works for the news network, Managing Editor Andrew Metz recommends "setting out with the ambition to understand and cite multiple perspectives and sources of your subject, to spend time in the investigative process." At Frontline, the documentarian is considered a journalist. Metz describes the process as one that works with "people who are reporters, who approach things with a tough, skeptical, open mind, determined to do extensive research and reporting, and not attached to any particular point of view, organization or advocacy. We go towards the complexity of subjects that appear confusing and are hard to decipher. That's what you're after when you produce on Frontline. You accumulate so much source material and such a deep understanding of your subject that it allows you to build a fact-checking process for yourself and for our external review. We require sound, narration and picture to be vetted before it comes to us. That doesn't mean that when we look at it we’re not going to raise more questions. The goal is to be fair and accurate. We just want to avoid the rare instances where we end up having to do our own reporting, from scratch, with entirely new sources."
Metz emphasizes the publicly available guidelines that journalists and filmmakers who work with Frontline are expected to follow. "These long-standing, guiding principles continue to push and guide us. We do long-form documentary film, but at heart we are journalists. Documentary film is a wide, big family and some people don't identify as journalists. They use the tools of journalism but don't come after the endeavor as journalists. Frontline is about fairness, rigor, reporting against your reporting, examining contrary information thoroughly."
Capps notes that filmmakers without institutional support have far fewer fact-checking resources. "Conversations about resources for freelancers need to continue, whether it's legal review resources or who to go to when you don't have organizational backing." For filmmakers like Capps, who often fact-check independently, using the right tools is key. "For stories about the criminal justice system or legal cases, I have gone to courthouses to get physical records," recounts Capps. "Deposition videos taken under oath are powerful visual tools in investigative pieces. Even if you don't use them, transcripts of depositions and court testimony can be helpful in backing up facts."
Fact-checking is also a vital protection against liability and may ensure protection for the documentarian and her sources when the subject matter is sensitive. "I did a film about minors facing abuse in the criminal justice system," Capps recalls. "It can be difficult to have those conversations. But a lot of the stories that we told and the claims made were backed by legal documents and public records." Fact-checking can begin sometimes as early as the conception of the idea. "Even before you begin your film or before an interview, it's useful to think about what documents or research will help to fact-check what the interviewees are saying or what the narration is going to be," says Capps.
"The fact-checker should be on board by the time the filmmaker is getting a cut together," advises Jones. "I discuss the subject with the filmmaker, maybe watch a cut and set up a schedule of delivery dates. You don't want to be bugging the filmmaker every time you hit a road block on your facts or something needs to be changed. I get a feel for the script and its fairness and balance. Then I do what I call 'sweeps.' I underline all factual assertions made in the script, and begin to fact-check by accessing the source material. With Frontline, it's between one to three months [of fact-checking] before a piece locked. You are basically working backwards from when you want to be locked."
Op-Docs videos initially come to Crouse in rough-cut stage. "The best someone can possibly do when they come to me is have a script that's fully annotated in terms of sourcing for each part of an interview," she notes. "'This fact came from this Times article,' for example." Crouse recommends a thorough consulting of experts in the documentarian's subject field. "I reach out to Harvard or Princeton or Yale to find a scholar who is familiar with the topic. The important thing in researching and vetting is to try to figure out the other side of the story ahead of time. See who is going to disagree with your information. Go find that person. You don't want to publish it, have them discover it later and then realize that you have no idea where something came from. The best-case scenario for me is to not discover anything in my fact-checking that the filmmaker did not already know." Crouse cautions the documentarian submitting to Op-Docs in allowing any singular stance to infiltrate the material. "Sometimes we discover that perhaps a filmmaker was a little too credulous or a little too advocacy-based without thinking enough about the other side of the story for us to want to run it and endorse their perspective," she says. "Of course, Op-Docs films have a strong point of view. Filmmakers don't have to create a balanced perspective, but they do have to show that they are aware and engaging with all the other sides to the story."
Factual errors are sometimes the product of repetition and belief without verification. "Many news reports and academic studies say that the Firestone plantation in Liberia is a million acres," Jones observes. "It's completely wrong but keeps getting repeated in publications. So even though you can find sources backing up your assertion that it is a million acres, it's an error. A fact-checker would hopefully be able to prevent errors like that. The more circumspect you are of sources, the more sources you need. A primary source is a much better source than a bunch of secondary sources, where you need additional steps to verify that the information is true. Depending on your time and resources, your fact-checker can do additional interviewing and site visits, becoming a kind of reporter to fill in the gaps. It also takes a lot of coordination with the filmmaker so that you don't mess up the process for them."
Canby, too, stresses the importance of primary sources. "The key to good journalism is original sources, as much as possible. The more you have secondary and tertiary sources, the less original and interesting and thorough a piece is. We find that really well reported pieces are ones where reporters persist for the right answer from somebody who’s actually involved in their subject."
For the documentarian to be prepared for self or institutional review, best practices include organization, appropriate naming conventions, security and storage. At Frontline, highly sensitive material is rote and documentarians are required to provide all sourcing. Metz works in close conjunction with a lawyer. "He reviews all films long before broadcast so that any issues that he might cite or question may be addressed," says Metz. "Filmmakers must be prepared for external and legal review. We use encryption and other ways of getting sensitive material [from the filmmaker]. Not every film requires that. In all cases, we have access to all interviews and transcripts. Any underline reporting, documentation, video, depositions and visual elements are open for review."
The relationship between fact-checker and documentarian may become tenuous when ownership and boundaries come into question. "As a filmmaker doing point-of-view journalism, sometimes a fact-checker can find things that don't fit that point of view," observes Jones. "If I'm saying that the sources you are using to make an assertion are not correct or strong enough, that inevitably creates some tension. An important part is also presenting ways to fix it. Ultimately, it's up to the filmmaker whether they want to accept, not accept or modify your changes."
"We don't want to undo the reporter's work," notes Canby. "We're very mindful of not wandering into things and creating a mess that wasn't there—for example, have [a subject] deny something that the person actually said. There's a lot of conflict built into [checking] because you're taking somebody’s material and combing through their sources. They may see things one way and you may gradually come to see that maybe it's not quite the way you see it in there. When we see something important, we focus on it and make sure that people are willing to hear us out."
At a news network, final editorial say usually lies with the organization. There is often a liability waiver and set rules that the documentarian must subscribe to. "The best filmmakers and journalists recognize that a rigorous process where things are examined, makes their work and Frontline stronger," says Metz. "It makes it harder, but the most experienced people recognize that we're all pushing in the same direction."
Says Crouse, of Op-Docs, "Sometimes we have had disagreements [with filmmakers] about sources and the extent to which a source is sufficient, and we have killed a few pieces. If I don't feel that I can answer questions about the veracity and relevance of a piece, then we can't publish it. We have had filmmakers who aren't able to prove that their piece is accurate to the point where I am comfortable publishing it with potential implications that might occur."
Capps advocates communication as key to working with news corporations. "Have conversations before you even get started when you're producing or creating something for an organization," says the filmmaker. "Be very clear what their expectations, formats and deliverables are going to be. If it is investigative, and you have never done anything like that before, consult with an investigative journalist before starting the project. Don't be afraid to talk to other people and see what they have done in the past."
The work of fact-checking is more complex than perusing information that may be boiled down to data. Opinion and fact may overlap. Argument and images may miss vital context.
"It's important not to misconstrue fact as opinion," Capps notes. "The best way is to report counterpoints, against your story. Sometimes you discover things that don't necessarily accommodate your assumptions but take you in a different, more interesting direction. People repeat recollection as fact all the time, but you can't represent memory as fact unless it can be backed up. You have to be accountable to your viewers, the people whose stories you are telling and the people you are working for, to be trusted in the future."
Fact-checking, to Canby, is largely critical thinking. "Sometimes people think that you check the facts but you don’t check opinions," says Canby. "We wind up with long narratives of opinion based on a certain understanding of facts. Part of the challenge of journalism and documentary is to be sufficiently entertaining but mindful of other points of view. With all the fake news now, that's very important. You can be sensitive to [alternate opinions] and address them in ways that make people feel included in your argument. Which is not to say that you have to give them any more credit than they deserve. But you listen to people and their trust issues and beliefs, regardless of whether you believe them yourself. In the Alex Gibney series The New Yorker Presents, we had a short documentary [Protect and Serve] about the brutality of the rogue police force in Albuquerque, New Mexico. When the documentary film people went to Albuquerque, they focused on victims and didn't really look at the way these incidents were looked at by law enforcement. We felt that it was essential to have law enforcement at least give their account of what happened. That's the kind of thing we typically look for."
Filmmaker Jesse Moss, who directed Protect and Serve, describes the process of being fact-checked by The New Yorker as similar to "a medical exam—uncomfortable but probably necessary, although there is an element of subjective interpretation. Prior to working with Jigsaw and going through The New Yorker's fact-checking process, I didn't have a formal fact-checking process of my own. I didn't consider the work I was doing to be investigative reporting; it was more observational documentary. That said, if somebody said something to me, on or off camera, that I thought to be untrue or needed clarification, I would ask them or do follow-up research on my own."
Moss acknowledges the importance of external, rigorous review but outlines the difficulties of making corrections in post-production and holding on to often opposing footholds of objective journalism and the complex contexts of stories. "I really value The New Yorker’s assessment of the rough cut of the documentary—it was impressive and helpful." But he also notes, "When you're checking your documentary, as opposed to nonfiction prose or journalism, you have a different set of rules and formal options. You can't make parenthetical statements in context, as you can in prose. You can't say, 'This claim was disputed' or 'Actually, the truth is this.' What are your options? You can cut out the statements that the fact-checker has flagged as factually incorrect. You can, as they propose, insert a text card, correcting your subject. But even they recognize that, stylistically and editorially, that isn't a good solution."
Metz stresses context and clarity when fact-checking and presenting opinion. "A person is certainly entitled to express their point of view. However, there are murky areas. In those cases, are you being transparent enough in the film, that viewers understand what they're seeing and hearing? [Frontline’s] responsibility is that viewers can ultimately have enough information, context and perspective to make up their own minds and come to their own conclusions."
Crouse offers an example. "At the Times, we did a film called Separatist, featuring the arguments of a white supremacist. There were many factual inaccuracies in what he was saying. But they were his beliefs. We want to showcase a lot of different beliefs as long as it is clear that we at the Times do not endorse this belief. It's understood that our audience knows that these beliefs exist and that to some people, are facts. We published the piece because we felt that it was clear that we were showing that this was a person's opinion. You make it clear that you are not necessarily endorsing the perspective presented. Presenting both sides can be a good way to do that. Also, as soon as you have a text card, that's your own opinion and should be a fact that can be verified."
The best practices of fact-checking ultimately lie in understanding the crossover between documentary and journalism and that a viewer may trust in the veracity of both, equally. As Crouse puts it, "The strongest documentary film will meld journalistic underpinnings with top-notch, artistic storytelling." Jones notes, "Some of the best journalism has been done by filmmakers who were simply trying to get their heads around a subject. Investigative journalists are good at finding out things, but their storytelling may be dry. As long as a filmmaker has experience working on complex issues, they are able to understand nuance and don't have a meltdown every time something must be changed. Fact-checking needs confidence. You need to trust that what you're doing is good and that the goal is the same for everyone: to make the best film possible."
Nayantara Roy is a writer and journalist based in New York City.