One morning last month, Seymour Hersh set off to buy a newspaper. The reporter walked for 30 minutes, covered six blocks of his neighbourhood, Georgetown in Washington DC, and didn’t see a single sign of life. No newsstands on street corners selling the glossies and the dailies. No self-service kiosk where you can slide in a dollar and pull out a paper. “Finally, I found a drugstore that had two copies of the New York Times in the back,” Hersh recalls. He bought one for himself. He can’t help but wonder whether anybody bought the second.

Hersh was born in Chicago in 1937, the year the Hindenburg airship blew up and the aviator Amelia Earhart disappeared over the Pacific. That makes him a man of hot metal, the media’s ancient mariner, with metaphorical newsprint on his fingers and a cuttings file that reads like an index of American misadventure. Hersh has been a staff writer at the New York Times and the New Yorker. He’s broken stories on Vietnam, Watergate, Gaza and Ukraine. But the free press is in crisis, newspapers are in flux and investigative journalism may be facing a deadline of its own. “I don’t think I could do now what I did 30, 40, 50 years ago,” says the now 88-year-old. “The outlets aren’t there. The money’s not there. So I don’t know where we all are right now.”

Cover-Up, a new documentary from Laura Poitras and Mark Obenhaus, at least reminds us where he’s been, rewinding the clock to chart Hersh’s galvanic, obstreperous course through American journalism. It’s a film that gives us the newsman’s greatest hits, with particular focus on his exposés of the 1968 My Lai massacre of unarmed South Vietnamese civilians by US army soldiers, and the 2004 Abu Ghraib prison torture scandal, while also acknowledging his occasional missteps and contentious reliance on unnamed single sources.

‘His anger was mostly directed at me’ … Oscar-winning documentary-maker Laura Poitras. Photograph: Linda Nylind/The Guardian

Along the way, it whips up a vivid sketch of Hersh himself: a prickly hothead who makes enemies in the editorial office and the Oval Office alike. “This Seymour Hersh is a son of a bitch, probably a communist agent,” President Nixon tells Kissinger on an unearthed White House tape. “But,” he adds grudgingly, “he’s usually right.”

Initially, Hersh had no interest in participating in the documentary. He says Poitras chased him for years. He’s still not sure what made him change his mind. Hersh had collaborated with Obenhaus several times in the past. But Poitras was a different kind of director and came at him from disconcerting new angles. “Mark and I, when we interview people, we just ask them what happened. Whereas Laura would ask me, ‘And how did you feel?’” He snorts at the memory. “That’s not a guy’s question. That’s more psychoanalytical.”

The shoot was a chore and his patience ran short. Hersh hated letting cameras snoop around his office, which was heaped with legal notepads and Rolodexes. He guarded his list of contacts like a dragon guards its treasure. It was only a matter of time before his mistrust boiled over and he threatened to quit the film altogether.

Poitras won an Oscar for Citizenfour, her documentary on the NSA whistleblower Edward Snowden, whose revelations were published in the Guardian. Her previous picture, All the Beauty and the Bloodshed, about the artist and activist Nan Goldin, won the Venice Golden Lion. So she’s used to working with volatile subjects to the point where she relishes the drama they bring. Her main thought, when Hersh briefly bailed on the project, was a sense of relief that he had considerately done so on-camera.

Frontline … an image from Cover-Up of US troops in Vietnam.* * Photograph: Ron Haeberle/Courtesy of Netflix

“Sy has a pattern of quitting,” Poitras tells me. “He quit the New York Times, for instance. So we knew it wasn’t inconceivable that he would quit the movie. It could well have happened. But I thought he was invested, I thought we’d get through it. His anger was mostly directed at me. We hit a small bump. He quit the movie. But 24 hours later he was back.”

The film is about Hersh but it’s also about journalism. It shows the news media’s in-built contradictions and its flawed business model. Cover-Up suggests that the best investigative reporters are natural outsiders who rarely last long inside risk-averse institutions. Editors and management might claim they want good stories, but in practice they fear them, because scoops tend to cause trouble and involve a big fight. Tellingly, the film includes an archive clip of Hersh speaking on stage in the 1970s. He says: “What we have here in America is not so much censorship as self-censorship by the press.”

If that was true then, Poitras says, it’s doubly so today. She’s alarmed not just by Trump’s authoritarian push to stifle a free press but by the alacrity with which several media giants have already rolled over. Two major networks – ABC and CBS – recently agreed to settlements with Trump instead of fighting the case out in court. The Washington Post’s owner, Jeff Bezos, has ordered the paper to focus less on politics and more on promoting “personal liberties and free markets”.

The situation is parlous, Poitras says. “What we’re seeing in the US is the preemptive capitulation of institutions to avoid a legal battle they would have won. That’s shameful. I don’t know how they explain that to themselves. It’s the worst precedent you can possibly set.” She shakes her head. “If institutions aren’t willing to back aggressive reporting, it’s dangerous. We all suffer.”

Shocked by the crowd’s response … Poitras and Hersh at the Venice film festival in August. Photograph: Yara Nardi/Reuters

Times are tough, agrees Obenhaus, Cover-Up’s co-director. The first amendment is under daily siege; it’s the closest to McCarthyism, the era of repression and persecution in the 1950s, that he has witnessed in his lifetime. Throw in the wider issue of an increasingly atomised media landscape and it might amount to the perfect storm.

“There are no gatekeepers on information any more,” says Obenhaus. “The so-called legacy media is so dispersed. And without that centre – that base – it’s hard for good journalism to break through, which means people are increasingly relying on unreliable sources. It troubles me tremendously that the Sy Hersh of today might be writing on Substack or some other platform – and you’d never even hear of them unless the algorithm connected you to their work.”

As it happens, Hersh does write on Substack. The platform suits him because he has a large dedicated readership that will happily pay for his work; also because it allows him to cover the stories he wants, free from editorial interference. “Substack is self-publishing,” Hersh explains. “So it’s a subculture. It works financially. It’s a living, I’m not knocking it. But it’s not like writing for the New York Times.” He doesn’t miss the office politics, the corporate culture and what he sees as the cowardice and compliance of senior editors. What he misses, though, is the thrill of performing on the big stage.

If Cover-Up shows us anything, it’s that the journalist’s role has always been sisyphean. It’s a constant uphill struggle in which every triumph risks being immediately rolled back. Hersh’s career-making exposé of the My Lai massacre, for instance, dismantled the US army’s official version of events and helped swing public opinion against the war in Vietnam. But it led to the conviction of only one out of the 26 soldiers involved, Lieutenant William Calley, whose prison sentence was later commuted by Nixon.

One bonus of a six-decade career is that it gives Hersh a sense of perspective. Investigative reporting is usually thankless and frequently fruitless, he knows. But it still speaks truth to power and remains a vital engine of social change. “The principle of journalism is incredible,” he says. “Just imagine how the world would have been if we didn’t have the journalism we’ve had, and still have today. I don’t like what’s going on in the US. I don’t like the kowtowing to Trump. But there’s still the Wall Street Journal. The New York Times is still a good newspaper. The LA Times used to be but it’s now owned by a Trumpist.” He is referring to Patrick Soon-Shiong. “But journalism matters,” he adds, collecting himself. “It’s necessary.”

In September, Hersh attended the Cover-Up premiere at the Venice film festival. The audience response was so overwhelming, Poitras says, that it moved the newsman to tears. “He’s always been this lone-wolf troublemaker,” she explains. “So maybe he’s not used to being recognised and celebrated. I don’t think he was prepared for it. He was very emotional. He was crying.”

Hersh tells a different story. He claims he was shocked by the crowd’s response. The audience didn’t get the film’s humour, he adds, didn’t laugh at the right moments, then applauded for an unseemly amount of time at the end. “I was embarrassed by that,” he says. “I know they like to measure the length of the applause at these festivals, but come on, enough. I was trying to stop it.”

I like the image of Hersh standing in the aisle and signalling for people to shut up. It feels neatly on-brand for this most awkward of customers: a thorn in the side of every great institution – an unashamed party-pooper, even at his own party.