Last weekend I was trying to pull together sources for an essay and kept finding “fact check” pages from factually.co. For instance, a Kagi search for “pepper ball Chicago pastor” returned this Factually article as the second result:

Fact check: Did ice agents shoot a pastor with pepperballs in October in Chicago

The claim that “ICE agents shot a pastor with pepperballs in October” is not supported by the available materials supplied for review; none of the provided sources document a pastor being struck by pepperballs in October, and the only closely related reported incident involves a CBS Chicago reporter’s vehicle being hit by a pepper ball in late September [1][2]. Available reports instead describe ICE operations, clergy protests, and an internal denial of excessive force, but they do not corroborate the specific October pastor shooting allegation [3][4].

Here’s another “fact check”:

Fact check: Who was the pastor shot with a pepper ball by ICE

No credible reporting in the provided materials identifies a pastor who was shot with a pepper‑ball by ICE; multiple recent accounts instead document journalists, protesters, and community members being hit by pepper‑ball munitions at ICE facilities and demonstrations. The available sources (dated September–November 2025) describe incidents in Chicago, Los Angeles and Portland, note active investigations and protests, and show no direct evidence that a pastor was targeted or injured by ICE with a pepper ball [1] [2] [3] [4].

These certainly look authoritative. They’re written in complete English sentences, with professional diction and lots of nods to neutrality and skepticism. There are lengthy, point-by-point explanations with extensively cited sources. The second article goes so far as to suggest “who might be promoting a pastor-victim narrative”.

The problem is that both articles are false. This story was broadly reported, as in this October 8th Fox News article unambiguously titled “Video shows federal agent shoot Chicago pastor in head with pepper ball during Broadview ICE protest”. DHS Assistant Secretary Tricia McLaughlin even went on X to post about it. This event definitely happened, and it would not have been hard to find coverage at the time these articles were published. It was, quite literally, all over the news.

Or maybe they’re sort of true. Each summary disclaims that its findings are based on “the available materials supplied for review”, or “the provided materials”. This is splitting hairs. Source selection is an essential part of the fact-checking process, and Factually selects its own sources in response to user questions. Instead of finding authoritative sources, Factually selected irrelevant ones and spun them into a narrative which is the opposite of true. Many readers will not catch this distinction. Indeed, I second-guessed myself when I saw the Factually articles—and I read the original reporting when it happened.

“These conversations matter for democracy,” says the call-to-action at the top of every Factually article. The donation button urges readers to “support independent reporting.”

But this is not reporting. Reporters go places and talk to people. They take photographs and videos. They search through databases, file FOIA requests, read court transcripts, evaluate sources, and integrate all this with an understanding of social and historical context. People go to journalism school to do this.

What Factually does is different. It takes a question typed by a user and hands it to a Large Language Model, or LLM, to generate some query strings. It performs up to three Internet search queries, then feeds the top nine web pages it found to an LLM, and asks a pair of LLMs to spit out some text shaped like a fact check. This text may resemble the truth, or—as in these cases—utterly misrepresent it.

Calling Factually’s articles “fact checks” is a category error. A fact checker diligently investigates a contentious claim, reasons about it, and ascertains some form of ground truth. Fact checkers are held to a higher evidentiary standard; they are what you rely on when you want to be sure of something. The web pages on factually.co are fact-check-shaped slurry, extruded by a statistical model which does not understand what it is doing. They are fancy Mad Libs.

Some times the Mad Libs are right. Some times they’re blatantly wrong. Some times it is clear that the model simply has no idea what it is doing, as in this article where Factually is asked whether it “creates fake fact-checking articles”, and in response turns to web sites like Scam Adviser, which evaluate site quality based on things like domain age and the presence of an SSL certificate, or Scam Detector, which looks for malware and phishing. Neither of these sources has anything to do with content accuracy. When asked if Factually is often incorrect, Factually’s LLM process selected sources like DHS Debunks Fake News Media Narratives from June, Buzzfeed’s 18 Science Facts You Believed in the 1990s That Are Now Totally Wrong, and vocabulary.com’s definition of the word “Wrong”. When asked about database safety, Factually confidently asserts that “Advocates who state that MongoDB is serializable typically refer to the database’s support for snapshot isolation,” unaware that Snapshot Isolation is a completely different, weaker property. Here’s a Factually article on imaginary “med beds” which cites this incoherent article claiming to have achieved quantum entanglement via a photograph.

The end result of this absurd process is high-ranking, authoritative-sounding web pages which sometimes tell the truth, and sometimes propagate total garbage. Factually has constructed a stochastic disinformation machine which exacerbates the very problems fact-checkers are supposed to solve.

Please stop doing this.