Pollster Nate Silver on Monday savaged the analytics behind a recent Washington Post column claiming President Biden was being treated just as badly, or worse, by the media than former President Trump.
In the piece published last week, liberal columnist Dana Milbank complained about Biden’s media coverage being overly tough and implored journalists to do “soul-searching” and “think about what it is we’re delivering to people.”
In a series of tweets, Silver argued the piece’s “sentiment analysis” measuring the positivity and negativity of particular articles written about Trump and Biden was “complete crap,” and gave examples to show how the data could be skewed more positively or negatively than it should have been.
“To this good thread explaining why the ‘sentiment analysis’ cited in the [Dana Milbank] WaPo article this weekend is complete crap—the analysis was used to make the claim that the press is just negative toward Biden as Trump—I’ll also add a couple of comments based on their data,” Silver wrote.
Silver listed the top articles that the algorithm found were the most favorable towards Biden, and argued that they were actually “totally random.” He noted that there were a few stories about the stock market, and some that didn’t even have anything to do with Biden, such as a story about the assassination of Haitian President Jovenel Moïse.
“What’s probably happening here is that words like ‘high’ and ‘rise’ are viewed as favorable by the algorithm, even if they’re taken completely out of context (e.g. ‘higher taxes’ or ‘stock futures rise’),” Silver wrote.
Silver pointed out that the same thing appeared to be the case when looking at the articles that were listed as most negative towards Biden, citing stories about stocks, commodity prices and approval ratings.
Silver added that polling-driven stories didn’t reflect bias, but that they were naturally considered negative if it was about dropping poll numbers as opposed to rising ones.
Biden’s poll numbers have been steadily declining throughout his first year in office, likely explaining one aspect of the high amount of “negative” articles pertaining to him.
Silver pointed to an article on his own site, FiveThirtyEight, to show how the algorithm classified it as “neutral” to Trump even though the focus of the story was how the former president “poses big threats to democracy.”
“Here is a story we did on Trump/GOP efforts to undermine democracy, for example. Exactly the sort of story that [Dana Milbank] says there should be more of (I agree),” Silver wrote. “How was the story scored? As being neutral to Trump (-0.0176). Now I think the story was fair and accurate (it’s accurate to report that Trump poses big threats to democracy). Still, if this is the sort of story you want more of, the algorithm isn’t helping to ID them.”
“Designing good algorithms is hard, but this is an especially bad one. And as a news consumer, you should be extremely wary of statistical methodologies you don’t understand but that confirm your priors,” Silver wrote.