Judges rely on Wikipedia for their opinions, new study finds
Wikipedia’s administrators – community-elected volunteers who control how an article can be edited – rejected these readers’ requests as unsourced and then “locked” the item so only established publishers could make a change, which only fueled conspiracy theories. Fox News personality Sean Hannity posted the title of a blog post on Truth Social, Donald Trump’s social network: “Wikipedia changes definition of recession then locks page.” Elon Musk tweeted at Jimmy Wales, the co-founder and public face of the project: “Wikipedia is losing its objectivity. Wales sharp musk to an explanation of what happened, adding, “Reading too much nonsense on Twitter makes you stupid.”
This struggle over whether the United States is in a recession is striking because of the high status it bestows on Wikipedia as an objective truth-teller. Here are people who’ve convinced themselves the government is lying to them, and they’re turning to a collaborative encyclopedia to make sure they’re right, like settling a bet at the bar with Guinness World Records or checking a suggested Scrabble word with Merriam-Webster.
Now comes a new paper from MIT and Maynooth University in Ireland, providing further evidence of Wikipedia’s high status, finding that judges routinely rely on its articles not only for background information, but also for basic legal reasoning and the specific language they use in their decisions.
It may go without saying, but extremely rare is the judge who openly credits Wikipedia, so how can scholars say for sure that judges trust it? How to prove not just correlation (it sure looks like the judge was using Wikipedia) but causation (this decision could have been written that way only after the judge read Wikipedia)?
One way might be to introduce a subtle error that serves as a marker – if a party in a case has a strange, made-up middle name that only appears in the Wikipedia article and then appears in a judge’s decision, the proof would seem pretty clear. (Cartographers trying to prevent their maps from being copied are known to invent a street name to catch the culprit.) However, the researchers did not want to introduce an error on purpose. Perhaps you could examine the judges’ computers for sites they have visited or conduct interviews? But the legal system is not exactly a petri dish, designed for unimpeded in-depth study. Instead, the researchers proved the link through a randomized control experiment, with Irish judges as involuntary test subjects.
From early 2019 until early 2020, law professors and their students at Maynooth were tasked with preparing 154 Wikipedia articles on influential decisions of the Irish Supreme Court for publication; fortunately, only nine such Wikipedia articles existed at the time. Each author would be assigned a pair of articles. A senior Wikipedia editor guided half of the articles (one of each pair) onto the platform, letting other curious editors know that the massive introduction of articles came from the university. Researchers have taken great care to add formatting to articles so that Google and other search engines notice it quickly. The other articles were withheld from publication. “The only difference between them is that one of them is put on Wikipedia and the other is not, and then you just wait,” said Neil C. Thompson, a researcher at the Computer Science and Artificial Intelligence Laboratory of the MIT and lead author of the study.
Conspiratorial videos? Fake news? Enter Wikipedia, the “good cop” of the Internet
Researchers reported that the 77 published articles instantly found an audience, receiving a total of 56,733 views through January 16 this year. By analyzing court decisions written after the new articles were published, they detected a statistically significant trend. Supreme Court decisions regarding Wikipedia articles saw a 20% increase in the number of times they were cited by judges, compared to cases in which Wikipedia articles were withheld. All of this increase came from lower-level judges, who the authors of the article said were overburdened and lacked the time and resources to conduct formal legal research. Additionally, the researchers found in the decisions a similar spike in the use of certain words and phrases that first appeared in Wikipedia articles – another way of demonstrating causation.
The document does not offer details on how the cases were decided. There is no example of a decision being particularly indebted to a certain Wikipedia article. And the authors go to great lengths to say that they have not found an example of a case wrongly judged – they maintain the accuracy of the articles they have published. Rather, they wanted to prove Wikipedia’s omnipresence in our lives, what they call “knowledge at your fingertips.”
In 2017, Thompson helped lead a similar experience involving chemistry to test whether Wikipedia “doesn’t just reflect the state of the scientific literature, it helps form this.” Pairs of articles were also created, one to be published, the other not. After the introduction of an article on a certain chemistry topic, the researchers noticed that the journal articles reflected the language and conclusions of the Wikipedia account Academic research that was footnoted in Wikipedia articles was also found to be cited more often in later academic publications.
“I think what we learned from the first one is that scientists are like everyone else in society – we all read Wikipedia all the time, right?” Thompson said in an interview. “Some people had this opinion that you look up everyday things on Wikipedia, but when you were doing science and serious stuff, you just used textbooks and stuff like that. And that’s not what that I’ve seen happening around me. And so we said, okay, I think we should look at that effect.
That judges depend on Wikipedia seems more serious, Thompson said, if only for the reason that judges can hold a person’s fate in their own hands. The newspaper’s disdain for this shortcut is palpable, contrasting such behavior with Alexander Hamilton’s description of judges as members of an elite who engage in “long and laborious study to acquire a competent knowledge” of the law. What about the potential, the article’s authors ask, for a party in a legal argument to edit the article on a relevant case to support its argument and persuade the judge? Should justice be allowed to cling to such a thin reed?
In practical terms, the most important finding of the research is not that judges may be vulnerable to self-serving edits of Wikipedia articles, but that we now know for sure that judges trust these articles. By proving this about judges, but also scientists, the authors of the articles help to demystify these priestly classes. They live in our world and use the same resources as us. “I think we in academia are leading that line in terms of, you know, feeling two different ways of Wikipedia in what we say and what we do,” Thompson said. “I absolutely think it’s very important that we be upfront about the fact that we use Wikipedia.”
Armed with this fact, the solution should not be to shame those who use Wikipedia, but rather to make Wikipedia as reliable and inclusive of all parts of society as we all need it to be. It turns out that the outcry over the recessions article led to a revision. While the description of how the United States formally defines a recession has not changed, the article’s introduction now puts a bit more emphasis on the definition of the two quarters, noting that it is ” commonly used as a working definition of a recession”.
VIPs expect special treatment. On Wikipedia, don’t even ask.
Among the researchers’ recommendations for Wikipedia is that experts be enlisted to create and oversee articles on their subject areas, which may seem to challenge the groundbreaking premise of “the encyclopedia anyone can edit” (as I do, for example). But as long as the term “expert” is not tied to the number of university degrees possessed, but rather to the interest, experience and knowledge demonstrated, the idea seems to correspond to the operation of Wikipedia. People who care about a topic — whether it’s war, medicine, or LGBTQ+ issues — are already watching these articles. Campaigns are underway to create stories about women and minorities, who are grossly underrepresented on its pages. Curiously, the first chemistry article helped Wikipedia convince experts to get involved, Thompson said: “We showed in the first article that if you add content and there is a citation to your article, you will have more quotes. So it was an incentive for them.
Although the new articles from the Supreme Court of Ireland were steered to the site by an experienced editor, averting suspicion that someone was trying to make inappropriate and sweeping changes to the draft, at least one article was challenged and proposed for deletion. It was a case described as procedural, related to the question of whether a court could review the constitutionality of a law if the lawsuit had been settled. However, the facts of the case involved allegations of domestic violence and the fact that a Wikipedia article suddenly highlighted events that had long remained obscure. A new editor has posted to Wikipedia demanding the article be removed as it contains “in camera and outdated information about the private life of an Irish citizen”, adding that “these posts have caused particular distress to the individual and they were mostly used to black out the person’s name in the workplace.The Wikipedia community was not convinced: “The Irish Supreme Court decision in 2004 has been public knowledge, apparently for more than 15 years and it’s hard to see how the genie can now be put back in the bottle.If there are genuine privacy issues, this is not the way to go.
And we’re left to ponder the philosophical question of our time (pardon the hyperbole): if a tree falls in a forest and Wikipedia doesn’t mention it, did it happen?