Humanist Discussion Group, Vol. 32, No. 345. Department of Digital Humanities, King's College London Hosted by King's Digital Lab www.dhhumanist.org Submit to: email@example.com Date: 2019-01-15 20:42:34+00:00 From: Ken Friedman
Subject: My Thoughts on Wikipedia, part 2/3 [Continued from part 1; unfortunately it has proven necessary to send the note out in 3 parts! --WM] The problem that bothers me can be stated in a simple way. There should be a way to balance the two conflicting aspects of Wikipedia policy. A broad contributor base for an encyclopaedia should include some of the experts who also write for paper encyclopaedias and reference works, and it should certainly include the kinds of experts who write for peer-reviewed journals and teach in the subject field. With several hundred major topic areas and several thousand sub-topics, a reference work like Wikipedia could benefit from at least 5,000 active expert contributors to develop and polish the content of 5,700,000 articles. Before wrapping this up, I'd like to address a few specific issues that have come up in queries I sent to a number of people and to the Digital Humanist discussion list. -- Critical Reading Many comments suggest that scholars tell students not to trust Wikipedia because it is not peer-reviewed. This is not the case. The problem is not a case of peer review, but of reliability. As a teacher, author, reviewer, and journal editor, I accept citations to many reliable sources that are not subject to peer review. Well researched and carefully reported newspapers such as The New York Times, The Guardian, The Economist, Svenska Dagbladet, Politiken, and many others qualify. So do government reports, reports from NGOs and treaty organizations such as the UN or the OECD. So do careful documents from well organised research centres. There are many more useful sources. The need to read critically is the key for all documents. -- Transparency Those who use Wikipedia frequently raise the issue of transparency. In my view, the question of transparency involves three somewhat different questions. 1) The Transparent History of Each Article in Wikipedia The first is the assertion that the completely documented history of any article makes editorial changes completely transparent. Each state of every article is preserved in the article history. This is true on the face of it, but false or possibly meaningless in reality. A check as I write these words show that there are 5,783,747 articles in the English language Wikipedia. There have been 873,775,490 edits to achieve this number of articles. This means an average of 151 archived states per article. In reality, some articles have many more archived states, while other have far fewer. The sheer mass of archival data makes the idea of reasonable transparency impossible. The Library of Babel in Jorge Luis Borges's story contains a universal library with every possible state of every possible book. However, it also contains every possible state of many useless books, some containing only one letter on a single page, others containing pages full of random characters with two pages of important information buried in the middle, and so on. The sheer mass of data hides the potentially useful information. While every article contains better states and worse, it would take far too much work for any ordinary user to locate the best article states and restore them. Even were this to happen, however, nothing would ensure the stability of the restored article. In fact, a process known as edit wars demonstrates the frequent back-and-forth revision of articles between states where competing editors have different views on what an article should be. Many of the processes involving edit wars have now been automated, handed over to artificial intelligence bots Sample, Ian. 2017. 'Study reveals bot-on-bot editing wars raging on Wikipedia's pages.' The Guardian. Thursday, February 23, 2017. https://www.theguardian.com/technology/2017/feb/23/wikipedia-bot-editing-war- study Tsvetkova, Milena, Ruth GarcÃa-Gavilanes, Luciano Floridi, and Taha Yasseri. 2017. "Even good bots fight: The case of Wikipedia.' PLOS ONE. February 23 2017. DOI: https://doi.org/10.1371/journal.pone.0171774 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0171774 2) Transparency in the Editorial Process While the talk pages supposedly provide transparency in the editorial process, many editors do not engage in conversation on their edits. They simply edit, and in some cases they refuse to respond to queries on the reasoning behind their edits. Some experienced or senior editors argue from personal authority as Wikipedians. Others rely on policy rulings, whether or not they have interpreted policy correctly. In some cases that I have seen, the knowingly reject correct or improved information on the basis that it does not conform to Wikipedia standards. In essence, they are saying, 'Yes, this is true. But I'm rejecting it until you go back and do it right.' In many cases, the kind of conversation the appears on talk pages is curt or rude. It is true that these kinds of conversations are often seen in academic circles where reviewers can be harsh. There are nevertheless several major differences between revising a journal article and revising or improving a Wikipedia article. The first is the fact that a journal article counts as a publication, while a Wikipedia article does not. That might not be a crucial difference to most contributors -- after all, everyone who contributes to Wikipedia does so knowing that the contribution that he or she makes will become part of a larger collective work. The second difference is far more serious. Once one goes through the pain of the review process, a journal article is published in stable form. A Wikipedia article is never stable, and constant change and revision may swiftly delete a contribution and all the work involved. The third difference is also significant. This is the fact that one can usually discuss a review with editors and -- through the journal -- with reviewers. In Wikipedia, this process is often flawed, problematic, or absent. -- Coverage, Notability, and Discouraged Contributors While Wikipedia has extraordinarily wide coverage, there is reason to question some of the gaps and exclusions. Wikipedia has a carefully constructed notability policy -- but many editors don't follow the policy. Instead, they frequently delete articles based on their own lack of knowledge. A well-known case involves the decision to remove a biography of Nobel laureate Donna Strickland on the basis that she was not notable enough. Even before winning the prize, Strickland was a highly cited scientist. Her work fit within Wikipedia Notability policies. The editor who removed her simply lacked the knowledge -- and he apparently lacked the tools and skill to determine that Strickland was, in fact, a well known scientist. My suspicion is that many Wikipedia editors and administrators make seat-of-the-pants decisions without knowing what they don't know -- and in many cases, they justify their quick actions by asserting that they are too busy to take the time that a proper investigation requires. Every experienced journal editor and every skilled reviewer knows that determining the validity or reliability of an article often takes several hours of work. This work involves careful reading and checking on the subject field. Wikipedia coverage on scholars, researchers, and scientists suffers from significant gaps and omissions. A specific study describes this problem: Samoilenko, Anna and Taha Yasseri. 2014. "The distorted mirror of Wikipedia: a quantitative analysis of Wikipedia coverage of academics.' EPJ Data Science. (2014) Vol. 3, No. 1. DOI: https://doi.org/10.1140/epjds20 https://link.springer.com/article/10.1140/epjds20 Samoilenko and Yasseri write, 'We found no statistically significant correlation between Wikipedia articles metrics (length, number of edits, number of incoming links from other articles, etc.) and academic notability of the mentioned researchers. We also did not find any evidence that the scientists with better WP representation are necessarily more prominent in their fields. In addition we inspected the Wikipedia coverage of notable scientists sampled from Thomson Reuters list of âhighly cited researchers'. In each of the examined fields Wikipedia failed in covering notable scholars properly. Both findings imply that Wikipedia might be producing an inaccurate image of academics on the front end of science.' One of the odd issues with rigid and ignorant enforcement of Wikipedia's notability policies is the fact that there is no shortage of space in Wikipedia. Unlike a paper reference book, Wikipedia has room for every serious, well- written, properly referenced biographical entry that any contributor might write. Given that Wikipedia servers house 5,783,747 articles in the English alone plus 873,775,490 archived edits, there should be no problem housing high quality articles at any length on any topic interesting enough for a serious contributor to write it. Since 1901, 904 individuals and 24 organizations have won the Nobel Prize or the Prize in Economic Sciences. Just over 70 mathematicians have won the Fields Medal. One imagines that each of these deserves a serious article covering their lives, achievements, and the influence of their work. Nevertheless, but many have a Wikipedia entry with only a paragraph or two of basic facts. Surely an encyclopedia that can archive nearly 900,000,000 edits in English alone can create a notability policy that yields serious articles on any one notable enough to warrant a specialist biography. With over 45,000,000 articles total in 293 languages, the complete archives must be far more extensive. If English language articles get an average of 151 edited states each, it is probably fair to allow for an average of 100 edited states even for smaller language editions. A back-of-the-envelope estimate would suggest that all Wikipedia editions together must archive at least 4,500,000,000 articles. This being the case, one must wonder why any editor or administrator is stingy with space on the basis of notability. I know many people who are noted contributors to the fields in which I do research. While they are not celebrities or public names, they are all well within the notability criteria. Similar problems affect topical articles on minor sports, serious but extremely small research fields, and so on. Despite the need for in-depth articles on people and topics that lack coverage, attempts to contribute to articles -- or to expand available information in current articles -- seems to be a problem in Wikipedia. This is in part a reflection of the tendency to favour earlier article states over serious, good- faith improvements. But few of the people who can write high quality specialist biographies want to argue with entrenched Wikipedia editors and administrators. Instead, they give it a try, make a few efforts at explaining their work, become discouraged, and leave. I recognise that dedicated Wikipedians say this should not be the case. Nevertheless, it is the case. The situation is probably exacerbated by bot editors programmed to protect articles by reverting changes. Bots do not explain their actions. They simply act. -- Expert Authors In the past, it was my view that Wikipedia did not have many expert authors. I was mistaken. It seems that Wikipedia has a great many expert authors. Even so, many of these authors participate briefly and leave. Their expertise vanishes with them as articles revert to a mediocre consensus. There are some expert authors who are deeply committed to Wikipedia and willing to go through the struggle of Wikipedia debate to preserve improved article states. Nevertheless, a large review of articles in any field shows that expert articles are in the minority. While expertise is an issue, this is not a simple debate between credentialed expertise and careful reference writing. Anyone who works with good journalists or with copy editors who work across several fields understands that people who know little about a field can nevertheless contribute significantly to high quality articles. The problem with Wikipedia is that too many non-experts make problematic judgements on articles where they hold ill-informed opinions. Wikipedia contains a useful essay on the issue of expert editing and retaining more excellent writers with subject field knowledge. https://en.wikipedia.org/wiki/Wikipedia:Expert_retention [continued in part 3/3 _______________________________________________ Unsubscribe at: http://dhhumanist.org/Restricted List posts to: firstname.lastname@example.org List info and archives at at: http://dhhumanist.org Listmember interface at: http://dhhumanist.org/Restricted/ Subscribe at: http://dhhumanist.org/membership_form.php
Editor: Willard McCarty (King's College London, U.K.; Western Sydney University, Australia)
Software designer: Malgosia Askanas (Mind-Crafts)
This site is maintained under a service level agreement by King's Digital Lab.