Humanist Discussion Group, Vol. 32, No. 345.
            Department of Digital Humanities, King's College London
                   Hosted by King's Digital Lab
                       www.dhhumanist.org
                Submit to: humanist@dhhumanist.org




        Date: 2019-01-15 20:42:34+00:00
        From: Ken Friedman 
        Subject: My Thoughts on Wikipedia, part 2/3

[Continued from part 1; unfortunately it has proven necessary to send the note 
out in 3 parts! --WM]

The problem that bothers me can be stated in a simple way. There should be a way
to balance the two conflicting aspects of Wikipedia policy.

A broad contributor base for an encyclopaedia should include some of the experts
who also write for paper encyclopaedias and reference works, and it should
certainly include the kinds of experts who write for peer-reviewed journals and
teach in the subject field. With several hundred major topic areas and several
thousand sub-topics, a reference work like Wikipedia could benefit from at least
5,000 active expert contributors to develop and polish the content of 5,700,000
articles.

Before wrapping this up, I'd like to address a few specific issues that have
come up in queries I sent to a number of people and to the Digital Humanist
discussion list.

-- Critical Reading

Many comments suggest that scholars tell students not to trust Wikipedia because
it is not peer-reviewed. This is not the case. The problem is not a case of peer
review, but of reliability.

As a teacher, author, reviewer, and journal editor, I accept citations to many
reliable sources that are not subject to peer review. Well researched and
carefully reported newspapers such as The New York Times, The Guardian, The
Economist, Svenska Dagbladet, Politiken, and many others qualify. So do
government reports, reports from NGOs and treaty organizations such as the UN or
the OECD. So do careful documents from well organised research centres. There
are many more useful sources.

The need to read critically is the key for all documents.

-- Transparency

Those who use Wikipedia frequently raise the issue of transparency. In my view,
the question of transparency involves three somewhat different questions.

1) The Transparent History of Each Article in Wikipedia

The first is the assertion that the completely documented history of any article
makes editorial changes completely transparent. Each state of every article is
preserved in the article history. This is true on the face of it, but false or
possibly meaningless in reality. A check as I write these words show that there
are 5,783,747 articles in the English language Wikipedia. There have been
873,775,490 edits to achieve this number of articles. This means an average of
151 archived states per article. In reality, some articles have many more
archived states, while other have far fewer.

The sheer mass of archival data makes the idea of reasonable transparency
impossible. The Library of Babel in Jorge Luis Borges's story contains a
universal library with every possible state of every possible book. However, it
also contains every possible state of many useless books, some containing only
one letter on a single page, others containing pages full of random characters
with two pages of important information buried in the middle, and so on. The
sheer mass of data hides the potentially useful information. While every article
contains better states and worse, it would take far too much work for any
ordinary user to locate the best article states and restore them.

Even were this to happen, however, nothing would ensure the stability of the
restored article. In fact, a process known as edit wars demonstrates the
frequent back-and-forth revision of articles between states where competing
editors have different views on what an article should be. Many of the processes
involving edit wars have now been automated, handed over to artificial
intelligence bots

Sample, Ian. 2017. 'Study reveals bot-on-bot editing wars raging on
Wikipedia's pages.' The Guardian. Thursday, February 23, 2017.

https://www.theguardian.com/technology/2017/feb/23/wikipedia-bot-editing-war-
study 

Tsvetkova, Milena, Ruth García-Gavilanes, Luciano Floridi, and Taha Yasseri.
2017. "Even good bots fight: The case of Wikipedia.' PLOS ONE. February 23
2017. DOI: https://doi.org/10.1371/journal.pone.0171774

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0171774

2) Transparency in the Editorial Process

While the talk pages supposedly provide transparency in the editorial process,
many editors do not engage in conversation on their edits. They simply edit, and
in some cases they refuse to respond to queries on the reasoning behind their
edits. Some experienced or senior editors argue from personal authority as
Wikipedians. Others rely on policy rulings, whether or not they have interpreted
policy correctly. In some cases that I have seen, the knowingly reject correct
or improved information on the basis that it does not conform to Wikipedia
standards. In essence, they are saying, 'Yes, this is true. But I'm
rejecting it until you go back and do it right.' In many cases, the kind of
conversation the appears on talk pages is curt or rude.

It is true that these kinds of conversations are often seen in academic circles
where reviewers can be harsh. There are nevertheless several major differences
between revising a journal article and revising or improving a Wikipedia
article. The first is the fact that a journal article counts as a publication,
while a Wikipedia article does not. That might not be a crucial difference to
most contributors -- after all, everyone who contributes to Wikipedia does so
knowing that the contribution that he or she makes will become part of a larger
collective work. The second difference is far more serious. Once one goes
through the pain of the review process, a journal article is published in stable
form. A Wikipedia article is never stable, and constant change and revision may
swiftly delete a contribution and all the work involved. The third difference is
also significant. This is the fact that one can usually discuss a review with
editors and -- through the journal -- with reviewers. In Wikipedia, this
process is often flawed, problematic, or absent.

-- Coverage, Notability, and Discouraged Contributors

While Wikipedia has extraordinarily wide coverage, there is reason to question
some of the gaps and exclusions. Wikipedia has a carefully constructed
notability policy -- but many editors don't follow the policy. Instead, they
frequently delete articles based on their own lack of knowledge.

A well-known case involves the decision to remove a biography of Nobel laureate
Donna Strickland on the basis that she was not notable enough. Even before
winning the prize, Strickland was a highly cited scientist. Her work fit within
Wikipedia Notability policies. The editor who removed her simply lacked the
knowledge -- and he apparently lacked the tools and skill to determine that
Strickland was, in fact, a well known scientist. My suspicion is that many
Wikipedia editors and administrators make seat-of-the-pants decisions without
knowing what they don't know -- and in many cases, they justify their quick
actions by asserting that they are too busy to take the time that a proper
investigation requires.

Every experienced journal editor and every skilled reviewer knows that
determining the validity or reliability of an article often takes several hours
of work. This work involves careful reading and checking on the subject field.

Wikipedia coverage on scholars, researchers, and scientists suffers from
significant gaps and omissions. A specific study describes this problem:

Samoilenko, Anna and Taha Yasseri. 2014. "The distorted mirror of Wikipedia: a
quantitative analysis of Wikipedia coverage of academics.' EPJ Data Science.
(2014) Vol. 3, No. 1. DOI: https://doi.org/10.1140/epjds20

https://link.springer.com/article/10.1140/epjds20

Samoilenko and Yasseri write, 'We found no statistically significant
correlation between Wikipedia articles metrics (length, number of edits, number
of incoming links from other articles, etc.) and academic notability of the
mentioned researchers. We also did not find any evidence that the scientists
with better WP representation are necessarily more prominent in their fields. In
addition we inspected the Wikipedia coverage of notable scientists sampled from
Thomson Reuters list of ‘highly cited researchers'. In each of the examined
fields Wikipedia failed in covering notable scholars properly. Both findings
imply that Wikipedia might be producing an inaccurate image of academics on the
front end of science.'

One of the odd issues with rigid and ignorant enforcement of Wikipedia's
notability policies is the fact that there is no shortage of space in Wikipedia.
Unlike a paper reference book, Wikipedia has room for every serious, well-
written, properly referenced biographical entry that any contributor might
write. Given that Wikipedia servers house 5,783,747 articles in the English
alone plus 873,775,490 archived edits, there should be no problem housing high
quality articles at any length on any topic interesting enough for a serious
contributor to write it.

Since 1901, 904 individuals and 24 organizations have won the Nobel Prize or the
Prize in Economic Sciences. Just over 70 mathematicians have won the Fields
Medal. One imagines that each of these deserves a serious article covering their
lives, achievements, and the influence of their work. Nevertheless, but many
have a Wikipedia entry with only a paragraph or two of basic facts.

Surely an encyclopedia that can archive nearly 900,000,000 edits in English
alone can create a notability policy that yields serious articles on any one
notable enough to warrant a specialist biography. With over 45,000,000 articles
total in 293 languages, the complete archives must be far more extensive.  If
English language articles get an average of 151 edited states each, it is
probably fair to allow for an average of 100 edited states even for smaller
language editions. A back-of-the-envelope estimate would suggest that all
Wikipedia editions together must archive at least 4,500,000,000 articles. This
being the case, one must wonder why any editor or administrator is stingy with
space on the basis of notability.

I know many people who are noted contributors to the fields in which I do
research. While they are not celebrities or public names, they are all well
within the notability criteria. Similar problems affect topical articles on
minor sports, serious but extremely small research fields, and so on.

Despite the need for in-depth articles on people and topics that lack coverage,
attempts to contribute to articles -- or to expand available information in
current articles -- seems to be a problem in Wikipedia. This is in part a
reflection of the tendency to favour earlier article states over serious, good-
faith improvements. But few of the people who can write high quality specialist
biographies want to argue with entrenched Wikipedia editors and administrators.
Instead, they give it a try, make a few efforts at explaining their work, become
discouraged, and leave.

I recognise that dedicated Wikipedians say this should not be the case.
Nevertheless, it is the case.

The situation is probably exacerbated by bot editors programmed to protect
articles by reverting changes. Bots do not explain their actions. They simply
act.

-- Expert Authors

In the past, it was my view that Wikipedia did not have many expert authors. I
was mistaken. It seems that Wikipedia has a great many expert authors. Even so,
many of these authors participate briefly and leave. Their expertise vanishes
with them as articles revert to a mediocre consensus.

There are some expert authors who are deeply committed to Wikipedia and willing
to go through the struggle of Wikipedia debate to preserve improved article
states. Nevertheless, a large review of articles in any field shows that expert
articles are in the minority.

While expertise is an issue, this is not a simple debate between credentialed
expertise and careful reference writing. Anyone who works with good journalists
or with copy editors who work across several fields understands that people who
know little about a field can nevertheless contribute significantly to high
quality articles. The problem with Wikipedia is that too many non-experts make
problematic judgements on articles where they hold ill-informed opinions.

Wikipedia contains a useful essay on the issue of expert editing and retaining
more excellent writers with subject field knowledge.

https://en.wikipedia.org/wiki/Wikipedia:Expert_retention


[continued in part 3/3


_______________________________________________
Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php

Editor: Willard McCarty (King's College London, U.K.; Western Sydney University, Australia)
Software designer: Malgosia Askanas (Mind-Crafts)

This site is maintained under a service level agreement by King's Digital Lab.