Home About Subscribe Search Member Area

Humanist Discussion Group

< Back to Volume 34

Humanist Archives: July 28, 2020, 7:25 a.m. Humanist 34.193 - on GPT-3 & beyond

                  Humanist Discussion Group, Vol. 34, No. 193.
            Department of Digital Humanities, King's College London
                   Hosted by King's Digital Lab
                Submit to: humanist@dhhumanist.org

        Date: 2020-07-28 04:40:25+00:00
        From: Bill Benzon 
        Subject: NEW SAVANNA: 1. No meaning, no how: GPT-3 as Rubicon and Waterloo, a personal view

[The following concerns an AI text-generator called GPT-3, for which e.g. see https://www.wired.co.uk/article/gpt-3-openai-examples --WM]

Willard and fellow humanists -- The second installment (of 5 planned) in my
series on GPT-3 and beyond.

1. No meaning, no how: GPT-3 as Rubicon and Waterloo, a personal view

I say that not merely because I am a person and, as such, I have a point of view
on GPT-3, and related matters. I say because the discussion is informal, without
journal-class discussion of this, that, and the others, along with the attendant
burden of citation, though I will offer a few citations. More over, I'm pretty
much making this up as I go along. That is to say, I am trying to figure out
just what it is that I think, and see value in doing so in public.

What value, you ask? It commits me to certain ideas, if only at a certain time.
It lays out a set of priors and thus serves to sharpen my ideas developments
unfold and I, inevitably, reconsider.

GPT-3 represents an achievement of a high order; it deserves the attention it
has received, if not the hype. We are now deep in "here be dragons"
territory and we cannot go back. And yet, if we are not careful, we'll never
leave the dragons, we'll always be wild and undisciplined. We will never
actually advance; we'll just spin faster and faster. Hence GPT-3 is both a
Rubicon, the crossing of a threshold, and a potential Waterloo, a battle we
cannot win.

Here's my plan: First we take a look at history, at the origins of machine
translation and symbolic AI. Then I develop a fairly standard critic of semantic
models such as those used in GPT-3 which I follow with some remarks by Martin
Kay, one of the Grand Old Men of computational linguistics. Then I look at the
problem of common sense reasoning and conclude be looking ahead to the next post
in this series in which I offer some speculations on why (and perhaps even how)
these models can succeed despite their sever and fundamental short-comings.


Bill Benzon



Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php

Editor: Willard McCarty (King's College London, U.K.; Western Sydney University, Australia)
Software designer: Malgosia Askanas (Mind-Crafts)

This site is maintained under a service level agreement by King's Digital Lab.