Steven Chalem's Blog

  • GPT-3 Confabulates about Kittens

    (Originally posted May 27, 2022)

    I decided to revisit my discussion of kitten scientists with AI language generation model GPT-3 from OpenAI.

    GPT-3 sometimes seems to be just making things up. This is called “confabulation” and it can be a troublesome bug or a valuable feature depending on one’s intent.

    Since GPT-3 is driven by probability and not primarily by logic, it often returns results that are incorrect or don’t make sense but that are somehow in line with the prompt or question. For example:

    (My prompts are in bold. GPT-3’s responses are in plain type. I take sole responsibility for this post.)

    What famous scientists were kittens?
    Some famous scientists who were kittens include:
    – Isaac Newton
    – Marie Curie
    – Albert Einstein
    – Benjamin Franklin
    – George Washington
    – Thomas Jefferson
    – Abraham Lincoln
    – John F. Kennedy
    Are all scientists kittens?
    Certainly not! But a number of well-known and respected scientists started out as kittens.
    When was Isaac Newton a kitten?
    Isaac Newton was a kitten in the 17th century.
    Did he know he was a kitten?
    It’s unclear if Isaac Newton knew he was a kitten, but it is well-documented that he was, in fact, a kitten.