What is ChatGPT-4? I’ll break it down.

“Chat” is for Conversation

The “Chat” part indicates that ChatGPT-4 is a conversational interface to an AI model (namely, the model GPT-4). An AI model is a type of software that processes examples in some area (in this case, words on the internet), finds patterns, and uses those patterns to make predictions or decisions. In addition to the huge corpuses of text that ChatGPT uses as examples, it was also trained on how conversations work – through “Reinforcement Learning with Human Feedback”.

	Please give very brief responses. This is for a blog post.
	Sure, I’d be happy to provide brief responses. Please proceed with your questions.
	Upper Volta!
	Upper Volta was the former name of Burkina Faso, a landlocked country in West Africa. The name changed in 1984.
	Capital?
	The capital of Burkina Faso is Ouagadougou.

G is for Generative

GPT-4 is a “generative” model; it generates text. Some AI models recognize faces, some drive cars, some generate images or music. GPT-4 generates text.

	Please write one short marketing slogan for turnips.
	Turn Up with Turnips: Unbeatable Flavor, Unbelievable Health!

P is for Pre-Trained

Generating text in response to a prompt is the easy part. What makes it (relatively) easy is having on-hand a store of the patterns that were found in the training data.

That’s the pre-training. It takes a long time and massive computing resources to distill everything on the internet (and more) into a form that can be referenced for quick responses to prompts. For each version of ChatGPT, it’s only done once.

That’s why some of ChatGPT’s knowledge is stuck in September 2021.

	Who is King Charles III of the United Kingdom?
	As of my knowledge cutoff in September 2021, Charles III has not ascended to the throne of the United Kingdom. The current monarch is Queen Elizabeth II and her heir apparent is Charles, Prince of Wales. If he ascends to the throne, he could choose the name Charles III or another regnal name.

T is for Transformer

I’ll let ChatGPT handle this one.

	In AI, what is a Transformer? Please provide a very brief explanation at a college freshman reading level. Focus on parallel processing.
	[Long explanation]
	that was great, but not brief enough
	Sure, let me make it simpler: A Transformer is a type of AI model used to understand language. It’s special because it can process all words in a sentence simultaneously (parallel processing) and figure out which words are most important (using “attention”). This makes it faster and smarter at understanding context.

4 is for the moment

ChatGPT-4 is the fifth publicly-released version of ChatGPT-4 (there was a 3.5). The biggest surprises came with ChatGPT-3, when natural-sounding conversations began to emerge. Getting there from ChatGPT-2 was the result of many incremental improvements plus a much larger model. (The size of the model determines how detailed the patterns found will be.)

ChatGPT-3.5 brought more natural conversation and many other improvements. It is still more widely used than ChatGPT-4.

There’s a lot of talk already about ChatGPT-5, where OpenAI is currently working. Researchers may be reaching the limits of how much electrical power and cooling it’s feasible to consume for this purpose. I’m guessing it will take another conceptual innovation on the scale of Transformers to build something as surprising as ChatGPT-4.

What will be the next big innovation in large language models?

Predicting future innovations is always speculative, but a few potential directions include: More efficient training and inference: Reducing the computational resources required to train and use these models. Better understanding and reasoning: Improvements in a model’s ability to comprehend complex contexts, generate more accurate responses, and handle logical reasoning tasks. Multimodal capabilities: Integrating text with other types of data like images or audio, enabling the model to understand and generate content across different formats. Ethics and fairness: Advances to address bias in AI, make its decisions more transparent, and align its behavior with human values. Remember, these are just possibilities based on the state of AI as of 2021, and the actual future could look quite different!

The text inside the dialog boxes is copied verbatim from chat sessions. Some dialog boxes show the beginnings of sessions and some show continuations of sessions. None of the results are “cherry-picked.” That is, the responses shown are the first responses ChatGPT made to the given prompts.

I wrote the parts outside the boxes, with research assistance and ideas from ChatGPT-4.