OpenAI releases ChatGPT, new AI chatbot based on GPT-3 that can converse, write code and music

OpenAI has released ChatGPT, a conversational AI system built on GPT-3 that can engage in back-and-forth dialogue, answer follow-up questions, and reject inappropriate requests. The company trained ChatGPT using human feedback and reinforcement learning, building on techniques from InstructGPT to reduce toxic outputs. Despite these improvements, ChatGPT still generates inaccurate information, including errors demonstrated in live demos where it confused diffusion models and misidentified DALL-E as a GAN. OpenAI researcher John Schulman acknowledges the fabrication problem remains far from solved, while external researchers like Teven Le Scao argue that fine-tuning alone cannot achieve factuality. OpenAI is developing WebGPT with web search capabilities and may integrate this feature into ChatGPT within months to improve trustworthiness.

ChatGPT's conversational capabilities and training methodology

• ▪To build ChatGPT, OpenAI first asked people to give examples of what they considered good responses to various dialogue prompts
• ▪OpenAI released a demo of ChatGPT, a spin-off of GPT-3 geared toward answering questions via back-and-forth dialogue
• ▪ChatGPT's conversational format allows ChatGPT to answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests
• ▪ChatGPT will not answer questions about events that took place after 2021
• ▪ChatGPT is a sister model to InstructGPT, a version of GPT-3 that OpenAI trained to produce text that was less toxic
• ▪Human judges gave scores to ChatGPT's responses that OpenAI fed into a reinforcement learning algorithm to train the final version of ChatGPT
• ▪ChatGPT refuses to answer questions about topics ChatGPT has not been trained on

Persistent factual accuracy problems and hallucination issues

• ▪Making a large language model more cautious in its answers often stops the model from answering questions that the model would otherwise have gotten correct
• ▪John Schulman stated that OpenAI has made some progress on the problem of ChatGPT making stuff up, but the problem is far from solved
• ▪ChatGPT can admit when ChatGPT doesn't know what ChatGPT is talking about
• ▪Teven Le Scao stated that fine-tuning on human feedback won't solve the problem of factuality in large language models
• ▪In a live demo, ChatGPT initially responded with information about the diffusion process in chemistry when asked about diffusion models in machine learning
• ▪ChatGPT incorrectly stated that DALL-E is a GAN during a live demo
• ▪ChatGPT still makes stuff up despite addressing some problems with large language models
• ▪There is no way to train a large language model so that the model tells fact from fiction

Potential solutions through web search integration

• ▪John Schulman stated that OpenAI might upgrade ChatGPT with the ability to look up information on the web in the next few months
• ▪Teven Le Scao stated that the ability to look up information will be key if large language models are to become trustworthy
• ▪OpenAI is working on another language model called WebGPT that can look up information on the web and give sources for its answers

Perspective of AI safety researchers and critics

• ▪There is no way to train a large language model so that the model tells fact from fiction