A large language model (LLM) is an AI system trained on massive amounts of text that generates and understands natural language by predicting the most likely next word in a sequence.
What is a large language model?
A large language model, or LLM, is a type of neural network trained on huge collections of text — books, articles, code, and web pages — to predict what word (or part of a word) comes next in a sequence. That simple prediction task, repeated across billions of examples, is enough to teach the model grammar, facts, reasoning patterns, and writing style.
How LLMs work
Most LLMs are built on the transformer architecture, which lets the model weigh how relevant every other word in a passage is to the word it's currently generating. Training happens in two stages: pretraining, where the model learns general language patterns from a broad corpus, and fine-tuning, where it's adjusted for a specific behavior like following instructions or refusing harmful requests.
What LLMs are used for
LLMs power chatbots, coding assistants, search summarization, translation, and content generation. Because they generate text one token at a time based on probability, they can also produce a hallucination — stating something confidently that isn't true — which is why systems built on them are often paired with retrieval-augmented generation or other grounding techniques.
Limitations of LLMs
LLMs don't "know" facts the way a database does — they reproduce patterns learned during training, which is why they can be outdated, biased toward their training data, or wrong about niche topics. Their knowledge is also bounded by a context window: the amount of text they can consider at once.
Frequently asked
- What does LLM stand for?
- LLM stands for large language model — an AI system trained on huge volumes of text to understand and generate natural language.
- What is the difference between an LLM and a chatbot?
- An LLM is the underlying model that generates text; a chatbot like ChatGPT is a product built around an LLM, adding a conversational interface, memory, and safety rules on top.
- How are LLMs trained?
- LLMs are pretrained on massive text datasets to predict the next word, then fine-tuned, often with reinforcement learning from human feedback, to follow instructions and avoid harmful outputs.
- Why do LLMs hallucinate?
- LLMs generate the statistically most likely next words rather than retrieving verified facts, so they can produce fluent, confident-sounding text that isn't actually true.