Foundation Model
A foundation model is a large AI model trained on broad, general-purpose data that can be adapted, through fine-tuning or prompting, to many different downstream tasks rather than being built for just one.
Key takeaways
- A foundation model is trained on broad, general data and adapted to many tasks, rather than built for one narrow purpose.
- Foundation models let teams build new AI applications on top of an existing model instead of training from scratch.
- Large language models like GPT, Gemini, and Claude are examples of foundation models.
- A fine-tuned model is a foundation model further specialized for a specific task or domain.
- Most AI products people use are adapted versions of an underlying foundation model, not the raw model itself.
What is a foundation model?
A foundation model is a large-scale model, usually a transformer, trained on a broad and diverse dataset so it develops general capabilities that transfer across many different tasks. Rather than being built to do one narrow thing, like just translation or just spam detection, a foundation model can be adapted to a wide range of applications through prompting or lightweight fine-tuning.
Why foundation models matter
Before foundation models became common, building an AI system for a new task usually meant training a model from scratch for that specific purpose. Foundation models flip that: one large, expensive-to-train model becomes the shared starting point that many products and teams build on top of, dramatically lowering the cost and time needed to launch a new AI-powered application.
Examples of foundation models
Large language models like GPT, Gemini, and Claude are foundation models for text and increasingly other modalities. Similar general-purpose base models exist for images, audio, and even robotics, all following the same idea: train once at massive scale, then adapt for many specific uses.
Foundation model vs fine-tuned model
A foundation model is the general-purpose starting point; a fine-tuned model is a foundation model that has been further trained on task-specific data to specialize in a particular use case, such as a legal assistant or coding tool. Most AI products people interact with are fine-tuned or prompted versions of an underlying foundation model, not the raw foundation model itself.
Frequently asked
- What is a foundation model in AI?
- A foundation model is a large, general-purpose AI model trained on broad data that can be adapted to many different tasks through prompting or fine-tuning.
- Is GPT a foundation model?
- Yes. GPT, along with models like Gemini and Claude, is a foundation model for text and, in newer versions, other modalities as well.
- What is the difference between a foundation model and a fine-tuned model?
- A foundation model is the general-purpose base; a fine-tuned model is that same base further trained on specific data to specialize in a particular task or domain.