Fine-tuning vs prompting
When to customize a model with extra training, and when better prompts are enough.
What is it?
Prompting changes what you tell the model each time. Fine-tuning changes the model itself: you train an existing model further on many examples of the inputs and outputs you want, so it picks up a style, format or specialized task.
For most needs, good prompts plus relevant documents (for example through RAG) are faster, cheaper and easier to update. Fine-tuning helps when you have many high-quality examples and need very consistent behaviour, a particular format or a smaller, cheaper model that performs a narrow task well. It is not a reliable way to teach a model new facts.
Which models use it?
- OpenAI offers fine-tuning for some of its models through its API.
- Google offers model tuning through Google Cloud.
- Fine-tuning is offered for some models on cloud platforms such as Amazon Bedrock.
- Open-weight models such as Llama and Mistral are often fine-tuned using open-source tools.
How to use it
- Start with clear prompts and examples in the prompt.
- Add relevant documents through retrieval if the model lacks knowledge.
- Measure results on a fixed set of test cases.
- Consider fine-tuning only if prompting still falls short and you have many good examples.
- Prepare clean, consistent training examples and keep some aside for testing.
- Compare the fine-tuned model with the original on the same tests before switching.
Benefits
- Prompting: quick to change and needs no training data.
- Fine-tuning: more consistent style and format on narrow tasks.
- Fine-tuning can let a smaller, cheaper model handle a specific job.
- Fine-tuning can shorten prompts, since behaviour is built in.
Limits and risks
- Fine-tuning needs good data, time and money, and must be redone when the base model changes.
- A fine-tuned model can lose some general abilities.
- Fine-tuning is a poor way to add facts that change often; retrieval is usually better.
- Training data may contain personal or confidential information that needs care.
Prompt template
Copy it, replace the parts in [brackets] and paste it into any AI chat.
I want an AI to [task] in a consistent [style/format]. Here are [number] examples of ideal input and output: [paste examples]. First, tell me whether a well-written prompt with these examples is likely enough or whether fine-tuning is worth considering, and why. Then write the best prompt you can using the examples, and list the test cases I should use to check it.
More in Building with AI
- Using AI through APIsHow apps and scripts send requests to AI models in code instead of a chat window.
- AI agents and tool useAI systems that can take actions, such as searching, running code or using apps, to finish a task.
- MCP (Model Context Protocol) and connectorsAn open standard for connecting AI assistants to tools and data sources in a consistent way.
- RAG (retrieval-augmented generation)Letting an AI answer from your own documents by finding relevant passages first.
- Embeddings and semantic searchTurning text into numbers that capture meaning, so you can search by idea rather than exact words.
- Evaluating AI outputPractical ways to check whether AI answers are accurate, useful and safe before you rely on them.