AI3 min read
RAG, fine-tuning or a better prompt? Choosing how to add AI to your product
Three ways to make a language model useful with your data, and a simple decision guide for picking the right one. Spoiler: start with the cheapest.
ALBy the Adviqlabs team
When teams decide to "add AI", the conversation often jumps straight to fine-tuning a model. In practice, most useful LLM features in products today are built with good prompting and retrieval, and fine-tuning is reserved for a narrow set of problems.
Here's how the three approaches differ, and how we choose between them.
Option 1: Prompting
You give a general-purpose model clear instructions, examples and whatever context fits in the request.
Good for: summarising, rewriting, classifying, extracting fields from text, drafting replies, and anything where the model already knows enough about the world.
Why start here: it's fast to build, cheap to change and easy to evaluate. You can often ship a working feature in days. With structured output (JSON schemas), prompting also handles a lot of "turn this messy text into data" work reliably.
Where it breaks: when the answer depends on information the model has never seen, like your product catalogue, internal policies or a customer's account history.
Option 2: Retrieval-augmented generation (RAG)
Before calling the model, you search your own data for the most relevant pieces and include them in the prompt. The model answers using that material, ideally with citations.
A typical RAG pipeline looks like this:
- Ingest: pull documents from their sources (help centre, PDFs, database rows)
- Chunk: split them into passages of a few hundred words
- Embed and index: store vector embeddings, often alongside keyword search
- Retrieve: for each question, find the best matching passages
- Generate: ask the model to answer only from those passages, and cite them
Good for: support assistants, internal knowledge search, "chat with your docs", product Q&A, and anything where facts change often.
Why it's usually the right call: your data stays in your control, updates are instant (re-index a document and the next answer reflects it), and citations let users verify answers.
Where teams go wrong: skipping retrieval quality. Most bad RAG answers are retrieval problems, not model problems. Hybrid search (vector plus keyword), sensible chunking and re-ranking matter more than which model you pick.
Option 3: Fine-tuning
You further train a model on examples of the inputs and outputs you want.
Good for: a consistent style or format that's hard to describe in a prompt, narrow classification tasks at high volume, or getting a smaller, cheaper model to match a larger one on one specific task.
Not good for: teaching the model facts. Fine-tuned knowledge goes stale, is hard to update and can't be cited. If you want the model to know your data, that's a retrieval problem.
Costs to consider: you need a few hundred to several thousand high-quality examples, an evaluation set, and a process to retrain as things change.
A quick decision guide
| Your situation | Start with |
|---|---|
| The model already knows enough; you need a transformation of text | Prompting |
| Answers depend on your documents or data | RAG |
| Your data changes weekly or daily | RAG |
| Users need to trust and verify answers | RAG with citations |
| You need a very specific output style at high volume | Prompting first, fine-tune later |
| You want to cut cost on one narrow, high-volume task | Fine-tuning a smaller model |
Whatever you choose, build the evaluation first
The most important piece of any AI feature isn't the model. It's the set of test questions with known good answers that you run every time you change a prompt, a retrieval setting or a model version. Start with 30 to 50 real examples from users or support tickets. Without this, every change is a guess.
Our default path
- Ship with prompting where it's enough
- Add retrieval as soon as answers depend on your data
- Measure quality and cost with an evaluation set
- Consider fine-tuning only when you have a narrow, stable, high-volume task and the numbers justify it
Thinking about AI in your product? We help teams pick the simplest approach that works, then build it properly. Talk to us.