Skip to content
ADVIQ LABS
All insights

AI3 min read

RAG, fine-tuning or a better prompt? Choosing how to add AI to your product

Three ways to make a language model useful with your data, and a simple decision guide for picking the right one. Spoiler: start with the cheapest.

ALBy the Adviqlabs team

When teams decide to "add AI", the conversation often jumps straight to fine-tuning a model. In practice, most useful LLM features in products today are built with good prompting and retrieval, and fine-tuning is reserved for a narrow set of problems.

Here's how the three approaches differ, and how we choose between them.

Option 1: Prompting

You give a general-purpose model clear instructions, examples and whatever context fits in the request.

Good for: summarising, rewriting, classifying, extracting fields from text, drafting replies, and anything where the model already knows enough about the world.

Why start here: it's fast to build, cheap to change and easy to evaluate. You can often ship a working feature in days. With structured output (JSON schemas), prompting also handles a lot of "turn this messy text into data" work reliably.

Where it breaks: when the answer depends on information the model has never seen, like your product catalogue, internal policies or a customer's account history.

Option 2: Retrieval-augmented generation (RAG)

Before calling the model, you search your own data for the most relevant pieces and include them in the prompt. The model answers using that material, ideally with citations.

A typical RAG pipeline looks like this:

  1. Ingest: pull documents from their sources (help centre, PDFs, database rows)
  2. Chunk: split them into passages of a few hundred words
  3. Embed and index: store vector embeddings, often alongside keyword search
  4. Retrieve: for each question, find the best matching passages
  5. Generate: ask the model to answer only from those passages, and cite them

Good for: support assistants, internal knowledge search, "chat with your docs", product Q&A, and anything where facts change often.

Why it's usually the right call: your data stays in your control, updates are instant (re-index a document and the next answer reflects it), and citations let users verify answers.

Where teams go wrong: skipping retrieval quality. Most bad RAG answers are retrieval problems, not model problems. Hybrid search (vector plus keyword), sensible chunking and re-ranking matter more than which model you pick.

Option 3: Fine-tuning

You further train a model on examples of the inputs and outputs you want.

Good for: a consistent style or format that's hard to describe in a prompt, narrow classification tasks at high volume, or getting a smaller, cheaper model to match a larger one on one specific task.

Not good for: teaching the model facts. Fine-tuned knowledge goes stale, is hard to update and can't be cited. If you want the model to know your data, that's a retrieval problem.

Costs to consider: you need a few hundred to several thousand high-quality examples, an evaluation set, and a process to retrain as things change.

A quick decision guide

Your situationStart with
The model already knows enough; you need a transformation of textPrompting
Answers depend on your documents or dataRAG
Your data changes weekly or dailyRAG
Users need to trust and verify answersRAG with citations
You need a very specific output style at high volumePrompting first, fine-tune later
You want to cut cost on one narrow, high-volume taskFine-tuning a smaller model

Whatever you choose, build the evaluation first

The most important piece of any AI feature isn't the model. It's the set of test questions with known good answers that you run every time you change a prompt, a retrieval setting or a model version. Start with 30 to 50 real examples from users or support tickets. Without this, every change is a guess.

Our default path

  1. Ship with prompting where it's enough
  2. Add retrieval as soon as answers depend on your data
  3. Measure quality and cost with an evaluation set
  4. Consider fine-tuning only when you have a narrow, stable, high-volume task and the numbers justify it

Thinking about AI in your product? We help teams pick the simplest approach that works, then build it properly. Talk to us.

Start a project

Have a product in mind? Let's scope it together.

Book a free call. We'll talk through your idea, suggest the leanest path to launch and follow up with a written proposal.