AI & Enterprise Intelligence
Retrieval or fine-tuning? Start with the failure
Changing what a model can look up and changing how it behaves solve different problems. The choice should follow evidence from the task.
An enterprise assistant can fail because it never received the right information. It can also receive the right information and handle it badly. Those failures may look similar in the interface, but they suggest different engineering decisions.
Before choosing retrieval-augmented generation, usually called RAG, or fine-tuning, inspect a small set of failed tasks. Identify the information the system needed, what it actually received and the behaviour expected from the model. That record is more useful than selecting an architecture because it appears on a technology shortlist.
Two different ways to change the result
RAG retrieves relevant material and supplies it as context when the model answers. It can connect a response to private or changing information and provide citations. The retrieval step might use keyword, vector or hybrid search; RAG does not prescribe one universal search technology. [1]
Supervised fine-tuning adapts a model through training examples of the desired task or behaviour. Google describes using labelled examples to adjust learned parameters and recommends starting with prompting before tuning is necessary. A recurring problem with classification, output structure or a domain-specific task can justify investigating tuning. It requires suitable training data and evaluation, rather than simply uploading a document library. [2]
Choose retrieval when the answer depends on the source
Imagine a hypothetical service assistant answering questions about product warranties. The terms vary by product and purchase date, and the approved documents change. The useful first question is whether the application can find the applicable terms and show where they came from. Retrieval is a reasonable starting point for that requirement.
The supporting pipeline still matters. Google's RAG overview describes ingestion, transformation, indexing, retrieval and generation as distinct stages. A document update will help only after the relevant pipeline has processed it. Keep ownership, effective dates and document identity with the content so the system can distinguish an applicable policy from an obsolete one. [3]
Permissions need equal attention. Microsoft's guidance recommends enforcing access when retrieving information and warns that a grounded answer can still expose sensitive material if source access is uncontrolled. Being able to cite a document is not evidence that the person asking was entitled to read it. [1]
Choose tuning when the behaviour is the bottleneck
In another hypothetical workflow, a model receives complete support messages but repeatedly assigns the wrong categories from an established taxonomy. Better document retrieval may not address that failure. First make the categories and instructions unambiguous. Then compare a prompt-based baseline with a tuned candidate if enough representative, well-labelled examples are available.
Training examples should resemble the work expected in use. Google explicitly recommends examining mistakes before adding data and prioritising label quality. Keep a separate evaluation set so improvements are judged on examples outside the training material. Check difficult boundaries between categories as well as easy cases; a familiar-looking answer is not sufficient evidence of a useful improvement. [2]
Tuning also introduces work around dataset review, model versions and regression testing. Include that maintenance in the decision. A modest improvement that depends on an expensive, fragile labelling process may be less attractive than a simpler prompt or workflow change.
Fix the part that is actually failing
Retrieval creates several places to investigate: document preparation, chunk boundaries, search configuration and the final response. Microsoft's architecture guidance recommends evaluating these stages separately while retaining an end-to-end view of the user's result. If the needed passage never reaches the model, changing its response style leaves the underlying problem in place. [4]
There are operational tradeoffs too. Retrieval adds search work, network round trips and context tokens, with consequences for latency and cost. It can still produce incomplete or inaccurate answers when retrieved passages are poor. Source links help inspection; they do not remove the need to check whether the cited material supports the claim. [1]
Combine methods only for a clear reason
The two methods can serve different responsibilities in one application: retrieval supplies the applicable material, while a tuned model follows a specialised response or extraction task. Treat that as an architecture hypothesis to test, not an automatic upgrade. Each added component creates another configuration to understand.
Compare candidates using the same representative questions, source material and acceptance criteria. Record answer quality, handling of missing information, response time and the effort required to maintain the system. Include questions for which no answer exists. The best fit is the simplest arrangement that meets the actual requirement with enough evidence to support the choice. [4]
Sources & further reading
The references behind this perspective. Reviewed . Product documentation can change; check the source before making an implementation decision.
- Microsoft — Retrieval augmented generation and indexes
Primary explanation of RAG, search choices, citations, retrieval-time access control, latency and cost, and the limits of grounded generation.
- Google Cloud — Introduction to tuning
Primary explanation of supervised tuning, task examples, prompting as a baseline and the importance of representative, well-labelled training data.
- Google Cloud — RAG Engine overview
Primary description of the ingestion, transformation, indexing, retrieval and generation pipeline. Document freshness recommendations in the article are engineering implications of that pipeline.
- Microsoft — Design and develop a RAG solution
Primary architecture guidance for representative queries, chunking, retrieval choices and independent component evaluation alongside end-to-end acceptance criteria.
