Choosing AI Models: Buy, Build, Fine-Tune, or Go Small

Choosing AI models has become a board-level conversation that most boards are poorly equipped to have. The options multiply, suppliers make confident claims, and engineering teams often favour whatever they used last. Yet the decision carries real consequences for cost, data protection, supplier dependence and how quickly the organisation can change course.

Our view is that leaders should stop looking for the best model. There is no such thing. There is only the model that is good enough for a specific task, at an acceptable cost and speed, within your data constraints, and replaceable when something better arrives. This guide sets out the options, the trade-offs, and the discipline that makes the choice manageable.

The options when choosing AI models

Most organisations will end up using more than one of these routes.

Hosted frontier models. The largest and most capable general models, accessed as a service. They handle the widest range of tasks with the least effort, and they show quickly whether a task is feasible at all. The trade-offs are usage-based cost that grows with volume, dependence on a supplier's roadmap and terms, and data leaving your environment.

Open-weight models. Models whose weights are published, so you can run them on infrastructure you control. You gain control over data location, versioning and change. You take on hosting, scaling, security patching and the skills to run them. Licence terms vary and some restrict commercial use, so have them reviewed.

Small language models. Compact models, open or commercial, designed for narrower work. For classification, extraction, routing, summarising a known document type and similar bounded tasks, a small model can often match a large one with much lower running cost and latency. They are weaker at open-ended reasoning and broad knowledge.

Building your own. Training a foundation model from scratch is out of reach and out of scope for nearly every enterprise. When leaders say "build", they almost always mean building an application around someone else's model, which is sensible, or adapting a model through fine-tuning, which needs a closer look.

Fine-tuning versus retrieval and good prompts

Fine-tuning means further training a model on your own examples so that it behaves differently. It is often proposed first and should usually come last.

Start with the prompt. Clear instructions, good examples and a well-structured task resolve many quality problems at almost no cost. Next, if the model lacks your knowledge, give it that knowledge at the moment of the question through retrieval. This is the approach described in our article on grounded enterprise AI search. Retrieval keeps knowledge current, respects permissions, and lets you cite sources. Facts trained into a model are hard to update, hard to remove and hard to attribute.

Fine-tuning earns its place in narrower circumstances:

• Consistent format or style. The output must follow a strict structure or house tone that prompting cannot hold reliably.
• Specialised language. The domain has vocabulary or conventions that general models handle poorly.
• Making a small model good enough. A small model, tuned on good examples of one task, can replace a large one at high volume. This is the most common economic case.
• Latency or cost pressure. Long prompts full of instructions and examples can be shortened once the behaviour is trained in.

You need a well-labelled dataset, an evaluation set, and the capacity to repeat the work whenever the base model is updated or retired. A tuned model is an asset you now maintain.

There is no best model. There is the model that is good enough for this task, at this cost, within these constraints, and replaceable next year.

Data residency, cost and latency

Residency and sovereignty

Know where your data is processed, where it is stored, who can access it, and under which jurisdiction the supplier operates. For personal data, GDPR obligations on transfers and processors apply. Regulated sectors face additional requirements, and EU financial entities have had DORA's rules on third-party technology risk to consider since January 2025. Read supplier commitments on regional processing and training in the contract, not in the marketing, and confirm your position with counsel. Where the constraints are strict, open-weight models in your own environment may be the only acceptable route for some workloads, and your AI governance and compliance process should record which data classes may go where.

Cost and latency

Usage-based pricing makes a pilot look cheap and a scaled service look alarming. Estimate cost per unit of business work, such as per case or per document, at production volume, including retries, long contexts and the multiple model calls that agents make. Compare that with the full cost of self-hosting, including the people to run it. Apply the discipline outlined in our guide to cloud cost strategy and FinOps.

Latency decides whether a product is usable. A batch job can wait. A customer conversation cannot. Routing helps: send simple requests to a small, fast model and reserve the large one for the hard cases.

Lock-in and portability

Models will be superseded, repriced and withdrawn on the supplier's timetable, not yours. Assume you will change models more than once, and design for it.

• Put an abstraction layer between applications and models. Applications call your interface, not a supplier's directly.
• Keep your assets outside the model. Prompts, evaluation sets, retrieval indexes, fine-tuning data and logs are yours. Store them in forms you can move.
• Negotiate notice. Seek contractual notice of model changes and retirements, and pin versions where you can.
• Rehearse the switch. Run your evaluation set against an alternative model periodically, so that moving is a known quantity.

The goal is that leaving is a project, not a crisis.

Evaluation as the deciding discipline

Every question above is settled by the same thing: an evaluation set of your own. Public benchmarks and supplier demonstrations measure general ability on someone else's tasks. They say little about your contracts or your customers' phrasing.

An evaluation harness is a collection of real examples of the task, with expected results or scoring criteria agreed by the people who own the process, run automatically against any candidate model. It turns an argument about preference into a comparison of quality, cost and speed. You rerun it whenever a model, prompt or data source changes.

Questions to ask your team before approving a model decision:

• What is the task, precisely, and what does a good enough answer look like to the business owner?
• Do we have an evaluation set built from our own examples, including the difficult ones?
• Have we tried a smaller or cheaper model against that set before settling on the largest?
• Have we exhausted prompting and retrieval before proposing fine-tuning?
• What is the cost per unit of work at production volume, and who owns that number?
• How would we switch, and how long would it take?

If the team can answer these, the choice of model is usually obvious. If it cannot, no model will rescue the project.

Common questions

Should we use an open-weight model or a hosted AI model?

Start with a hosted model to prove the task is feasible and valuable, unless data constraints rule it out. Move to an open-weight model when you need control over data location and versions, or when volume makes self-hosting cheaper and you have the skills to run it.

When is fine-tuning an AI model worth it?

When prompting and retrieval have been tried and quality is still short, and the gap concerns behaviour, format or specialised language, not missing knowledge. It is also worth it when tuning a small model lets you replace a large one at high volume.

Are small language models good enough for business use?

For bounded tasks such as classification, extraction, routing and summarising familiar document types, often yes, and with lower cost and latency. They are less suited to open-ended reasoning or questions needing broad knowledge. Test one against your own evaluation set.

Where Ganexa can help

Ganexa helps leadership teams make model decisions on evidence. Through our AI strategy and implementation service we define the use cases, build the evaluation sets, compare options on quality, cost and risk, and design an architecture that keeps you portable. Where senior technical judgement is missing, our fractional CTO and CIO support can own the decision with you. To discuss your options, book a free consultation.