Open-Weight LLMs vs API Models: Which Should Your Business Use?

Related: WhatsApp Chatbots for Business: How They Work, What They Cost, and the Rules

Every business that builds with AI hits this question. Pay a provider per token, or run an open model on your own hardware?

The arguments for self-hosting sound strong: no per token bill, and your data never leaves. Both are sometimes true. Neither is automatically true. Here is how to think it through.

Get the terms right first

• API model. A model you can only use through the provider's service. You pay per token. You never have the model itself.
• Open-weight model. The trained model file is published and you can run it yourself. Families such as Llama, Mistral, Qwen, Gemma and DeepSeek are common examples.
• Open-source model. A stricter idea, where the training code and ideally the training data are open too. Many models called open-source are really open-weight.

The licence matters more than the label. Some open-weight licences are fully permissive. Others restrict commercial use above a certain size, ban specific uses, or require attribution. Read the licence before you build a product on a model.

What API models give you

• The strongest models available. On the hardest reasoning and multi step tasks, the leading API models are still ahead.
• No infrastructure. No GPUs to rent, patch, monitor or scale.
• Cost that follows usage. Zero traffic costs zero. Traffic spikes do not need spare hardware.
• Constant improvement. New models arrive without you doing anything.

The costs:

• Your data goes to a third party. Check the terms. Business tiers commonly exclude your data from training by default, and some providers offer stricter retention options, but you are still trusting their controls.
• Rate limits can cap you at peak.
• Models get retired, and behaviour can shift between versions.
• Per token cost at very high, steady volume.

What self-hosting gives you

• Data control. Prompts and documents stay on hardware you control. For some regulated work this is the deciding factor.
• Fixed, predictable cost once the hardware is running.
• Full control over the model. Fine tune it, pin a version forever, run it offline.
• No rate limits beyond what your hardware can serve.

The costs:

• GPUs are expensive, and idle ones cost the same as busy ones. A server you pay for 24 hours a day but use for 3 is mostly waste.
• Operations work. Serving software, scaling, monitoring, security updates, model upgrades. Someone has to own it.
• Lower quality on hard tasks. The gap has narrowed a lot for everyday work. It has not closed for the hardest problems.

A rough hardware guide

Very approximate, and it depends on quantisation and context length:

• Small models, around 7 to 9 billion parameters. Run on a single good consumer or workstation GPU when quantised. Fine for classification, extraction and simple chat.
• Mid-sized models, around 30 billion. Need a large-memory GPU or two.
• Large models, 70 billion and up. Need several data-centre GPUs.
• The largest open models. Need a full multi-GPU server, which is a serious monthly cost.

The cost maths

The question is utilisation. A self-hosted server has a fixed monthly cost. It only beats per token pricing if you keep it busy.

Self-hosting tends to win when:

• Volume is high and steady, around the clock
• The task is simple enough for a small or mid-sized model
• You already have people who run infrastructure

APIs tend to win when:

• Volume is low, uncertain, or spiky
• The task is hard enough to need a frontier model
• You are still working out what the product is

Most businesses starting out are in the second group. Price it properly before assuming otherwise, including engineering time.

The middle option

Several inference providers host open-weight models and charge per token. You get open models without running GPUs, often at a low price, and you can move to self-hosting later without changing the model.

It does not solve data residency, since a third party still sees the data. It does solve cost and flexibility.

How to decide

1. Is there a hard rule that data must stay in your infrastructure? Then self-host, and size the model to the task.
2. Does the task need the strongest reasoning available? Use an API model.
3. Is volume high and steady, and the task simple? Price self-hosting properly. It may win.
4. Otherwise, start with an API, and keep your code independent of any one provider so you can switch later.

Test on your own tasks

Public benchmarks tell you very little about your specific job. Build a set of 50 to 100 real examples with known good answers. Run each candidate model against it. Compare accuracy, cost per completed task, and speed. Decide on that, not on a leaderboard.

Many good systems end up hybrid: a small self-hosted or hosted open model for high volume simple work, and an API model for the hard cases. See estimating and cutting your LLM bill for the cost side.

If you want help choosing and testing models for a real use case, tell us what you are building.