Online or offline language models: which fits your organization?
Cloud-hosted language models compared with models that run on your own servers: data security, cost, hardware, and where each one fits.
Large language models, the models that read and write text, are now used in many organizations for search, summaries, question answering and drafting. One of the first decisions is where the model runs: on a provider's servers, reached over the internet (online), or on the organization's own servers (offline).
Online models
Online models run on the provider's servers, and the organization sends them requests over the internet; models from OpenAI, Anthropic and Google Gemini are examples. There is no hardware to buy, the latest version is always available, and the provider handles maintenance. In return, every text sent to the model leaves the organization, costs grow with use, and nothing works without a stable internet connection.
Offline models
Offline models are models whose makers allow them to be downloaded and run, and the organization runs them on its own servers, usually with tools such as Ollama. Data never leaves the organization, work continues without the internet, and the model can be tuned to the organization's needs. In return, the organization has to provide suitable hardware, above all graphics processors and enough memory, and handle updates and maintenance itself.
At a glance
| Online | Offline | |
|---|---|---|
| Where the data goes | Sent to the provider's servers | Stays in the organization |
| Internet | Required | Not required |
| Cost | Subscription or pay per use | Hardware, power and maintenance |
| Updates | Automatic, by the provider | By your own technical team |
| Customization | Limited to what the provider offers | Full |
| Scaling up | Fast | Needs more hardware |
Which one fits you
- An online model suits work without sensitive data, workloads that rise and fall, or tasks that need the very latest model capabilities.
- An offline model suits organizations with confidential data, strict regulation or limited internet, such as industry, government bodies and research centres.
Many organizations use both: an offline model for confidential documents and an online model for general work. The condition is that the boundary between the two is clear and under control.
In Vakav Autonomous you can choose a model for each workspace separately: an in-house model for technical and confidential documents, for example, and an online model for general work where that is acceptable. The guide to connecting language models shows how.