How to Install Ollama: Run AI Models on Your Own Server

Running your first model

To download a model and start a chat:

ollama run llama3.2

On the first run, the model is downloaded; after that you can talk to the model on the command line. Type /bye to exit. Commonly used commands:

You can see the available models and their sizes in the model library on Ollama's website.

Choosing a model and RAM

When running on the CPU, the model is loaded into the server's RAM. Roughly, you need as much RAM as the model's file size, plus a few GB of headroom:

• Quantization: Compressed versions of models, such as 4-bit ones, use much less RAM and run faster; the loss in quality is small for most uses.
• Speed: On the CPU, response speed depends on the core count, memory speed and model size. Start with a small model and scale up as you need.
• License: Each model has its own license; check it before commercial use.

Using the API

Ollama offers an HTTP API so your applications can connect to the model:

curl http://127.0.0.1:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why should I back up my server?",
  "stream": false
}'

For chat-style requests, the /api/chat endpoint is used. Ollama also offers an OpenAI-compatible API (/v1/chat/completions); you can connect your applications that use OpenAI client libraries to Ollama by changing the address.

A web interface with Open WebUI

To use models from the browser with a chat interface, you can install the open-source Open WebUI with Docker:

docker run -d -p 127.0.0.1:3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Because the port is bound only to 127.0.0.1, the interface isn't exposed to the internet. We cover installing Docker and port security in our Docker installation guide.

Secure access

Ollama's API has no authentication. If you open port 11434 to the internet, anyone who finds it can use your server's resources. Instead:

• SSH tunnel: Access the API or Open WebUI securely from your own computer:

  ssh -L 3000:127.0.0.1:3000 -L 11434:127.0.0.1:11434 user@203.0.113.10
  

  
  Then open http://127.0.0.1:3000 in your browser. The details are in our What is an SSH tunnel? article.
• Reverse proxy: If you're going to open Open WebUI to your team on a domain name, put Nginx and SSL in front of it and use Open WebUI's own user accounts. The details are in our Nginx installation guide.

If you need to change the address Ollama listens on, you can set the OLLAMA_HOST environment variable in the systemd service; but if you set the address to 0.0.0.0, open the port in the firewall only to addresses you trust.

CPU usage and performance

While a model generates a response, it uses all of the processor's cores heavily. For an assistant that a few people use now and then this isn't a problem, but constant, heavy use is a different matter:

• Limit concurrent requests. Many requests at the same time slow down every request.
• Don't keep models in memory longer than needed. Ollama unloads unused models from memory after a while; this time is configurable.
• Monitor resource usage. We cover tracking CPU and memory usage in our Server monitoring guide.

On virtual servers, CPU usage is subject to the provider's fair use rules. For AI workloads that need high CPU usage all day without a break, a dedicated server whose hardware is reserved entirely for you is a better fit.

Ollama at Nodesty

Nodesty AMD EPYC 7763 VPS plans, with their high-RAM options, let you comfortably keep 7–8 billion parameter models in memory. On virtual servers, average CPU usage has to be kept below 75% under our Fair Use Policy; if you're going to run the model heavily all day, choose our dedicated servers, where you can use the whole processor at 100% capacity 24/7.