How to Host Your Own AI API on a VPS

There’s a specific dread that hits when your AI app starts getting traction: the API bill. What started as a few cents a day in testing quietly becomes a line item you flinch at every month. Then come the rate limits, the surprise model deprecations that break your app overnight, and the nagging feeling that your users’ prompts are training someone else’s model.

When you host your own AI API on a VPS, all of that changes. Your endpoints, your keys, your rate limits, your data never leaving your server. An OpenAI-compatible API running on a $20–40/month VPS can serve real traffic for a fraction of what per-token pricing costs at scale — and nothing goes down because a provider changed their pricing page. This guide walks through the two practical ways to do it, the VPS specs that actually matter, a full step-by-step setup with auth and HTTPS, and the honest cost math.

Table of Contents

Why Host Your Own AI API? The Real Reasons

Most tutorials frame self-hosting as a cost hack. It can be, but the deeper reasons are control and predictability:

• Fixed costs. Per-token pricing is wonderful at zero traffic and brutal at scale. A VPS costs the same whether you serve 10 requests or 10,000. If you can predict your traffic, you can beat the API bill.
• Your data stays yours. User prompts, internal documents, proprietary workflows — none of it transits a third-party API. For internal tools, client work, or anything regulated, this alone justifies the setup.
• No rate-limit roulette. Your concurrency limit is your server’s capacity, not a tier a provider assigned you. No 429s at the worst moment.
• No surprise deprecations. When a hosted provider sunsets the model your app depends on, you rewrite or scramble. Your own API runs the same weights until you decide to change them.
• Drop-in compatibility. Modern open models serve an OpenAI-compatible endpoint, so your existing code usually changes one line: the base URL.

The honest caveat: you become the ops team. Uptime, updates, and scaling are your problem. If your app makes three API calls a week, just pay per token — self-hosting is for steady, real traffic. New to servers entirely? Read our VPS hosting primer first.

Pick Your Approach: Ollama vs LiteLLM vs Custom FastAPI

There are three serious ways to serve an AI API from a VPS, and the right one depends on what you’re actually building:

Approach	What it is	Best for	Biggest weakness
Ollama’s built-in server	Ollama exposes an OpenAI-compatible API out of the box on port 11434	Single-model chat/completion APIs, personal projects, internal tools	One model family per setup; minimal auth and rate-limiting built in
LiteLLM proxy	Open-source gateway that unifies 100+ providers and local models behind one OpenAI-compatible endpoint	Mixing self-hosted + paid APIs, multiple keys, usage tracking, fallbacks	Another service to run and secure
Custom FastAPI wrapper	Your own Python service in front of a local model (llama.cpp, vLLM, transformers)	Custom logic: RAG, tool use, validation, bespoke billing	You’re building and maintaining real software

My recommendation: start with Ollama’s server, graduate to LiteLLM when you need keys and routing. Ollama gets you a working, OpenAI-compatible API in minutes. LiteLLM becomes worth it the day you want per-user API keys, spend tracking, or to route cheap requests to your VPS and overflow to a paid provider. A custom FastAPI wrapper only earns its keep when your API needs logic that isn’t just “talk to a model.”

If serving models is your main goal, pair this guide with our deep dive on running AI models on a VPS for the model-selection side.

VPS Specs That Matter for an AI API

An API server’s specs are driven by the model, not the API layer. The web server part is trivially light — it’s inference that eats RAM. Quantized open models (4-bit) are the sweet spot for VPS hosting:

Tier	Workload	RAM	CPU	Disk	Approx. cost
Starter	1–3B models, dev/test API, low concurrency	8 GB	2–4 vCPU	100 GB SSD	$10–20/mo
Sweet spot	7–8B models (Llama 3.1 8B, Qwen 2.5 7B), small production API	16 GB	4–6 vCPU	200 GB NVMe	$20–40/mo
Serious	13–14B models, higher concurrency, embeddings + chat together	32–64 GB	8–12 vCPU	400+ GB NVMe	$50–100/mo
GPU	Fast responses, 30B+ models, real-time feel	16–32 GB	4–8 vCPU + NVIDIA GPU	200+ GB NVMe	$80–300+/mo

Concurrency is the hidden spec. A 7B model on CPU might take 5–10 seconds per response with one user; with five simultaneous users, requests queue and everyone waits. If your API will see real concurrent traffic, either size up, add a GPU, or be honest about expected wait times. Our roundup of cheap VPS plans that don’t sacrifice performance covers good CPU tiers, and server location matters too — put the VPS near your users, because API latency is user-facing latency.

How to Host Your Own AI API on a VPS: Step-by-Step

This assumes a fresh Ubuntu 22.04 or 24.04 VPS. First run through our new VPS setup checklist — firewall, SSH keys, updates — then work as a non-root user with sudo.

Step 1 — Install Ollama and pull a model

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.1:8b

For an API, pick a model with strong instruction-following in its size class — Llama 3.1 8B and Qwen 2.5 7B are the safe defaults in 2026. Download once; the weights live on your disk and never phone home.

Step 2 — Make Ollama listen beyond localhost

By default Ollama binds to 127.0.0.1, which is fine for local testing. For a real API you want it reachable through your reverse proxy — but never expose port 11434 directly to the internet. Bind it to localhost and let Nginx be the only door in:

# /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"

sudo systemctl daemon-reload
sudo systemctl restart ollama
curl http://127.0.0.1:11434/api/tags   # should list your pulled model

Step 3 — Put Nginx in front with an API key

Ollama has no built-in API-key auth, so this is the step most guides skip and most self-hosted APIs regret skipping. Use Nginx as a reverse proxy (see our Nginx on Ubuntu guide) and require a key header:

# /etc/nginx/sites-available/ai-api
map $http_authorization $api_ok {
    default 0;
    "Bearer sk-your-long-random-secret-here" 1;
}

server {
    listen 80;
    server_name api.yourdomain.com;

    location / {
        if ($api_ok = 0) { return 401 '{"error":"unauthorized"}'; }
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
        # allow long streaming responses
        proxy_read_timeout 300s;
    }
}

Generate the secret properly — openssl rand -hex 32 — and rotate it like you’d rotate any credential. One shared key is fine for personal use; for multiple users, skip to the LiteLLM step below.

Step 4 — Add HTTPS with Certbot

sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d api.yourdomain.com
sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'   # HTTPS only from here on

Your API is now live at https://api.yourdomain.com/v1/chat/completions — the same path shape as OpenAI’s API. That’s the whole trick of OpenAI-compatible endpoints: existing client code barely changes. For the full hardening picture, work through securing your VPS.

Step 5 — Test it like a client would

Test the exact interface your apps will use — the OpenAI-compatible chat completions endpoint:

curl https://api.yourdomain.com/v1/chat/completions \
  -H "Authorization: Bearer sk-your-long-random-secret-here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}]
  }'

And the one-line change in existing Python code (the OpenAI client just needs a new base URL):

from openai import OpenAI

client = OpenAI(
    base_url="https://api.yourdomain.com/v1",
    api_key="sk-your-long-random-secret-here",
)
resp = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Explain VPS hosting in one sentence."}],
)
print(resp.choices[0].message.content)

If that works, you have a private, OpenAI-shaped API. Reference the official OpenAI API reference for endpoint shapes — the compatibility means their docs double as your docs.

Step 6 (optional) — Graduate to LiteLLM for keys, routing, and spend tracking

The day you need per-user keys, usage dashboards, or fallbacks (“use my VPS first, overflow to a paid provider”), add LiteLLM as a proxy in front of Ollama. One docker-compose file:

# docker-compose.yml
services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports:
      - "127.0.0.1:4000:4000"
    volumes:
      - ./litellm_config.yaml:/app/config.yaml
    command: ["--config", "/app/config.yaml", "--port", "4000"]

# litellm_config.yaml
model_list:
  - model_name: my-chat
    litellm_params:
      model: ollama/llama3.1:8b
      api_base: http://host.docker.internal:11434
general_settings:
  master_key: sk-master-secret-here   # keep this in a vault, not in chat logs

Point Nginx at port 4000 instead of 11434, and you get a key-management UI, per-key budgets, and request logging — the operational layer a real API needs. This is also where our n8n automation server guide pairs well: point your n8n AI nodes at your own endpoint and your automation costs drop to the VPS bill.

Cost Math: Self-Hosted vs Pay-Per-Token (Honest Numbers)

Here’s the comparison everyone wants, with the assumptions stated. A 16 GB VPS at ~$30/month serving a 7–8B model:

Scenario	Self-hosted (16 GB VPS)	Paid API (typical mid-tier model)
Light use — ~50k tokens/day (personal projects, dev)	$30/mo (mostly idle)	~$3–8/mo
Moderate — ~1M tokens/day (small app, internal tool)	$30/mo	~$60–150/mo
Heavy — ~10M tokens/day (real product traffic)	$30–80/mo (bigger VPS or GPU)	~$600–1,500/mo

The break-even is roughly 300–500k tokens per day on a CPU VPS — below that, pay per token and save yourself the ops work. Above it, self-hosting wins by an order of magnitude, and the gap widens the more you scale. The other honest cost: your time. Budget a weekend for setup and an hour a month for updates. If your time is worth more than the savings, that’s a valid reason to stay on a paid API.

Model quality is the real tradeoff, not money. An 8B open model is excellent for chat, summarization, classification, and RAG — but it won’t match frontier models on hard reasoning or niche knowledge. Be honest about what your API needs to be good at before you commit.

Security Checklist: Don’t Skip This

An API endpoint on the open internet with a model behind it is a cryptominer’s dream and a spammer’s playground. Minimum viable security:

• Never expose Ollama directly. Port 11434 should be localhost-only, always. Every request goes through Nginx.
• Require the key on every route. The Nginx map above returns 401 without it. Test this — curl without the header and confirm you get rejected.
• HTTPS only. API keys in plaintext HTTP are credentials handed to anyone on the network path.
• Firewall everything else. UFW with only 22, 80, and 443 open; SSH key-only, no passwords.
• Rate-limit per key. Even a simple Nginx limit_req zone stops one runaway client from melting your CPU. LiteLLM adds per-key budgets.
• Watch the logs. A sudden spike in 401s means someone found your endpoint and is guessing keys. fail2ban on the Nginx log is cheap insurance.
• Keep Ollama updated. Model servers have had real CVEs; apt updates and the occasional Ollama reinstall are part of the deal.

Common Mistakes That Bite Self-Hosted API Owners

Under-provisioning RAM. The #1 failure: a 7B model on an 8 GB VPS that also runs the database, the web app, and the kitchen sink. The model gets OOM-killed under load and your API returns 500s. Give the model its own headroom — 16 GB for an 8B model isn’t generous, it’s the minimum for production.

No request timeouts. A stuck inference request holding a worker forever will queue everything behind it. Set timeouts at every layer: Nginx’s proxy_read_timeout, your client library’s timeout, and a queue with a max length.

Streaming ignored. For chat APIs, stream tokens ("stream": true) instead of waiting for the full response. It doesn’t make inference faster, but perceived latency drops dramatically — users see words in under a second instead of staring at a spinner for ten.

Treating it like a fire-and-forget deploy. Models get updated, disks fill with logs, certs renew (or don’t). Put a monthly 30-minute maintenance reminder on your calendar. Boring, but it’s the difference between an API that runs for years and one that mysteriously dies in month three.

FAQ

Can I really use my existing OpenAI code with a self-hosted API?

Yes — that’s the main reason to use Ollama or LiteLLM. They implement the /v1/chat/completions shape, so you change the base URL and model name and keep everything else. Edge features (fine-tuning endpoints, assistants API, some tool-call formats) may not map 1:1, so test your specific calls.

Do I need a GPU to host an AI API?

No, but it changes the experience. CPU inference on a 7–8B model gives you a few tokens per second — fine for background jobs, summaries, and low-traffic tools. If users wait on responses in real time, a GPU VPS (or a smaller model) is the difference between “usable” and “painful.”

How many requests can a $30 VPS handle?

Rule of thumb for an 8B model on a 4–6 vCPU box: a handful of concurrent chat requests with multi-second responses each. It’s a queue, not a wall — ten simultaneous users means the tenth waits. For bursty traffic, add a GPU or run a smaller model (3B) for the fast path.

Is it legal to serve open models as an API?

Generally yes, but check each model’s license. Most popular open weights (Llama 3.x community license, Qwen, Mistral) allow commercial API use with conditions — Llama’s license, for example, has terms for very large-scale services. Read the license of the specific model you deploy; “open weights” is not the same as “no strings attached.”

What about embeddings and image models?

Same pattern, different server. Ollama serves embedding models too (same API shape), and you can run a second Ollama instance or a dedicated embeddings endpoint alongside chat. Keep heavy image generation on a GPU tier — CPU image APIs are a patience test, as our AI website hosting guide notes for media workloads.

Conclusion

To host your own AI API on a VPS is to trade a variable bill for a fixed one and a dependency for a responsibility. For steady traffic above a few hundred thousand tokens a day, the math is overwhelmingly in your favor — one $30 VPS replaces a triple-digit monthly API bill, your data never leaves your machine, and your endpoints don’t change because someone else’s roadmap said so.

Start with Ollama behind Nginx this weekend: key auth, HTTPS, one model, real traffic. Add LiteLLM when you need keys and routing. And keep the server boring — updated, firewalled, monitored — because the best API is the one you forget you’re running. That’s the whole game: your models, your keys, your rules.