Try Turborg Free → Open App →
← All posts

Stop Paying for Every API Call: Run Borg with Your Own Local LLM

October 9, 2026 · 4 min read · The xShellz Team

TL;DR

  • borg is a single Go binary you install with curl -fsSL https://turborg.com/install.sh | sh, and it supports bring-your-own-model providers like Ollama and LM Studio.
  • You switch borg from OpenAI to Ollama by editing ~/.config/borg/ or exporting a BORG_* environment variable, then starting borg as usual.
  • Running borg on a local model costs no per-token API fees: you use your own hardware, and the four hosted models from xShellz remain optional.

To use borg with Ollama instead of OpenAI, install borg, then configure it to point at your local Ollama endpoint in ~/.config/borg/ or via a BORG_* environment variable, and run borg or borg "your task". We build and operate xShellz (the borg terminal agent, the Agent Shell boxes and the hosted model gateway behind them), and everything below is what we run in production ourselves, not a demo.

how do i use borg with ollama instead of openai?

Install borg, start your Ollama server, then tell borg to use that local endpoint instead of an OpenAI key. The borg binary is a single static Go file, so there is no runtime to set up beyond the model provider. The config lives in ~/.config/borg/ and accepts BORG_* environment variables.

curl -fsSL https://turborg.com/install.sh | sh
borg

The first line installs borg, the second starts the REPL in your current directory. After you point borg at Ollama in the config, every prompt goes to your local model.

how to connect borg ai agent to local ollama

Borg treats Ollama as one of its bring-your-own-model providers, so you do not need a hosted account. Edit ~/.config/borg/ to set the provider to Ollama and fill in the address where Ollama listens plus the model name you pulled with Ollama. If you prefer environment variables, export the matching BORG_* entries before you run borg. Once that is done, a plain borg session sends its prompts to Ollama. No API key changes hands because the model runs on your machine.

can i run borg with a local model for free

Yes. Local models like Ollama and LM Studio charge nothing per token: the inference happens on your own CPU or GPU, so the only costs are your hardware, power, and the one-time model download. The borg binary installs via the curl command at https://turborg.com/install.sh and does not require a paid plan. The hosted xShellz models (Chuppa Flash, Chuppa Pro, Floko, Axiom) remain optional, but nothing forces you to use them. See https://www.xshellz.com/borg for the full model list.

how to point borg to lm studio endpoint

LM Studio runs as a local server and exposes an OpenAI-compatible endpoint on your machine. Borg lists LM Studio as a supported provider, so the setup mirrors the Ollama case: in ~/.config/borg/ or via a BORG_* environment variable, set the provider to LM Studio, enter the endpoint and port LM Studio prints in its UI, and pick the model you loaded. Then run borg as usual. No API key is involved because the request never leaves your network.

is it possible to use borg with my own api key for local llms

Local LLM servers like Ollama and LM Studio do not use API keys: they accept requests from localhost by default, and borg connects directly. If you run a self-hosted model behind your own authentication layer, borg's config can hold that key, but none is required for the standard local setups. So you typically skip borg auth login entirely; that command is for the hosted xShellz models, not for Ollama or LM Studio.

what are the performance trade-offs of running borg on local models

The main trade-off is speed and model quality. A local 7B or 13B model runs with no network latency but is less capable than a frontier hosted model, and larger local models need more RAM or VRAM. Borg itself adds almost no overhead: it is one Go binary, so the model inference time dominates. If you want more capability without API bills, you can run the same local model on a bigger machine, including an always-on Agent Shell box (https://www.xshellz.com/agent-shell) with borg preinstalled. The sandbox runs on gVisor (https://gvisor.dev/docs/), so root access stays contained.

Frequently asked questions

Does borg work offline once I set up Ollama?

Yes. After you install borg and download your Ollama model, both the agent and the model run on your machine. No internet connection is needed for borg sessions, and no prompts leave your network. The initial install via curl and the model pull are the only online steps.

Can I switch between Ollama and a hosted model without reinstalling borg?

Yes, the borg binary is the same for every provider. Edit ~/.config/borg/ or change the BORG_* environment variable to point at a different model, then restart borg. There is no separate install or plugin; borg reads the provider from its config each time you launch it.

Do I need a GPU to run borg with Ollama?

No. Ollama runs on CPU as well as GPU, and borg simply sends prompts to whatever Ollama serves. Larger models will be slower on CPU, but small 7B models are usable for many terminal tasks on a laptop. A GPU improves speed, not correctness.