Run a local Ollama model and check its chat API

Ollama can run a downloaded model on your computer and serve it at a local HTTP address. This guide takes one model from installation through terminal chat and a complete API reply, using the same model name at each step. It is a documentation-based procedure: BIG CHANGE has not installed the software or observed the results on your machine.

The big change

• What changed: Ollama offers a direct route from a downloaded model to both terminal chat and a local API. A beginner can check the model name and response at each interface instead of assuming that a chat window proves where inference ran.
• Why it matters: The local server and Ollama's cloud API are different destinations. Someone handling an approved work or personal task can keep this exercise on their own computer by choosing a downloaded model and addressing localhost, while checking the server's configuration separately.
• What to watch: A local URL alone does not establish that every feature or connected app stays local. Ollama can also use cloud models through its local server. The decisive checks are the installed model tag, the request destination, the server bind address and, when strict local-only operation is required, the cloud-disable log after a restart.

Before you start

Use a computer where you are allowed to install software. Pick a harmless factual question whose answer you can check in an original document you have open. Paste that question when the commands ask for a practice prompt; do not enter private or regulated material merely to test the setup. Keep the document available to check the answer, because a fluent reply is not proof of accuracy.

You need storage for Ollama and the model, enough memory to load your chosen model, and an internet connection for installation and download. Ollama's quickstart uses gemma4:e2b as its local example and recommends 8 GB of available VRAM, or unified memory on a Mac. Ollama says it can fall back to system RAM with slower responses. Inspect the current model listing for the exact tag, download size and license before proceeding. A download size does not establish that your computer can run the model. If it does not fit your machine or permitted use, choose a smaller downloadable local model from the library and replace gemma4:e2b in every command below with its exact tag. Do not select a :cloud tag for this exercise.

1. Install Ollama and check the local server

Get Ollama from its official download page and follow the instructions for your operating system: Windows, macOS or Linux. On Windows and macOS, open the installed app. On Linux, follow your chosen installation method and start its service; if no service is running, the quickstart says to run ollama serve in a separate terminal. Do not start a second server over an app or service that is already listening.

In a new terminal, check the installed version and that the local server responds:

ollama -v
ollama ls

Record the actual version and operating system. The model list may be empty before your first pull; a command or connection error means you should resolve the installation or server startup before continuing. Ollama's FAQ says its default listener is 127.0.0.1:11434. Leave it on loopback for this exercise; do not configure a network bind, proxy or tunnel.

2. Download one exact local model

After checking the model listing, run:

ollama pull gemma4:e2b
ollama ls

The exact tag must appear in ollama ls after the pull. Record the tag, listed size and the license shown on its model page. If the pull fails or the tag is absent, check the spelling against the current library, free disk space, network access and any proxy or permission error before retrying. A partial download is not a successful setup.

3. Check interactive and one-off terminal chat

Start an interactive chat:

ollama run gemma4:e2b

Paste your approved practice question at the chat prompt. Inspect the reply against the document you used to make the question, then type /bye to exit. Keep the question and actual reply for your own check; this article supplies no sample result.

For a separate one-off run, use the same question. In PowerShell on Windows:

$practicePrompt = Read-Host 'Paste your approved practice question'
ollama run gemma4:e2b $practicePrompt

In a Bash terminal on macOS or Linux:

read -r -p 'Paste your approved practice question: ' practice_prompt
ollama run gemma4:e2b "$practice_prompt"

Check the actual one-off answer against the same source. The two answers need not use identical wording; the useful check is that both commands ran the intended tag and that any factual claim survives comparison with the source.

4. Ask the same model through localhost

The chat API takes a model name and messages at POST /api/chat. It streams by default; stream: false asks for one complete JSON response. The API introduction distinguishes the local base URL, http://localhost:11434/api, from the direct cloud URL, https://ollama.com/api. The local request needs no API key.

On Windows, PowerShell can build the JSON safely from the question you paste:

$practicePrompt = Read-Host 'Paste the same approved practice question'
$body = @{ model = 'gemma4:e2b'; messages = @(@{ role = 'user'; content = $practicePrompt }); stream = $false } | ConvertTo-Json -Depth 5
$response = Invoke-RestMethod -Uri 'http://localhost:11434/api/chat' -Method Post -ContentType 'application/json; charset=utf-8' -Body $body
$response | Select-Object model, done
$response.message.content

On macOS or Linux, the following Bash example uses Python 3 only to escape your prompt into JSON. Check that python3 and curl are installed before using it:

read -r -p 'Paste the same approved practice question: ' practice_prompt
export practice_prompt
python3 -c 'import json,os; print(json.dumps({"model":"gemma4:e2b","messages":[{"role":"user","content":os.environ["practice_prompt"]}],"stream":False}))' | curl -fsS 'http://localhost:11434/api/chat' -H 'Content-Type: application/json' --data-binary @-

The request should return HTTP success and one JSON object whose model matches your downloaded tag, done is true, and message.content is nonempty. Read and check the actual answer against your source. A successful response establishes this request's local destination and selected model; keep your own output rather than relying on an example transcript.

5. Check placement and the local-only boundary

While the model is loaded, run:

ollama ps

The PROCESSOR column reports whether the model is loaded on the CPU, GPU or both, according to Ollama's FAQ. Record what your machine actually shows. If it has already unloaded, send another prompt and check again. You can unload it when finished with ollama stop gemma4:e2b; removal with ollama rm gemma4:e2b is optional disk cleanup after you have saved your observations.

For a strict Ollama local-only setup, follow the official cloud-disable instructions: put disable_ollama_cloud in ~/.ollama/server.json for the account running the Ollama server (the .ollama folder in that account's user profile on Windows), retaining any other existing JSON settings:

{
  "disable_ollama_cloud": true
}

On Linux, the documented systemd service runs as User=ollama, whose home is /usr/share/ollama; editing your own home file would not configure that service. For a systemd installation, you can instead use Ollama's documented OLLAMA_NO_CLOUD=1 alternative in a service override. First inspect the existing service with systemctl cat ollama.service, then open an override with sudo systemctl edit ollama.service. Add the following under [Service], preserving any other override settings:

[Service]
Environment="OLLAMA_NO_CLOUD=1"

Save the override, then reload and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama.service

Restart the Ollama app on Windows or macOS after changing its configuration. Check the server log for the exact line Ollama cloud disabled: true. On Windows the documented log is %LOCALAPPDATA%\Ollama\server.log; on macOS it is ~/.ollama/logs/server.log; on a Linux systemd installation use journalctl -e -u ollama. An environment variable set in an unrelated terminal will not necessarily reach an already running app or service. Disabling cloud features removes Ollama cloud models and web search. If the log does not confirm the setting, fix the configuration and restart before describing the setup as local-only.

Confirm the listener is still on 127.0.0.1:11434 in the server log or with your operating system's listener inspection, and repeat the localhost request with the downloaded tag. OLLAMA_HOST can change the bind address, so inspect the running server rather than assuming its default. A separate Python library, browser interface, coding agent or other connected application needs its own endpoint and data-path check; Ollama's local response does not establish that application's privacy behavior.

On Windows, inspect the listening address in PowerShell:

Get-NetTCPConnection -LocalPort 11434 -State Listen | Select-Object LocalAddress, LocalPort

On macOS, use lsof -nP -iTCP:11434 -sTCP:LISTEN; on Linux, use ss -ltn '( sport = :11434 )'. If a platform tool is unavailable, check the server log. A wildcard or non-loopback listener needs investigation before treating the API as confined to this computer.

If a check fails

If ollama -v or ollama ls fails, return to the applicable operating-system installation and startup page. A refused localhost connection usually means the app or service is stopped or listening somewhere else; verify its logs and bind address instead of exposing the API on a network. If a response arrives as multiple JSON lines, make sure your request contains "stream": false. If the model runs out of memory or is impractically slow, check ollama ps and select a genuinely smaller local tag from the library, then repeat the pull, terminal and API checks with that one tag. Treat any stage you have not completed as unverified.

This local exercise needs hardware, electricity and download storage, but no Ollama API key or paid API account. A model's license can limit your intended use. Cloud models and optional third-party clients have separate terms and data paths. For a deeper hardware purchasing comparison, see our 128 GB local-model guide; for a distinct coding task that uses Ollama with Claude Code, see our checked-change guide.

Sources & further reading

• Ollama quickstart explains the installation route, local gemma4:e2b example and memory recommendation. Its example is documentation, not a test of your machine.
• CLI reference lists the current pull, run, ls, ps, stop, rm and serve commands used here.
• API introduction and chat endpoint separate the local and cloud addresses and define the nonstreaming request and response fields.
• FAQ documents loopback binding, processor placement and the cloud-disable setting and log check.
• Gemma 4 E2B listing provides the current exact model tag, size and license materials to check before downloading.