A private ChatGPT-style chat on your own laptop with open models
A private local chat runs an open language model on your own computer. Ollama downloads and runs the model, and Open WebUI adds a familiar browser chat on top. Your prompts do not leave the machine, and once the model is downloaded it works without a connection, at the cost of some quality and speed.

Some things you would hesitate to paste into a public site: a draft contract, meeting notes, a half-formed idea. A local model gives you somewhere to put them. This guide covers why it is worth doing, what your hardware must carry and the exact steps from mission 7 of the course.
Key ideas
- A local model keeps prompts on your machine and works offline after download.
- The trade is quality and speed: it is usually smaller than a hosted model.
- Two tools do the job: Ollama runs the model, Open WebUI gives the chat window.
- Local does not mean careless: keep passwords, keys and client secrets out anyway.
Why run a model locally
Two reasons stand out. Privacy: your prompts do not travel to someone else’s server. Independence: after the first download it works without a connection. Both matter when you work with drafts, notes or anything you would hesitate to paste into a public site.
The trade-off is honest. A local model is usually smaller than a hosted one, and your hardware sets the limit. The sensible setup keeps both: the local model for private drafts and notes, a hosted one when you need the strongest answer.
What your computer can carry
A model has to fit in memory to run at a usable speed. That means graphics memory if you have a dedicated card, or shared memory on an Apple-silicon laptop. Smaller models and compressed, or quantised, versions fit more easily and lose some quality.
Ask any local runner for the size of a model before you download it, because the files are large and a wrong choice wastes an evening. Start small and move up only if the answers are not good enough: a fast small model you use daily beats a large one you avoid. Whatever an advisor tells you, test its first recommendation for ten minutes with real work.
The steps: Ollama, then Open WebUI
On macOS or Linux, install Ollama with curl -fsSL https://ollama.com/install.sh | sh (Windows has a PowerShell line in the README). Then run a model with ollama run gemma4. The first run downloads it; type /bye to leave. Ask it something about your own work that involves no secrets, and notice the speed and quality.
For the chat window, install Open WebUI with pip install open-webui (the README asks for Python 3.12 on this route), then start it with open-webui serve and open http://localhost:8080 in your browser. Create the first account, which stays on your machine, choose the model you pulled and send a test message. The pieces connect in a line: browser, Open WebUI, Ollama, local model.
Make it yours, and know the limits
A local model has no memory of you and no web by default, so write a system prompt: your role, your format, and honesty rules such as saying “I am not sure”, never inventing sources and never pretending it browsed. These rules matter even more on a small model, which will otherwise answer everything with confidence.
Local does not mean anything goes. Keep passwords, keys, client names and personal documents out of every tool, including this one. Mission 7, “A private ChatGPT on your own laptop”, builds the whole thing step by step and ends with a private chat running on your machine.
In everyday language
A hosted chat is like a restaurant: someone else’s kitchen, and your order travels there. A local model is cooking at home. The menu is smaller and the stove is slower, but nothing leaves your kitchen.
Try it yourself
Install Ollama, run ollama run gemma4 and ask it one question about your own work that involves no secrets.
Expected result
A model answering offline in your terminal, with a feel for its speed and quality on your hardware.
Check your answer: Why should you start with a small model instead of the largest one?
Because a small, fast model you actually use daily beats a large one that is too slow, and you can always move up later.
Questions
Is a local model as good as ChatGPT or Claude?
Usually not. It is typically smaller than a hosted model, so use it for drafts, notes and private material, and a hosted model when you need the strongest answer.
Is everything safe to paste into a local model?
Do not paste passwords, keys or client secrets into any tool. Local keeps prompts on your machine, but good habits still apply.
A private chat on your own laptop takes one evening. Mission 7 of the VITON13 SCHOOL AI course shows each step, and the course’s first two missions are free so you can see how it works before you decide.
Sources and further reading
- VITON13 SCHOOL — Course 01: Artificial Intelligence ↗Sources checked:
- Ollama on GitHub ↗Sources checked:
- Open WebUI on GitHub ↗Sources checked: