The problem
Most AI assistants send every conversation to a cloud server. I wanted an assistant that is fast to open, looks premium and keeps conversations private by running the model locally.
Demo: a real run
Three screenshots from one real conversation on my PC on 8 November 2026. The model ran offline, and nothing is staged or edited apart from cropping to the app window.
The app opens and starts the local engine in the background.
I ask a question. The thinking indicator shows while Gemma 3 12B generates on my GPU and CPU.
The answer appears as a clean message. This first reply after launch took between 75 and 135 seconds, including model loading.
A screen-recorded video walkthrough will be added here once it’s recorded.
How it works
Four layers, all on the same computer:
- Interface: Qt Quick (QML). The chat view, suggestion cards, New Chat and the frameless navy-and-gold window.
- Bridge: a C++ backend exposed to QML. It starts the engine with
QProcess, writes each message to the engine’s standard input and reads the answer from its standard output. - Engine: a C++ console app built with CMake on llama.cpp. It loads the model with the Wynner system prompt, a 4,096-token context and 22 layers offloaded to the GPU, and strips the model’s thinking text.
- Model: Gemma 3 12B Instruct (Q4_K_M GGUF, 7.3 GB on disk), stored locally. No conversation is sent to a server.
Feature status
Last checked on 8 November 2026, on my Windows PC.
Implemented and tested offline
- Runs Gemma 3 12B fully on my PC, with no internet connection needed for chat.
- A real round trip: asked who it was, it answered “I’m Wynner, a local AI assistant created by Wynn.”
- Chat view with message bubbles and a “Wynner is thinking…” indicator.
- Suggestion cards that pre-fill prompts. New Chat clears the conversation.
- Frameless navy-and-gold window with custom window controls.
Measured
- The first reply after launch took 75–135 seconds, because the model loads first (about 6 GB of RAM).
- Later replies in the same session are faster.
Not built yet
- The AI Assistant, AI Models, Documents, Data Insights, Tools and Settings pages. The sidebar items are visual only.
- Attachments and code tools in the chat box.
- A packaged installer and a public release.
My role and AI-assisted development
What I did
- Came up with the product and its rule: chat must stay on the device.
- Designed the navy-and-gold look as a mockup, and asked for changes until the app matched it.
- Tested it with real questions and decided what is ready to show.
What AI coding assistants did
- Wrote and refactored much of the C++ and QML code under my direction.
- The latest redesign and the reply-parsing fix were made with Claude Code, then checked by running the app.
Challenges & solutions
The engine first prints Wynner: [thinking...] and then rewrites the line, so the app displayed the placeholder text. The backend now waits for the next You: prompt and takes the last Wynner: answer before it, so only the clean reply appears.
In the first version, replies only reached the debug console and the send button did nothing. I added a proper chat model in QML, so messages and answers render in the window.
The interface was redesigned from black and white to navy and gold, with a gear-and-bridge emblem drawn as a high-resolution asset so it stays sharp at any size.