Local LLM on Linux
Run an LLM locally on Linux
A large language model can run entirely on your Linux machine: private, offline and free. What you need, which model to choose, and the simplest way to start, with an AppImage or a .deb package.
Why run an LLM on your machine
The model answers and reads your files on the machine. Once the model is installed, it works without internet. There is no subscription, no quota, and no account needed. The trade-off: a model that fits a personal computer is smaller than the largest online models — excellent for writing, summaries and your documents, less so for the hardest reasoning.
What you need
An x86_64 PC running Linux. On the processor, the computer's memory (RAM) decides: 8 GB for about 4 billion parameters, 16 GB for 8 billion, 24 GB for 14 billion. Fast memory matters: DDR5 writes faster than DDR4.
Which model to choose
Qwen3.5 writes well in many languages at every size; Llama, Gemma, Mistral and gpt-oss are good alternatives. On the processor, stay with the small models, of about 4 to 9 billion parameters: a larger one fits in memory but writes too slowly for comfort.
What Twoody does for you
- It suggests the model that suits your machine and installs it in one click; its catalogue holds 57, each with the memory it needs.
- It reads your folders of documents, and its answers cite their sources, down to the page.
- As an AppImage or a .deb package, tested on Ubuntu 24.04; the model runs on the processor.
What differs from the Mac and Windows
It is the same app, with the same conversations and documents. On Linux, the model runs on the processor: Twoody does not use the graphics card there. Updates are manual for now: you download the new version yourself. And a few things are Mac only: reading scanned PDFs, the Mac's Calendar and Contacts, dictation by macOS speech recognition (on Linux, your voice is transcribed on the machine by whisper.cpp, which ships with the app) and MLX.
With Twoody, in three steps
-
1
Join the waitlist: Twoody is in private beta, on Linux as on the Mac and Windows, and we will write to you when it opens to you.
-
2
Install the model it suggests for your machine, in one click: 3.5 GB for Qwen3.5 4B, 6.7 GB for the 9B.
-
3
Ask your first question — and turn Wi-Fi off if you like.
Frequently asked questions
Is running an LLM locally on Linux free?
Yes: the open-weight models are free to download, and Twoody is free. The only cost is your machine's memory and electricity.
Does Twoody use the graphics card on Linux?
No: on Linux, Twoody runs the model on the processor, so stay with the small models, of about 4 to 9 billion parameters. An OpenAI-compatible server running on your machine with the graphics card can also serve Twoody: Models › “Advanced” › “Engine” › “Other address”.
Which distributions does Twoody run on?
Twoody is tested on Ubuntu 24.04 (x86_64). The AppImage runs on most distributions; the .deb package installs Twoody on Debian, Ubuntu and their derivatives.
How much memory do I need to run an LLM on Linux?
8 GB for a small model such as Qwen3.5 4B. On Linux, Twoody suggests Qwen3.5 9B from 32 GB; from 16 GB, you can choose it yourself. The calculator shows what a PC without a graphics card runs, with DDR4 or DDR5 memory, and how fast.
Can I use Twoody on Linux today?
In the private beta, yes: Twoody runs on Linux (x86_64), as on the Mac and Windows. Its downloads are for the people in the beta: leave your email to join the waitlist.
Try Twoody before it opens to everyone.
The private beta is free, on Mac, Windows and Linux, and on iPhone and Android with Twoody on your computer. Leave your email to ask for an invitation: we will write to you when it is ready for you.