Open-weight AI models now trail the best closed models by about four months on average. That figure comes from Epoch AI’s May 29 analysis covering the period since January 2026.
“Open-weight” means a company publishes the model’s weights, the files that hold what the model learned. Anyone can download those files and run the model on their own computer.
Two free programs, Ollama and LM Studio, handle the setup. Paid chatbot subscriptions cost about $20 a month, while these tools cost nothing.
A cloud chatbot sends every prompt to a company’s servers. A court ordered OpenAI to preserve ChatGPT logs, including deleted ones. A local model keeps conversations on the user’s own computer.
LM Studio’s documentation says the app “can operate entirely offline, just make sure to get some model files first.” Documents attached to a chat can also be used offline.
Licenses decide what users can do with downloaded models. DeepSeek’s R1 series uses the MIT License, while Google’s Gemma 4 and OpenAI’s gpt-oss use Apache 2.0. Both licenses allow changes and commercial use.
What Hardware Local AI Models Need
Models are measured in parameters, and bigger models mean bigger downloads. A model must fit in a graphics card’s video memory, called VRAM, or in a Mac’s unified memory.
Small models need about 8GB of VRAM. Heavier users may want a graphics card with 24GB or more, or machines with unified memory such as the Mac Mini or the Nvidia DGX Spark.
A method called quantization shrinks models by storing each weight with fewer bits. This cuts memory use but costs some accuracy.
LM Studio recommends 16GB or more of RAM on Apple Silicon Macs running macOS 14 or newer. On Windows, it recommends 16GB of RAM and 4GB of dedicated VRAM. Intel-based Macs are not supported.
Ollama lists Gemma 4 downloads from about 4.6GB to 20GB. OpenAI says gpt-oss-20b needs 16GB of memory, and gpt-oss-120b runs within 80GB.
How to Get Started With Ollama and LM Studio
LM Studio offers a simple graphical interface. Users download the app from lmstudio.ai, search for a model, download it and start a chat.
Ollama is aimed at developers who prefer a command line. Typing “ollama pull gemma4:e2b” downloads a model, and “ollama run gemma4:e2b” opens a chat.
Models ending in “:cloud” run on Ollama’s servers instead of the user’s machine. Users focused on privacy should skip those.
Google’s Multi-Token Prediction drafters for Gemma 4 can speed up responses by up to 3x. A small model guesses several tokens ahead, and the larger model checks them all at once.
Local models still have limits. Responses slow down when a model spills out of VRAM into system RAM, and larger context windows need more memory.
The biggest open models are far out of reach for home users. Alibaba’s Qwen 3.8-Max has 2.4 trillion total parameters, and in Alibaba’s own comparison, closed models won 24 of 31 tests.
