LM Studio is a popular app for running large language models (LLMs) locally on your computer.You can download a model, load it in LM Studio, and start chatting.LM Studio isn't the only option; I tried out three open-source alternatives, and each of them was useful in different ways.
Jan.ai is the most direct LM Studio alternative Download a model and start chatting If you're looking for a direct like-for-like replacement for LM Studio, Jan.ai is the best option of the three I tried.It feels like a proper desktop app; when I installed it, it automatically downloaded the default Jan model, and I was able to start chatting just like I can with ChatGPT or Claude.The first response was a little slow to appear as the model loaded, but after that the responses were reasonably fast, even running on my MacBook Air M2 with only 8GB of RAM.
The default model was mostly fine, although it knew frustratingly little about how to use the Jan.ai app, which was one of the first things I wanted to know.While the default model is fine for general chats, you can download and run other models.You can browse a list of models in the Hub, which is similar to how it works in LM Studio.
I did have trouble getting some models to run properly, with MLX models that are optimized for Apple Silicon failing to load, meaning I had to resort to downgrading to the previous release to get them working.As with LM Studio, you can add custom instructions to determine how the chatbot should respond.In Jan.ai, this is done by creating assistants with custom instructions that you call on within chats.
If you want a fairly simple way to set up a local chatbot running on your own computer, Jan.ai is a solid choice.Ollama is the best option for a home server It's more infrastructure than AI app Ollama is quite a different beast from Jan.ai or LM Studio.Launching the app gives you a recognizable AI chatbot interface, but it's noticeably lacking in any real features.
The menu has two options: New Chat and Launch, and there's a drop-down in the chat window with a small handful of downloadable and cloud models.All of the listed downloadable models were too large to work on my M2 MacBook Air with 8GB of RAM, so out of the box, the app was effectively useless to me for running a local LLM.The app's model management is limited; if you want to do more, you're better off using the command-line interface.
Related Ollama 0.10 Speeds up Local AI Models, Introduces Desktop App There are dozens of desktop app wrappers around Ollama, and now Ollama itself has made one.Posts By Corbin Davenport You can install other models in Terminal and use them in the Ollama app, or chat with them directly in Terminal, but this isn't the real purpose of Ollama.It's not intended primarily as a chatbot app; it's a tool for running and managing local models that you can then access with a local API.
I can run a local LLM on my Mac using Ollama and let other apps and services talk to it.For example, I can use a local LLM running in Ollama as a conversation agent in Home Assistant, effectively giving my voice assistant AI-powered abilities without any of my data having to leave my home.Claude Price $20 Claude is an AI assistant made by Anthropic. It can assist with a wide range of tasks—writing, coding, analysis, research, and more. Unlike a search engine, Claude reasons through problems conversationally, making it useful as a thinking partner rather than just an information retrieval tool.
See at Claude Expand Collapse AnythingLLM is great for working with documents Create a private assistant for your notes and files AnythingLLM is yet another way to use a local LLM.The main focus of AnythingLLM is to help you build assistants that can work with your own documents and data.Its core capabilities include local retrieval-augmented generation (RAG), which lets you upload files such as PDFs, Word documents, text files, and more, and then talk to a chatbot about their contents.
For example, I uploaded the rules to the game Settlers of Catan and was then able to ask the chatbot questions such as how many victory points I'd get for having the largest army, or what resources I'd need to build a road, and it was able to pull the information from the PDF.The problem was that on my Mac, the response was painfully slow, and this was only searching through a single PDF.With more documents, it would quickly become unusable on my hardware.
AnythingLLM can be used with cloud-based models, too, and you can combine models.For example, I could use a local LLM to redact personal information from my bank statements and then a cloud model to analyze them.It also has web scraping capabilities, although when I tried to get it to find me the latest deals on games from Deku Deals, it just flat-out told me that it couldn't do it.
There's also a pop-out assistant that you can summon when using any app, which will take a screenshot of the app you're using so that you can ask questions about it.On my MacBook Air, it was too slow to be of any use.If I had better hardware, I could see how AnythingLLM could be really useful, but my Mac just isn't up to the job.
I'm sticking with Ollama It's not really a true LM Studio alternative Having given all three a run for their money, the one I'm going to stick with is Ollama.While it's probably the least like LM Studio in terms of easily loading a model and starting to chat, it's the most useful for what I need.It essentially gives you API access to an LLM without having to pay.
While I can only use fairly lightweight models, they can still do some useful jobs.For example, I can use it in an n8n automation to clean up the data I pass to the automation before it gets sent to a Notion database, without having to worry about my data being exposed outside my local network.Local LLMs are a privacy godsend If you only have modest hardware, then a local LLM won't be able to match the performance of popular cloud-based LLMs.
What you lose in power, however, you gain in privacy.
Read More
