I have a subscription to Claude and use the free tier of Gemini.Both are incredibly powerful tools that I use on a daily basis, but I also run a small local model on my modest mini PC.I use this local LLM for almost all of my automated tasks, and despite the relative lack of power, it can offer things that Claude and Gemini can't.
Claude, Gemini, and ChatGPT come with API costs Even with a subscription, you need to pay for usage One of the things that annoys me most about paying for an AI subscription is that for many use cases, I can't take advantage of that subscription at all.For example, if I wanted to use an AI model in Home Assistant for things such as generating descriptions of the people who come to the front door or as the conversation agent for my voice assistant, I have to use the API.Frustratingly, API calls aren't covered by most AI subscriptions.
Despite paying for a Claude monthly plan, if I use the Anthropic API for AI tasks in Home Assistant, I have to pay additional API fees.The beauty of a local LLM is that there are no subscription costs and no API costs; everything runs for free on my local hardware.It means I don't have to worry about blowing a fortune in API fees or getting annoyed that I'm paying twice to use essentially the same service.
Claude Price $20 Claude is an AI assistant made by Anthropic. It can assist with a wide range of tasks—writing, coding, analysis, research, and more. Unlike a search engine, Claude reasons through problems conversationally, making it useful as a thinking partner rather than just an information retrieval tool.See at Claude Expand Collapse A local LLM keeps things private No need to share sensitive data with third parties This is one of the biggest reasons why my local LLM often gets used more than Claude or Gemini.Any messages you send to a cloud-based AI chatbot get sent to the cloud for processing, so all of that data ends up on third-party servers.
Even messages you enter in the chat field but never actually send may end up on those servers, and these messages could include sensitive data such as API keys, credit card information, personally identifiable information, photos of yourself or family members, and more.With a local LLM, everything stays on your local network, so you don't have to worry about the highly accurate profile that an AI company is capable of building based on all of your interactions with the chatbot.I use my local LLM for generating spoken morning briefings that include information about my kids and times and dates of when we're away from home.
I wouldn't want that data to be potentially accessible to anyone if there were a data breach at an AI company.Related I finally found a local coding LLM that I actually want to use Local AI coding assistants are actually useful now.Posts 5 By Nick Lewis Not every AI request is time-sensitive My local LLM is slow, but that's not always a problem I run my local LLM in Ollama on a mini PC with no dedicated GPU, and only 16 GB of RAM, and also occasionally run local models on my M2 MacBook Air.
Using an open-source tool, I found the best models that my hardware could support.These models are fairly small local models, and they're very slow to generate their responses.There are plenty of use cases where this isn't a problem.
For example, my morning briefing automation takes about 15 minutes to run from start to finish.It pulls the relevant information from calendars and other sources, such as local weather, and uses the local LLM to combine it into a written briefing.I then use a local text-to-speech (TTS) engine to convert the text into spoken audio.
The fact that the automation takes so long to complete isn't a problem, because it's set up to run automatically at 5 AM every day.It means that by the time we've got up and headed down to the kitchen for breakfast, the audio has already been generated, and it plays the instant we walk into the kitchen.With a local LLM, I'm in control I'm not at the mercy of AI companies Another reason why I much prefer using a small local LLM rather than relying on Claude or Gemini is that when I'm using a cloud-based service, I'm completely at the mercy of the AI provider.
If that company decides to nerf my favorite model or makes some questionable decisions, there's very little that I can do about it.Deals Save on Mini PCs and Local AI Hardware Deals Score discounts on compact PCs, RAM, SSDs, and peripherals ideal for local AI setups.Explore offers on mini PCs, upgrades, cooling, storage, and audio accessories to build a quiet, private LLM rig without overspending.
Deals Explore Computers & Work Setup Deals With a local LLM, I have control over the model I use, and I can change it up whenever I want.I don't have to worry about models being retired, because those models are saved on my own hardware, so they won't suddenly disappear.If the company behind the model suddenly proves to be secretly evil, I can easily change to a less problematic model.
Even a modest local LLM can be incredibly useful When I first tried running a local LLM on my mini PC, I was disappointed by the size of the models I was limited to and how slowly these models ran.However, I soon realized that even a small local AI model can do a lot, as long as time isn't a significant issue.My local LLM is now the backbone of the automations that I use the most each day.
Read More
