Local AI. What about Local AI?

I bet most people still open Google when they want to know how to reverse a string in Rust. I don’t. I don’t write Rust, but if I were I would do it by asking a locally hosted LLM.

Bye Websearch: Running a local LLM

Not because it’s cool or futuristic, but because it’s actually better for the things I used to rely on search engines for:

• Defining a word
• Copy/pasting professional sounding emails
• Asking help on how to use neovim when I forget a command
• Quick code snippets like this repo here for my groq-cli assistant for zsh

If I need deeper context or fact-checking, sure, I’ll move to bigger online models or read HackerNews/reddit. But for 90% of daily “I just need an answer now” queries? My Local LLM wins.

Search engines are broken on purpose

What happens to searching the web?
The typical oldschool journey into searching begins with the typing something that hopefully makes sense.
After submitting this happens:

• Walls of ads at the very top, literally can be anything these days
• Clickbait pages stuffed with SEO keywords appear at the top
• Long, content-free text written by a bot for the sole purpose of keeping you scrolling.

Google hates that I have my local LLM.
Why? Because if you find your answer, I experience no ads. No revenue.

It’s the same reason why dating sites don’t want you to find ur match.

Local LLMs are amazing at this, like early magic from doing a Google search before the Google Ads in year 2000.
They just give you the answer. Direct. No fluff. Speed is improving.

One last big red flag: Sometimes they hallucinate, but things are getting better.

Will they put ads in LLMs?

Right now, big cloud-hosted LLMs don’t shove ads in your face. But they will. Eventually locally hosted ones could too.

The moment we get used to trusting them, they’ll pivot. You’ll ask for Dell specs and suddenly you’re pitched an HP. Not if. bur rather when.

Currently Local LLMs aren’t riddled with ads: they’re just weights and compute. They’re not as reliable as the large hosted ones, but they can get pretty far even when you’re on a flight with no connectivity.

I build with LLMs, it’s fun again

Early days still for local LLMs but I can build, automate, and write things without having to ask anyone or paying. No API limits. No billing surprises. Just me and my little AI friend.

I was a sold from day 1: Local, portable LLMs that are specialized and can be dynamically loaded/tuned are the future.

Hosted models are becoming cheaper, more accurate, more energy efficient. And open source.
We then quantize them and bring them to your local environment.

And if you ignore it this you’ll fall behind the people who didn’t.
Forward can’t be stopped.

The next frontier

Right now we’re slapping AI in laptops, desktop and server clusters, but what’s coming looks even more interesting: serious models running in your pocket.
Before you blink it will be in your home: picture a box in your living room that hosts your AI. Like a cluster of RPis, all your devices connect to it and act as thin clients, maybe running a lightweight model locally to smooth out latency while the heavy lifting happens in your personal server.

It’s not even far-fetched — Apple or Amazon could drop a large box with WiFi called “AIHome: LLM Edition” and charge $10k–20k. People would buy it. Because the AI wouldn’t live in someone else’s datacenter anymore. It would live at home.