Privacy Tool Spotlight: Ollama
Can you believe some people have the audacity to say my head-shot is AI-generated?! In all seriousness, I've found AI to be a wonderful addition to my toolbox over the past few years. It has come a long way since its early days, and the rate of improvement is startling, to say the least. The deep-dive training from my previous employer has really helped me leverage its capabilities as we both continue to grow and adapt.
But, at its core, AI is a tool. Like most systems, it only understands what others have taught it, from the data it references to the security protocols and boundaries it follows. We've reached a point where the tool's evolution and widespread use have outpaced our grasp of the security opportunities it offers.
I rely on a mix of online AI agents for digging into current information research and, of course, generating images. Alongside that, I self-host an Ollama AI server in my home lab, integrated into my Home Assistant Smart Home.
Self-hosting an AI server in a Smart Home setup comes with several advantages. The biggest perk is that I'm not feeding a constant stream of data to Big Tech. Most "smart" speakers need to listen to everything, waiting for a wake word. With Home Assistant, all of that processing stays in-house. Plus, I'm not handing over details about what I'm curious about, what I need to research, what my schedule looks like, or every conversation it might pick up. I'm also not giving a third party a pile of logs that could end up anywhere or unintentionally sharing my info with the world.
The AI server I put together is compact yet powerful. It might not run the latest large (massive, really) models, but it didn't cost a fortune either. Here's the setup:
- Beelink SER8 Mini PC with 64GB of memory, 16GB dedicated to the GPU
- Canonical Ubuntu Linux (24.04 LTS)
- Docker, Inc Docker
- Ollama running in a Docker container
- llama3.2 model
This configuration works seamlessly with Home Assistant and connects to the Open Web UI, which I host on my desktop in another Docker container. It gives me a reliable thinking partner that never sends data beyond my local network. The system is small, quiet, energy-efficient, and sits neatly on my desk, always ready to assist.
So, if you're thinking about self-hosting your own AI, it's absolutely doable. It's a practical step toward keeping privacy and security in your control.
Remember, we may not have anything to hide but everything to protect.
Learn more about Ollama at https://ollama.com/
