Start Building AI Agents Locally with Ollama, Pi, and VS Code
Getting Started with AI Engineering: Ollama, Pi, and VS Code
Good evening, everyone — gonna be trying out my new dictation tool here. Hopefully, it works out all right. I'm using Whisper.cpp, a port of OpenAI's Whisper model that lets me do speech-to-text locally. It records what I'm saying, transcribes it, and gives me a clean transcript. Pretty cool, right?
Today's topic is a follow-up on my Ollama post from the other day — why you should be using Ollama models in your workflow. We're going to continue by talking about AI agents and how to get started becoming an AI engineer, with a few tools you can use locally, for free or at low cost.
You don't have to be some super-genius. You can use AI to supplement what you don't know. There's a ton of information out there, and all three tools I'm sharing today have active communities where you can get help. And of course, you can always talk to me too — that's kind of what we're here for.
Tool #1 — Ollama (ollama.com)
Your local LLM runtime. Download it, run it, and you have AI models running directly on your machine — no API keys, no cloud required.
That said, Ollama does offer cloud access if you need a stronger model temporarily. Their plans run from $20 to $100/month. If you're just getting into it, the $20 plan wouldn't be the worst move. You'll probably get more use out of it than with something like Claude's plans — though I'll admit I'm a little biased. I've been moving away from Claude Code and shifting to tools that keep things on my own device, giving me more control and letting me build without burning through usage on prototypes I might drop after a week.
With Ollama, you get access to models like Kimi K2.5, GLM 5.1, and many other solid options.
Tool #2 — Pi (pi.dev)
Probably one of the best tools for getting started building with AI agents — especially if you want something like Claude Code but with more control.
Claude Code lets you add plugins and some customization, and you can still use Ollama with it. But what I like about Pi is that you can build your own extensions from the ground up. You can customize everything explicitly for what you're trying to do. I could turn this dictation tool into an extension that lets me talk to my agent and never type again. Or hook in a crypto wallet. The sky's the limit.
If you like how Claude Code handles something, you can recreate it in your own way and improve it. Especially now that Claude Code's source has been leaked — you can sample from that and build something even more tailored to your workflow.
Tool #3 — Visual Studio Code (code.visualstudio.com)
You need somewhere to see what you're building. VS Code lets you open your project folder, watch the agent make progress in real time, and stay oriented as things develop.
Getting Your Stack Ready
Once you have Ollama + Pi + VS Code, you're set up to start building. If you're brand new, watch a few intro videos on JavaScript or Python basics — having some programming knowledge helps, and I'd recommend starting with one of those two languages for AI projects.
The real skill you want to develop, though? Prompt engineering. This is where most people skip steps.
They'll describe everything they want, but they never say what they don't want. A good prompt — or better yet, a good spec document — goes a long way. You have to give the agent:
- Positive constraints — what you want it to do and how
- Negative constraints — what it should never do
If you don't set negative constraints, the agent will do whatever it thinks necessary to reach the goal. Tell it to "keep moving until you reach the top of the mountain" without guardrails, and if it falls, it might just wiggle in place forever — or go sideways and never finish. Worse, tell it to "clean up" while moving files between machines, and it might delete the originals while it copies them, because you were too vague.
The more time you spend thinking through what you want before you hand it to the agent, the better your results will be.
Your First Agent
When you're ready to build your first agent, start with Pi. Open VS Code, pop open the terminal, type pi, and you'll see the TUI interface come up. From there, try something like:
/plan Let's create a Pi extension using the Godot documentation.
Pass in the Godot docs, and if you can find the Pi extension docs, pass those too. The agent can usually find itself since it knows it's running inside Pi. Go back and forth, refine the plan, and build out a spec document that maps the entire project — its flow, its features, what it should and shouldn't do. When you go to implement, you'll know exactly what you're building and why.
This post is also a live test of my dictation app — I wanted to see how well Whisper.cpp handles a long unscripted riff like this. Let me know what you think, and stay tuned for deeper dives on spec file creation and more AI engineering workflows.