I built my own AI Jarvis on a regular laptop

I always wanted my own Jarvis. The problem is I don't have Tony Stark's millions.

What I do have is a pretty basic laptop and a considerable amount of stubbornness (very considerable). So I told myself: how hard can it be?
(spoiler: pretty hard).

I named him Kyle. I talk to him and he answers out loud, he has a team of agents with different personalities, I control him with hand gestures in front of the camera, he reads me my calendar and scolds me when I've gone six hours without getting up from my chair. He helps me practice my English and French... Oui Oui!

But I'm not going to sell you the pretty version. Here's what really happened:

An entire all-nighter fighting an error that kept killing the connection in an infinite loop. Turns out Google had changed a single word in their system. One. Word.

The agents' voices kept switching on their own. One of them (Loki, obviously) stretched out his vowels like he was possessed: "looookiii".

Since I use a second monitor with audio, Kyle could hear himself and would answer himself. I had to teach him to ignore his own voice.

I wanted to wake him up by saying "Hey Kyle" and he just... never woke up. I retired that feature.

I programmed the gesture control, updated the library (MediaPipe) and it broke everything. Back to rewriting it from scratch.

And my favorite: at one point there were TWO Kyles running at the same time, fighting over the microphone. Absolute chaos.

That said: I gave him layers of security, he doesn't store my sensitive data in any log he has a persistent memory base and he doesn't keep session logs because I only save what matters in a Google Sheet that one of the agents organizes for me, and there's an agent dedicated to monitoring the system. Healthy paranoia.

The full stack, in case anyone wants to give it a shot:
Python · Claude Code (my partner: I direct, he codes) · Google's Gemini API (free tier, the brain and the voice) ·
Google Workspace (Calendar + Sheets as his memory) · Telegram (to control him from my phone) · MediaPipe (the
gestures) ·

How long did it take? About 8 hours over the weekend and a final 20-hour marathon straight. Do I regret it? Not at all. It's
been months since I had this much fun.

The best part: I spent almost $0. Just the subscriptions I already have. And I still want to add more stuff... if only I had
Stark's budget + a couple of free weeks.

What ridiculous idea have you been sitting on that you're dying to build?