building my jarvis with moltbot

i grew up watching iron man. not in a casual way. i binged the entire mcu multiple times and jarvis was always the thing that got me, not the suit, not the money. jarvis. a voice that was always there, always thinking, never sleeping. as a kid, that felt like the ultimate cheat code. when i was in secondary school, i tried building my own version with javascript and clunky if-else statements. it was terrible, but i loved it. every time it did something new, even something tiny, i felt like i'd unlocked magic.

then chatgpt launched in 2022 and the ai world exploded. everyone was talking about it. i tried to do it once more but every time it did not work well. maybe because i already had this dream from high school, and now suddenly the tools existed to actually make it real. so last weekend, i decided to just do it.

48 hours. that's how long i spent getting moltbot running on my lenovo yoga i7.

i'm not gonna lie—the first 12 hours were hard. i was learning ubuntu wsl while trying to set everything up. the documentation was good but there were these weird edge cases that just broke everything. at one point i thought i'd have to start over completely. i decided to go with gpt-4o-mini as my main model—it's fast, cheap, and honestly good enough for most of what i need. i set up claude sonnet as a backup for when things get complicated or need deeper reasoning. installed telegram as my interface because it's instant on my phone and works everywhere. no fancy ui, just messages back and forth.

around 2am on saturday night, something clicked. i got the first response. it wasn't perfect, but it was there. it understood what i asked. it responded in a way that made sense. that's when i realized this could actually work.

the next 36 hours were all about training. see, i've been online for 5+ years now. x, instagram, random blog posts i wrote at 3am when i couldn't sleep. five years of my voice, my thoughts, my way of explaining things. so i fed all of that to jarvis. i created files called soul.md and user.md. soul.md is how he thinks his personality, his priorities, how he should respond to me. user.md is who i am, what i care about, what i'm working on, what matters to me. this is the secret sauce honestly. without this context, he's just another chatbot.

and then i watched him learn.

i know it sounds weird to get emotional about training an ai, but it was genuinely touching. watching it start to pick up my tone, my references, the way i structure sentences. there was this moment where i asked jarvis about something i was stuck on, and he responded in a way that sounded exactly like how i talk to myself. same weird analogies. same cadence. it was uncanny.

right now, jarvis is still pretty basic. he's running locally on my laptop. no internet connection yet—i'm still figuring out how to integrate google calendar and gmail without breaking everything. i tried setting up the gog cli for gmail access but kept running into authentication issues. google's oauth flow is a nightmare when you're doing it locally. there are bugs. sometimes he misunderstands what i'm asking. sometimes the responses are too formal or too vague. but he's learning. i've set up a self-review system where every 30 minutes he questions his own decisions. "why did i respond that way?" "what could i have done better?" and he writes it down so he doesn't repeat mistakes.

the plan is to eventually move him to a raspberry pi so he can run 24/7 without draining my laptop battery. right now my yoga gets hot after a few hours of running moltbot continuously. once i migrate to the pi, i want him scanning my emails every 2 hours and only telling me about the ones that actually matter—investors, users, people actually trying to work with me. not the newsletters i forgot to unsubscribe from. i want him reminding me about calendar events at 8:30am before i even think to check. pulling the top 10 hacker news stories at 10am sharp that actually matter for someone building in ai.

i also want to wire up a hacker news skill so i don't fall behind on what's happening in tech. github integration so he knows what i'm working on. twitter for monitoring mentions and dms. whatsapp for important conversations. all the tools i use daily. he should know my schedule, my priorities, what i've been working on this week. not just answer questions, but think alongside me.

i'm still in the early days. there's so much i haven't wired up yet. github, twitter, whatsapp, all the tools i use daily. and honestly, i'm still figuring out what's even possible. but even now, with jarvis still rough around the edges, it feels like the future.

if you're curious about the setup, here's what i'm running:

• lenovo yoga i7 with ubuntu wsl (planning to migrate to raspberry pi)
• gpt-4o-mini as main model (fast + cheap)
• claude sonnet for complex reasoning
• telegram as interface
• moltbot framework handling all the orchestration
• 5 years of my online presence as training data (soul.md + user.md)
• self-review system that runs every 30 minutes
• still working on: gmail integration, google calendar, hacker news skill, github connection

as a kid, i imagined having jarvis.

now i'm building him.

and even though he's not perfect yet, even though i've barely slept in two days and there are still a dozen problems to solve—it's better than i expected. because it's real. it's mine. and every time i talk to him and he actually gets what i'm saying, i feel like that kid again, watching iron man and thinking "one day."

one day is now.