Skip to content
Hermes Household AI

Hermes Household AI

How I built an AI agent that runs my household's logistics. It has its own identity, its own computer, its own tools and a short list of things it is never allowed to do.

I run a household of four and a dog, and the load is not the big things — it is the small ones. Appointment times. School notices. Forms with deadlines. Deliveries. The weekly shop. A prescription that needs chasing. None of it is hard. All of it is constant, and all of it slips.

So I built an agent that handles it. It runs on a small computer at home, has its own identity, and the household talks to it in a normal messaging thread. It reads my email, keeps the family calendar true, tracks what is in flight, and tells me what changed overnight.

This is the blueprint. Not the source code — the decisions, the order I did them in, and the mistakes. It is written for someone technically curious but not an engineer, and it is written to be followed, step by step, in ten steps.

Ten stepsFrom a blank computer to a working household agent
Roughly a weekendTo get it answering. Months to get it good
No cloud applicationIt runs on a machine you own
One non-negotiableIt never acts as you. It drafts, you send

Not a feature list. The actual work, and what it needs underneath.

The workHow it reaches meWhat it needs
Reads my email and turns the actionable parts into tasksA task appears, or a morning summaryIts own mailbox, read access
Handles school notices — dates onto the calendar, reminders createdA short daily digestIts own mailbox, calendar write
Keeps the family calendar true and flags clashesA message, before it becomes a problemCalendar read and write
Chases things in flight — deliveries, bookings, forms, faresA message when something changesIts own mailbox, scheduled jobs
Builds the weekly grocery list, and fills the online cart for me to confirmA list I review, a cart I check outA shared list, a browser
Watches recurring purchases and tells me when to reorderA message on the right dayPurchase history, a scheduled job
Runs the logistics of an upcoming tripItinerary, costs, tasks, remindersEmail, calendar, a task list
Prepares forms and replies for meA draft in my mailboxDraft permission only
Reports the day before I wake upOne summary at 5amEverything above

I dug into the order because the order is the part nobody writes down. You cannot set permissions before you have an identity. You cannot connect tools before you have a machine.

#StepPart
1Name it, and give it a voiceAn Identity
2Create accounts that belong to it, not to youAn Identity
3Set up the computer it lives onA Machine
4Connect calendars, reminders and contactsIts Tools
5Connect email and tasksIts Tools
6Set the permission layersPermissions
7Write the job descriptionThe Job
8Connect the channels the household usesConversation
9Give it memory — and a wiki you can readMemory
10Set up backups and watchdogsHonesty

The last ten percent takes eighty percent of the time. Getting an agent to answer a question is a weekend. Getting it to behave well around a real family — not too talkative, correct about the details that matter — takes months of tuning. Plan for that, because it is the actual project.

What it cannot do matters more than what it can. Mine can never send email as me, never spend money, never invent a fact about my household, and never reach outside its own lane. Those four rules are in The Rules It Cannot Break, and they are why I am comfortable leaving it running.