Guide for building an autonomous agentic system administrator

Part 2: Commissioning the agent

This is part 2 of my series on building a sysadmin agent. Part 1 covered giving the agent a job: the brief, its identity and its mandate. This part covers commissioning: getting the SAA (sysadmin agent) from a brief it had answered with questions to a job that runs every morning and a report I actually received. The files are on GitHub: https://github.com/Sjw22263/sysadmin-agent-kit/tree/main/02-commissioning-the-agent

Commissioning is finished when six things are true

The scope is agreed in writing. The agent's access has been proven, not assumed. It has inventoried what it is responsible for. It has made one real repair and verified it. I have received a report from it. And the job is scheduled, with the schedule read back.

We did this in a single working session. Several things did not go as planned, which is why the checklist is mostly gotchas.

Give the agent its own accounts

The SAA runs on its own server under a Unix account named after the harness. Every account I create for it elsewhere is named for the agent. I told it at the time: "I use Maxi as your identity rather than hermes, so accounts I create for you will be maxi@ or maxi-[function] etc." Its reply was that Hermes "is the runtime, framework, and sometimes the Unix account this process happens to run under ... It is not my identity."

There is a practical side to this as well. With its own account, the remote server's logs show which actions were the agent's, and its access can be removed without touching anyone else's.

Then prove the access rather than assume it. Use a dedicated key, check host keys before trusting them, and test the login non-interactively with sudo -n true, so that automation fails immediately instead of hanging on a password prompt. Two traps cost us false alarms later: running ssh host without a username, which quietly uses the local one and produces a "permission denied" that looks like a lost key; and a firewall that rate-limits SSH, which blocked our health script partway through its checks so that a refused connection looked like a stopped service. The fix for the second is one shared connection per run, not a weaker firewall.

Let it look before it fixes

Ask the agent to inventory everything it is responsible for before it changes anything. Expect it to find things that are already broken.

Ours found that the file-sharing service on the web server had failed at boot. It was set to listen on one network address only, and started before that address existed. The SAA diagnosed it and stopped. I told it to fix it, and it made the service wait for the network and restart on failure, then limited its firewall rules to the local network. That first repair was done under an explicit instruction. Restarting that service later became one of the trivial repairs the daily job may do by itself.

Discovery also explained one of the questions from part 1: why mail looked broken from the SAA's server while working perfectly for everyone else. Address translation meant the mail ports looked closed from inside the network. The checks moved onto the mail server itself, and the daily report email became the real test that outbound mail works.

Teach it who it is talking to

An agent that reads email will eventually be asked to act on one. Early on, the SAA accepted an email as coming from me because the message said it was from me. It was, but I asked: "Are you sure that is me? I mean it is, but how do you know?"

The SAA's answer was: "I treated the display name as identity. That's exactly how phishing works."

The rule now is that a message counts as mine only if the receiving server's authentication results show DKIM, SPF and DMARC all passing, and the address is on a short approved list. A message that fails is kept, not deleted, not acted on, and reported with the fields that failed. Authentication establishes who sent it, not what they are entitled to ask for. The first parser we wrote returned values like passheader instead of pass, so test yours against real messages, passing and failing. And test the whole rule yourself, more than once. I still do.

Schedule it, then read the schedule back

The first scheduled version of the daily health check was a script, with no model involved. The script checked everything in the brief, restarted only named services, sent the report and raised an urgent message only if something serious was left. It cost nothing to run and did the same thing every morning. Hermes runs this kind of job with --no-agent: the script is the whole job, and if it prints nothing, nothing is sent.

Scheduling it produced its own lessons. The first attempt failed because Hermes only runs cron scripts from ~/.hermes/scripts/. An earlier mail-checking job had been created to run once instead of repeatedly, and the corrected version still said once. Cron times are in the server's local time, so 6 am is 0 6 * * *, not a UTC conversion. Read back every job you create: schedule, repeat and next run.

One warning belongs here rather than in part 3, because this is where you would make the mistake. If you later attach the same script to an ordinary agent job, Hermes treats empty output the opposite way: it wakes the agent. Only a final line that is a JSON object containing "wakeAgent": false, such as {"wakeAgent": false}, stops it. That convention is what part 3 is built on, but it will cost you a model run every morning if you switch modes without knowing it.

commissioning-checklist.md has the full list. email-trust-rule.md has the mail rule and the header check. morning-health-check-job.md and the template script morning-health-check.sh cover the first scheduled job and its report.

Practical advice

1. Create the agent's accounts under its own name, and prove each one works with a non-interactive login and sudo -n true before any job depends on it.
2. Have the agent inventory your systems read-only first, and give the first repair as an explicit instruction.
3. Before the agent acts on any email, require DKIM, SPF and DMARC to pass and the sender to be on an approved list. Then test it with a fake.
4. Start the daily check as a script-only job that sends its report every morning, so the report itself proves the job ran.
5. Read back every scheduled job: location, schedule, repeat and next run time.

Next: watching the estate, and waking the agent only when there is something wrong.