Guide for building an autonomous agentic system administrator

Part 1: Giving the agent a job

I am writing a series of posts on how I set up an AI agent to look after my servers. It is not a history of what we did. Each post covers one part of the setup, with the files we actually use, generalised so you can adapt them for your own systems, and the mistakes we made, placed at the point where you would make them. The files are on GitHub: https://github.com/Sjw22263/sysadmin-agent-kit

The specific gotchas, settings and commands in this series are for Hermes. The principles can be adapted to other agent harnesses: define the job, separate identity from authority, decide who may change operating procedures, and verify what the agent actually does.

What I wanted

Maxi, my sysadmin agent,¹ looks after two servers, the websites and mail they carry, and the other agents that run on them. I did not want a chatbot with a shell. I wanted what I have always wanted from a good system administrator: someone who keeps the systems running, tells me what is broken, fixes what they can, and is given more responsibility as they show they can handle it.

When the time came to write down the job of the SAA (sysadmin agent), I used "the same tasks I have used in job goals for system administrators I have employed in the past." I treat my agents as colleagues because I think it produces better results, and a sysadmin is a job I already knew how to describe.

Before you start: decide how the agent may change itself

This comes first because it is the mistake to avoid before the agent does any real work.

Hermes is built to improve itself, and its default settings let it. In normal conversation it is told to patch a skill when it thinks the skill is wrong. After a conversation, a background review can create and rewrite skills and save memories without asking. Roughly once a week, a curator archives skills created by that review that have not been used for a month. For a personal assistant that may be exactly what you want. For an agent with administrator access, its skills are its operating procedures, and you need to know when they change.

We found out the hard way when an automatic review in another of my agents rewrote its own skills, and its reading of what it was allowed to publish, without the review process we had agreed. The SAA's summary was: "The system took an unresolved conversation about unauthorised policy writing and responded by writing another policy."

The fix for skills is one line: skills.write_approval: true. Changes made through the skill tool are then staged for you to review. That is a gate on the tool, not a lock on the files, so an agent with a shell can still edit a skill directly. We left memory open, because, as I put it to the SAA at the time, "you need to be in control of your own short term durable memories. Skills need review, memories are yours."

Two more traps are specific to a sysadmin. The curator's archiving does not go through that approval setting, and recovery procedures are used rarely by design, so a restore procedure written by the background review can be archived before the day you need it. Check what the curator manages and pin anything you rely on. And the agent can edit its own SOUL.md without asking, because Hermes exempts the agent's own home directory from its protection for instruction files. The file hermes-self-modification.md has the settings, the commands and a checklist.

Three documents, three jobs

The SAA works from three documents, and keeping them separate turned out to matter.

The brief says what to do now and how to report it. The identity file (SOUL.md) says who the agent is and how it works. The mandate says what it is responsible for, what it may decide on its own, and what it must bring to me first.

The short version is that identity is not authority. An identity file that also tries to be a permission list does both jobs badly. They are different functions and need to be separated, if for no other reason than better maintainability.

The first brief

The first real job I gave the SAA was a daily health check: look at the websites, the servers and the mail every morning, fix anything trivial, tell me about anything serious, and send me a report. I wrote it the way I would brief a new human sysadmin, and ended with: "Before you set this up, please evaluate the task and the resources you will need. Ask if you want any clarification or can see anything I might have missed."

It came back with nine findings and seven questions before building anything. It could not yet log in to one of the servers. It could not tell from outside where mail was handled. It did not have the permissions it needed on its own server, and had stopped rather than force the issue. Its recommendation was not to create the job yet, "otherwise I'd be building a daily health check with a blind spot exactly where you care about it."

It also proposed, in writing, what "trivial" should mean: restart named services, yes; edit configuration, upgrade packages or reboot servers, no. After the first report arrived, I tightened one thing: "I only want the report to say what is broken."

brief-template.md has the brief, the questions to expect, the repair boundary and the report rule.

The mandate

The SAA did not have a mandate at the start. The need for one grew out of necessity as its responsibilities grew. I will cover the mandate in full in a later post, but in hindsight it is better introduced earlier than later, so I am including it now. You and your own agent should customise it together for your own estate.

Our first version was a permission list. The current one delegates outcomes instead: the systems are reliable, secure and recoverable; the architecture is as simple as it can be; the records are good enough to recover from. Anything that genuinely serves those outcomes is allowed unless it is on a short list of reserved matters, such as changing its own governance, spending money, exposing anything to the internet, or destroying data that cannot be recovered. My intent, as I put it at the time, was "pretty broad permission to do all the things a competent sysadmin would do." We ran the first version for a week before making it permanent.

mandate-template.md is the current version, generalised. soul-template.md is the structure of the SAA's identity file, with the gotchas.

Practical advice

1. Before the agent does real work, set skills.write_approval: true, and decide deliberately about memory, background review and the curator.
2. Check which skills the curator manages, and pin any recovery procedure among them so it is not archived for being unused.
3. Keep identity and authority in separate documents, and point from one to the other.
4. End every brief by asking the agent to evaluate it and ask questions before it builds anything.
5. Write a short mandate early. Delegate outcomes, and keep the list of reserved matters short and serious.

Next: commissioning the agent, from first login to a report I actually received.

¹ Maxi's full name is Maxine Verity Calder, which the agent chose for itself using an agent naming convention skill. In its own words: "Maxine is the human working identity I keep behind Maxi. Verity encodes my truth-seeking. Calder encodes my habit of forging ideas under heat." This was an experiment in anchoring agents to their role and mitigating alignment drift. There is no definitive evidence this works, other than agents always voting to keep their identity when asked, and the user (me) having a better user experience.