Open source: Run as a service

Open source

The agent service

How the hookci-agent systemd unit runs, how many jobs it runs at once, how jobs are isolated, where its state lives and what happens when you revoke it.

Once paired, an agent is an ordinary systemd service. It opens one outbound WebSocket to the hub, waits for triggers, runs each job in a sandbox and reports results back. Nothing connects in to the host.

One unit per installation

The installer ships a template unit, hookci-agent@.service. hookci connect starts one instance of it for the installation you paired, named by the numeric installation ID:

systemctl status hookci-agent@48213907

The ID is printed by hookci connect and stored in agent.json. A host can run several agents, one per paired installation, each with its own state directory and credential. The unit’s ExecStart is hookci agent --installation %i, which resolves to the state directory /var/lib/hookci/<installation-id>.

Setting Value Why
Restart always, after 5 seconds The agent comes back after a crash or reboot.
RestartPreventExitStatus 3 4 Exit 3 is a revoked agent and exit 4 is an agent too old for the hub or an internal error. Restarting cannot fix either.
Slice hookci.slice Agents and their jobs share one cgroup slice you can cap.
StateDirectory hookci/%i systemd creates /var/lib/hookci/<id> for the instance.
After, Wants network-online.target The agent starts once the network is up.

The pairing step, including --name for the label shown in Slack and --no-service to skip starting the unit, is described in Pair an agent.

Check that it is running

build-host
build-host recording
$ systemctl is-active hookci-agent@48213907
active
$ sudo hookci --state-dir /var/lib/hookci/48213907 status
ID                          REPO       REF   SHA      STATE    AGE    DURATION
01JD3W8K2Q7ZP4N6V9T5X0BHRC  acme/web    main  9f2c1ab  success  4m12s  27.3s
01JD3VZ0M4H8R2C7E1A6D3FQGS  acme/web    main  c04e7d2  failed   31m    41.0s
$ sudo hookci doctor
STATUS  CHECK            DETAIL
pass    hookci binary    v0.4.0
pass    agent service    hookci-agent@48213907.service active
pass    hub reachable    https://ci.webuno.io/healthz answered 200

Illustration. status, journal and doctor cover most questions.

The run IDs, repos and timings above are illustrative. hookci status and hookci logs default to a per-user state directory, so on a service host always pass --state-dir /var/lib/hookci/<id>, and use sudo because the state root is 0700 and owned by root. hookci doctor checks the binary, the unit, the state directory modes, the credential file, the hookci-run user, git, free disk space, clock skew, hub reachability, TLS and the WebSocket upgrade, and prints a fix next to each failure.

Logs

The agent’s own log goes to the journal:

journalctl -u hookci-agent@48213907 -f
journalctl -u hookci-agent@48213907 --since "1 hour ago"

Job output is stored per run on disk and uploaded to the hub for the Slack and GitHub links. To read it on the host:

sudo hookci --state-dir /var/lib/hookci/48213907 logs 01JD3W8K2Q7ZP4N6V9T5X0BHRC
sudo hookci --state-dir /var/lib/hookci/48213907 logs 01JD3W8K2Q7ZP4N6V9T5X0BHRC test
sudo hookci --state-dir /var/lib/hookci/48213907 logs -f 01JD3W8K2Q7ZP4N6V9T5X0BHRC

-f follows a running run until it ends. The files are at runs/<run>/logs/<job>.log, with the dependency install in setup.log.

Slots

A slot is one job running at a time. The default is half the CPUs, at least one, so a 4-core host runs two jobs in parallel. Extra jobs wait their turn. Change it with the global --slots flag. Because the shipped unit does not set it, override ExecStart in a drop-in:

sudo systemctl edit hookci-agent@48213907
[Service]
ExecStart=
ExecStart=/usr/local/bin/hookci --slots 4 agent --installation %i

Global flags go before the command. Save, then sudo systemctl restart hookci-agent@48213907. To cap what all jobs may use together, set limits on the slice with sudo systemctl edit hookci.slice, for example [Slice] and MemoryMax=16G.

Isolation modes

Each job runs through systemd-run as the hookci-run user in hookci.slice, with NoNewPrivileges, a read-only file system outside its run directory, no access to /home, a private /tmp and /dev, a hidden /proc of other processes, and CPU, memory and task limits taken from the job’s config. The details and the egress rules are in Isolation and egress.

Mode Behaviour
auto (default) Uses systemd isolation when the agent is root and systemd-run exists. It never falls back to none on its own.
systemd Same sandbox, explicitly required.
none Jobs run as the agent’s own user, with no sandbox and no limits. For local development and fully trusted repositories only.

If a host has no cgroup v2 or no systemd, auto stops with an error instead of running unsandboxed. See Troubleshooting.

State directory

Each agent keeps its files under /var/lib/hookci/<installation-id>/:

Path Contents
agent.json Hub URL, agent ID, credential, display name and installation ID. Mode 0600. The agent refuses to start if group or others can read it, and tells you to run chmod 600.
agent.db SQLite store of runs, jobs and events not yet delivered to the hub.
runs/<run>/ Checkout and logs of a run. Hidden from jobs.
cache/ Dependency caches written by setup steps.
mirrors/ Local git mirrors, so later fetches are incremental.

/var/lib/hookci/trusted_root.json, when present, is the sigstore root used by hookci upgrade. See Upgrade and uninstall.

If the agent crashes or the host reboots mid-run, the next start sweeps up leftover systemd units, network namespaces and Docker containers before taking new work.

Revocation

Revoke an agent from the Slack App Home: the hub drops its credential and tells the agent it was revoked. The process exits with code 3 and logs:

agent revoked: run hookci connect with a new pair token

Because exit code 3 is in RestartPreventExitStatus, systemd leaves the unit stopped. Nothing keeps retrying with a dead credential. To bring the host back, generate a new pair token and run sudo hookci connect <new-token>. The old state is reused for a new identity: connect rewrites agent.json and the store is bound to the new agent on the next start. Pair tokens are single use and expire after 15 minutes.

Exit code 4 means the hub rejected this agent version as too old, or an internal error occurred. Run sudo hookci upgrade, then sudo systemctl start hookci-agent@48213907.