Skip to main content

How the agent works

The atherops-otel-agent is a thin wrapper around the unmodified upstream OpenTelemetry Collector (otelcol-contrib). The wrapper is the only AtherOps-authored process on your host; the collector runs as a supervised child process.

The wrapper supervises the collector​

The wrapper does five jobs:

  1. Renders the collector config. It turns your simple config.yaml into full OpenTelemetry Collector YAML. You never write collector YAML by hand.
  2. Supervises the collector. It starts the collector, polls its health_check extension on :13133, and restarts it automatically if it exits or becomes unhealthy.
  3. Signs and forwards telemetry. The collector sends plaintext OTLP to a loopback auth-proxy on :9599. The proxy attaches HMAC credentials and forwards to AtherOps. The raw API key never crosses the network.
  4. Manages identity. It registers with an install token, persists the returned key, and sends a periodic heartbeat.
  5. Converges on remote config. It polls the platform for config and a target collector version, and re-renders + restarts to match.

Why supervise, not just systemctl?​

systemd restarts the agent if the agent dies (Restart=always). The agent restarts the collector if the collector dies, and it adds crash-loop protection: after 5 restarts in 60 seconds it pauses for 5 minutes instead of hammering a broken config. The convergence model is render-then-restart: any config or version change re-renders the collector YAML and restarts the collector. There is no live reload (no SIGHUP); a restart is the single, predictable mechanism.

Why a loopback auth-proxy?​

The upstream collector has no notion of AtherOps authentication, and we ship it unmodified. So the collector exports plaintext OTLP to 127.0.0.1:9599 only, never off the host. The agent's proxy is the trust boundary: it signs each request with HMAC and forwards it. This keeps the API key off the wire and lets us use the stock collector with no custom build.

See also​