EPIC CASTLE

Back in medieval England, an eyre was a travelling court. Royal justices would ride out to a county, set up, and go through everything; crimes, taxes, who owned what, who owed what. Before they could rule on anything they had to know the full state of the place. So the first job was always the same. Count it all up.

Most config tools start the same way. Before they touch anything, they probe the system to check what's already there. This is fact gathering. The tool looks at the machine, builds a picture of its current state, then decides what to do next.

Puppet has Facter, Chef has Ohai, Ansible has its setup module. They all have the same job. To profile the machine (OS, memory, network, filesystem) and hand back the results as data you can use. You can't manage a system well if you don't know what it looks like right now.

As part of cleaning up and modernising Spire, I'm putting out a new small library: Eyre. It gathers system facts through a shell. Spire will end up using it for it's facts.

You give Eyre a function that runs a shell script and hands back the result as {:exit exit-code :out stdout :err stderr}. That's it. Because you supply the executor, Eyre itself has zero dependencies. Whatever it needs is injected.

Put the following in a file gather.clj:

(ns gather
  (:require [babashka.process :as process]
            [clojure.pprint :as pprint]
            [eyre.core :as eyre]))

(defn make-exec [shell]
  (fn [script]
    (process/shell {:in script
                    :out :string
                    :err :string}
                   shell)))

(pprint/pprint
  (eyre/gather (make-exec "bash")))

then run it with babashka:

$ bb -Sdeps '{:deps {io.epiccastle/eyre {:mvn/version "0.1.1"}}}' gather.clj
{:shell
 {:type :bash,
  :version "5.3.15(1)-release",
  :shell "/bin/bash",
  :canonical-path "/usr/bin/bash"},
 :os
 {:family :linux,
  :kernel
  ...

You will see it dump all the facts it could find running as your user on a local bash shell.

What keys do we have?

(keys (eyre/gather (make-exec "bash"))
;;=> (:shell :os :hardware :users :filesystem :network :paths)

Lets just pull out the :shell portion of the response:

(:shell (eyre/gather (make-exec "bash"))
;;=>
{:type :bash,
 :version "5.3.15(1)-release",
 :shell "/usr/bin/bash",
 :login-shell "/bin/bash",
 :canonical-path "/usr/bin/bash"}

I can try launching it through other shells by changing "bash" to "zsh", "fish" or another shell and it continues to work.

(:shell (eyre/gather (make-exec "zsh"))
;;=>
{:type :zsh,
 :version "5.9.2",
 :shell "/usr/bin/zsh",
 :login-shell "/bin/bash",
 :canonical-path "/usr/bin/bash"}

(:shell (eyre/gather (make-exec "fish"))
;;=>
{:type :fish,
 :version "4.8.1",
 :shell "/usr/bin/fish",
 :login-shell "/bin/bash",
 :canonical-path "/usr/bin/bash"}

Here you can see the :login-shell continues to show the parent shell, while :shell shows the path of the shell process that you are running inside.

All decision on what to run in the executor is based on the :type of the shell. Eyre supports bash, zsh, sh, dash, ksh, busybox, fish, nushell, PowerShell and even cmd.exe. It can probe Linux, FreeBSD, NetBSD, macOS and Windows hosts.

Over SSH

Since Eyre just needs a function that runs a command and returns {:exit :err :out}, you're not stuck running it locally. Plug in an executor that runs over SSH, and now you're gathering facts from a remote machine instead.

Here's what that looks like using clojuressh:

(ns gatherssh
  (:require [clojure.pprint :as pprint]
            [clojuressh.core :as ssh]
            [clojuressh.session :as session]
            [eyre.core :as eyre]))

(let [session (ssh/ssh "remotehost.com" {:username "remote-username"})
      exec (fn [script]
             @(ssh/exec session script {:out :string :err :string}))
      facts (eyre/gather exec)]
  (session/disconnect session)
  (pprint/pprint (:shell facts)))
;; =>
{:type :bash,
 :version "4.3.48(1)-release",
 :shell "/bin/bash",
 :login-shell "/bin/bash",
 :canonical-path "/bin/bash"}

Run:

bb -Sdeps '{:deps {io.epiccastle/eyre {:mvn/version "0.1.1"} io.epiccastle/clojuressh {:mvn/version "1.0.0"}}}' gatherssh.clj

AI in Development

LLM assisted coding provided two great benefits during development. The first was script translation. They are very competent at translating software from one language to another and certainly I do not know the idiosyncrasies of every shell.

The second was help setting up a significant test platform. Helping to write Packer scripts to build VMs, or Docker scripts to build containers, there was a lot of work here. Without AI doing a lot of that drudgery the library would not be tested across so many operating systems and shells.

Improvements

Running over the network brings a problem you don't get locally: latency. Every network shell call has a delay, and if you split fact gathering into lots of small calls, those delays stack up.

Right now, some of the probe scripts are joined together and run as one, so a slow connection doesn't pay round trip cost over and over. But there's more to do. More scripts could be merged the same way. And beyond that, the gathering itself could be smarter. It could pull only the data you actually need instead of everything. These improvements will be left for later versions.

You can find the code here and the output documentation here.

I hope you find some uses for this tool.