line by line output - as commands progress - is this possible in consfigurator?
Andreas Reuleaux
rx at a-rx.info
Mon Aug 1 11:53:27 BST 2022
Hi,
I am playing with consfigurator again (on a fairly beginner level still,
and learning some common lisp along the way - so bear with me).
And I wonder: is it possible to see the line by line output of commands,
as they progress (not only at the end of their execution)? - I assume it
is possible with some programming / patching the ssh connection type in
the consfigurator source? - But maybe it is possible already by means of
the current state of the code?
To illustrate my point: Say I have a very simple list of properties (and
a host definition):
--8<---------------cut here---------------start------------->8---
(defproplist myapt :posix ()
(apt:updated)
(apt:upgraded)
(apt:autoremoved)
)
(defhost localhost ()
(os:debian-testing :amd64)
)
--8<---------------cut here---------------end--------------->8---
Now when I run these (apply them to my localhost):
--8<---------------cut here---------------start------------->8---
(deploy-these ((:ssh :user "root")) localhost (myapt))
--8<---------------cut here---------------end--------------->8---
I get to see the result / summary at the very end:
--8<---------------cut here---------------start------------->8---
;; localhost :: dpkg --configure --pending ... ok
;; localhost :: apt-get update ... done
;; localhost :: apt upgraded ... ok
;; localhost :: apt removed automatically installed packages ... ok
;; localhost :: done
;; Changes were made without any reported failures.
NIL
*
--8<---------------cut here---------------end--------------->8---
So far, so good - this works.
Really though, I would like to see the line by line progress, as these
commands proceed - the same example in Haskell-Turtle (here the output -
haskell code shown below):
--8<---------------cut here---------------start------------->8---
...
/usr/bin/ssh -l root localhost apt-get update ...
Hit:1 http://ftp.uk.debian.org/debian bookworm InRelease
Reading package lists...
/usr/bin/ssh -l root localhost DEBIAN_FRONTEND=noninteractive apt-get -y upgrade ...
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
/usr/bin/ssh -l root localhost DEBIAN_FRONTEND=noninteractive apt-get -y autoremove ...
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
--8<---------------cut here---------------end--------------->8---
I.e. I get to see all the lines: Reading package lists..., Building
dependency tree..., ... - Of course this is more interesting/relevant
when some packages are actually installed (and this may take some time):
some minutes ago I got to see (line by line - progressing):
--8<---------------cut here---------------start------------->8---
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following packages will be upgraded:
cl-interpol dmsetup kmod libdevmapper1.02.1 libkmod2 libpam-systemd libpci3
libpq5 libsystemd-shared libsystemd0 libudev1 parallel pciutils systemd
systemd-sysv udev xserver-common xserver-xorg-core xserver-xorg-legacy
19 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 18.7 MB of archives.
After this operation, 39.9 kB disk space will be freed.
Get:1 http://ftp.uk.debian.org/debian bookworm/main amd64 systemd-sysv amd64 251.3-1 [123 kB]
Get:2 http://ftp.uk.debian.org/debian bookworm/main amd64 libpam-systemd amd64 251.3-1 [304 kB]
Get:3 http://ftp.uk.debian.org/debian bookworm/main amd64 xserver-common all 2:21.1.4-1 [2,420 kB]
Get:4 http://ftp.uk.debian.org/debian bookworm/main amd64 xserver-xorg-legacy amd64 2:21.1.4-1 [2,425 kB]
Get:5 http://ftp.uk.debian.org/debian bookworm/main amd64 kmod amd64 30+20220630-3 [95.2 kB]
Get:6 http://ftp.uk.debian.org/debian bookworm/main amd64 libkmod2 amd64 30+20220630-3 [59.6 kB]
Get:7 http://ftp.uk.debian.org/debian bookworm/main amd64 libsystemd-shared amd64 251.3-1 [1,689 kB]
Get:8 http://ftp.uk.debian.org/debian bookworm/main amd64 libsystemd0 amd64 251.3-1 [410 kB]
Get:9 http://ftp.uk.debian.org/debian bookworm/main amd64 xserver-xorg-core amd64 2:21.1.4-1 [3,758 kB]
Get:10 http://ftp.uk.debian.org/debian bookworm/main amd64 systemd amd64 251.3-1 [3,122 kB]
Get:11 http://ftp.uk.debian.org/debian bookworm/main amd64 udev amd64 251.3-1 [1,656 kB]
Get:12 http://ftp.uk.debian.org/debian bookworm/main amd64 libudev1 amd64 251.3-1 [183 kB]
Get:13 http://ftp.uk.debian.org/debian bookworm/main amd64 pciutils amd64 1:3.8.0-1 [101 kB]
Get:14 http://ftp.uk.debian.org/debian bookworm/main amd64 libpci3 amd64 1:3.8.0-1 [64.6 kB]
Get:15 http://ftp.uk.debian.org/debian bookworm/main amd64 cl-interpol all 20220725.gitd4f49d4-1 [32.8 kB]
Get:16 http://ftp.uk.debian.org/debian bookworm/main amd64 libdevmapper1.02.1 amd64 2:1.02.185-1 [143 kB]
Get:17 http://ftp.uk.debian.org/debian bookworm/main amd64 dmsetup amd64 2:1.02.185-1 [92.7 kB]
Get:18 http://ftp.uk.debian.org/debian bookworm/main amd64 libpq5 amd64 14.4-1+b1 [173 kB]
Get:19 http://ftp.uk.debian.org/debian bookworm/main amd64 parallel all 20220722+ds-1 [1,849 kB]
...
Preconfiguring packages ...
setting xserver-xorg-legacy/xwrapper/allowed_users from configuration file
Fetched 18.7 MB in 4s (4,181 kB/s)
(Reading database ... 243771 files and directories currently installed.)
Preparing to unpack .../0-systemd-sysv_251.3-1_amd64.deb ...
Unpacking systemd-sysv (251.3-1) over (251.2-7) ...
Preparing to unpack .../1-libpam-systemd_251.3-1_amd64.deb ...
Unpacking libpam-systemd:amd64 (251.3-1) over (251.2-7) ...
Preparing to unpack .../2-xserver-common_2%3a21.1.4-1_all.deb ...
Unpacking xserver-common (2:21.1.4-1) over (2:21.1.3-2) ...
Preparing to unpack .../3-xserver-xorg-legacy_2%3a21.1.4-1_amd64.deb ...
Unpacking xserver-xorg-legacy (2:21.1.4-1) over (2:21.1.3-2+b1) ...
...
Preparing to unpack .../6-parallel_20220722+ds-1_all.deb ...
Unpacking parallel (20220722+ds-1) over (20210822+ds-2) ...
Setting up cl-interpol (20220725.gitd4f49d4-1) ...
Setting up libpq5:amd64 (14.4-1+b1) ...
Setting up parallel (20220722+ds-1) ...
Setting up libpci3:amd64 (1:3.8.0-1) ...
Setting up xserver-common (2:21.1.4-1) ...
Setting up xserver-xorg-legacy (2:21.1.4-1) ...
setting xserver-xorg-legacy/xwrapper/allowed_users from configuration file
Setting up libkmod2:amd64 (30+20220630-3) ...
Setting up kmod (30+20220630-3) ...
Installing new version of config file /etc/init.d/kmod ...
Setting up libsystemd-shared:amd64 (251.3-1) ...
Setting up pciutils (1:3.8.0-1) ...
Setting up systemd (251.3-1) ...
Setting up udev (251.3-1) ...
Setting up systemd-sysv (251.3-1) ...
Setting up xserver-xorg-core (2:21.1.4-1) ...
Setting up libpam-systemd:amd64 (251.3-1) ...
Setting up libdevmapper1.02.1:amd64 (2:1.02.185-1) ...
Setting up dmsetup (2:1.02.185-1) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.142) ...
update-initramfs: Generating /boot/initrd.img-5.18.0-2-amd64
Processing triggers for libc-bin (2.33-8) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for dbus (1.14.0-2) ...
...
--8<---------------cut here---------------end--------------->8---
Here my haskell Turtle code - I think the magic is just that
(Turtle-)procs (and in turn proc) inherits stdio and stderr
https://hackage.haskell.org/package/turtle-1.6.1/docs/src/Turtle.Prelude.html#proc
(I am omitting some details here: h is just some definition of a host,
and #{host h} turns that into a string, I am using QuasiQuotes +
OverloadedStrings + PackageImports etc. - and sorry for my Haskell
style: using tail + empty first strings everywhere):
--8<---------------cut here---------------start------------->8---
import Turtle hiding (u, env, s)
import System.Console.ANSI
import "interpolate" Data.String.Interpolate
vividCyan s = setSGRCode [SetColor Foreground Vivid Cyan] ++ s ++ setSGRCode [Reset]
myapt h = do {
; liftIO $ putStrLn $ vividCyan ( unwords $ words [i| /usr/bin/ssh -l root #{host h} apt-get update ... |] )
; procs "/usr/bin/ssh" (tail [ ""
, "-l", "root"
, host h
, "apt-get"
, "update"
]) empty
; liftIO $ putStrLn $ vividCyan ( unwords $ words [i| /usr/bin/ssh -l root #{host h} #{env} apt-get -y upgrade ... |] )
; procs "/usr/bin/ssh" (tail [ ""
, "-l", "root"
, host h
, env
, "apt-get"
, "-y"
, "upgrade"
]) empty
; liftIO $ putStrLn $ vividCyan ( unwords $ words [i| /usr/bin/ssh -l root #{host h} #{env} apt-get -y autoremove ... |] )
; procs "/usr/bin/ssh" (tail [ ""
, "-l", "root"
, host h
, env
, "apt-get"
, "-y"
, "autoremove"
]) empty
}
where env = "DEBIAN_FRONTEND=noninteractive"
--8<---------------cut here---------------end--------------->8---
Propellor by the way does a weird thing: it shows the line-by-line
progress for apt, but in a somehow shortened way (not all the details
i.e.). - And at some point (for local user scripts) I have thus created my
own ptyUserScriptProperty - giving "su" a "--pty" param - otherwise
copied from: https://hackage.haskell.org/package/propellor-5.13/docs/src/Propellor.Property.Cmd.html#userScriptProperty
--8<---------------cut here---------------start------------->8---
ptyUserScriptProperty :: User -> Script -> UncheckedProperty UnixLike
ptyUserScriptProperty (User user) script = cmdProperty "su"
["--pty", "--login", "--shell", "/bin/sh", "-c", shellcmd, user]
where
shellcmd = intercalate " ; " ("set -e" : "cd" : script)
--8<---------------cut here---------------end--------------->8---
Is this possible in consfigurator? - And of course, I would be
interested to see this kind of line-by-line output not only for
apt:updated, apt:upgraded, ..., but more generally, for example
for single commands:
--8<---------------cut here---------------start------------->8---
(defproplist mystuff :posix ()
(cmd:single "(cd /etc/exim4 && pwd)")
(cmd:single "echo hi there")
)
--8<---------------cut here---------------end--------------->8---
I assume, I could create some animated gifs, to make my point clear (see
the line-by-line output, as commands progress), but maybe it is clear by now
already?
And one other thing (concerning Python-devops frameworks) - even though
really I am not using Python any more:
* whereas in (the all so popular) Ansible I cannot see immediate line-by-line output
* in Infra I can:
https://pyinfra.com/
(usage is comfortable - even though I find many other things way too complicated/confusing there)
Thanks in advance.
-A
More information about the sgo-software-discuss
mailing list