make-secnet-sites: Move input file reading further down the file

This separates it from the option parser, which I want to move and
rewrite.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

make-secnet-sites: Introduce a notion of listish types.

A property of such a listish type can be assigned multiple times, and
the values accumulate, and get reported as a list in the output
configuration.

Currently none are defined, so you can't see what this does.

Signed-off-by: Mark Wooding <mdw@distorted.org.uk>

make-secnet-sites: Introduce a superclass for the config types.

Somewhere to put common behaviour.  Not that there is any yet, so
there's no functional change.

Signed-off-by: Mark Wooding <mdw@distorted.org.uk>

mtest/t-userv: Check for dangerous parsing of late options

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest/t-userv: Break out `good'

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Dump logfile(s) of failing test(s)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Print subdir in summary output too

Now we have multiple subdirs the output might be interleaved.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Provide `recheck' to rerun fast tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Tidy up output from Makefile

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Break out diff-output

No functional change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Provide run-mss

No caller yet

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest/t-userv: Check the expected output

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Set PYTHONBYTECODEBASE here too

This prevents ad-hoc manual runs from genrating unwanted cache files.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Set PYTHONHASHSEED

This will allow us to avoid test output being reordered due to hash
instability.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Wire up into toplevel Makefile

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Provide a makefile to run the tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-common: Set PYTHONBYTECODEBASE to /dev/null

Python is not entirely reliable at figuring out when its .pyc files
are out of date, especially if you do something like
  git-rebase -i --exec 'make check-mtest' <commitish>

So squash the bytecode cache entirely.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-common: Rename SECNET_TEST_BUILDDIR variable

No longer just stest.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-common.make: Add missing dependencies on makefiles

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-common.make: Fix hardcoded stest references

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Break out test-common.make

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: make clean calls clean in stest

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Add missing test-common.tcl to DEPS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Break out DEPS

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Honour MTEST_PYTHON

To allow running with different python versions.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Break out mss-run-userv

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: First test case

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-common: Handle mtest correctly too

The default value for tmp nees to be right for mtest/ too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Break out prefix_some_path

This incidentally fixes a bug: previously, we wrote PRELOAD rather
than LD_PRELOAD in one place, which meant that existing LD_PRELOADs
would be overwritten.  Now they no longer are.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Break out test-common.tcl

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Test files for make-secnet-sites userv mode

No test execution machinery yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Use proper builddir subdir as default tmp

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

.gitignore: ignore config.stamp.in too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Use topbuilddir (now in common.make)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Use common.make and therefore our standard CFLAGS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest/udp-preload: Fix some compiler warnings

These come up with our standard CFLAGS which we are erroneously not
using.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: stest: Fix out-of-tree builds

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: test-example: Fix out-of-tree builds

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Move srcdir setting out of common.make.in

This varies according to the cwd.  So for common.make.in it is always
the top-level.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Process test-example with autoconf

This makes configure make the directory during out-of-tree builds.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Rename from `test'

We want other tests too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Rename stamp-h to config.stamp

This makes more sense and gets it out of the way of "st..." tab
completion which we are going to want in a momen.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

ipaddrset-test: Fix network with host bits

2001:23:24:: has 3x16 bits set, ie /48.  This was always wrong.

We need to fix this now because we are going to switch to ipaddress
from ipaddr, which actually checks this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Add a missing dependency on the sites file

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Rerun tests only when deps changed

By touching the stamp file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: New t-dyni-kex

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Beak out proc test-kex

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Slurp test-example/sites.conf and paste it in

This will enable us to edit this common config.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Drop redundant headers

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Fix copyright dates and error message

Also upgrade the licence to GPLv3+ like the rest of secnet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Fix build dependencies so `make check' works in sbuild again

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Disconnect -j for check parallelism

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Wire into "make check"

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Use test/d-* instead of test/tmp for everything

Now it is actually ok to run multiple tests in parallel.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Makefile rune for `check'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Move sockets in a subdirectory

They need to be not world-accessible and this is the easiest way.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Specify the LD_PRELOAD etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Split "invoke" up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Use $(CC) for link, provide clean target

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Build system

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Makefiles: Break some settings out into common.make

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Provide recvfrom

Now we can do a key exchange!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Fix inet_ntop calling convention

inet_ntop has a weird error return protocol.  And our code for calling
it never worked properly because we didn't strip the leading directory
names from the bound socket name.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Proxy udp packets

We must change the config to specify localhost addrs explicitly,
because we don't implement any special logic for IN[6]ADDR_ANY.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Cope with -ve fds

Eg, Tcl passes -1 to close (!)

 #0  0x00007f62949883ca in close (fd=-1) at udp-preload.c:207
 #1  0x00007f6294719362 in Tcl_FinalizeNotifier () from /usr/lib/x86_64-linux-gnu/libtcl8.6.so

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Run secnet under strace

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Provide sendto

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Prepare for wrapping fns that don't return int

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Introduce sun_prep

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Consolidate program name in argl

This avoids pratting about with the weird way execl takes its
arguments.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Provide close

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Provide getsockname

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Provide setsockopt

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

iaddr_to_string: Do not falsely claim bad addrs are scoped IPv6

In particular, if the AF is neither INET nor INET6, adns_addr2text
quite rightly fails with EAFNOSUPPORT.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Remove now-obsolete `bound'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Fix binding, unlink

Avoids EADDRINUSE from the real bind(2).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Fix binding

inet_ntop needs just the addr field.  How "convenient".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: More actual implementation

Now needs to be invoked like this

  UDP_PRELOAD_DIR=test/tmp LD_PRELOAD=test/udp-preload.so test/invoke

It binds to test/tmp/...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Beginning of actual implementation

  gcc -Wall -D_REENTRANT -fPIC -c udp-preload.c && \
  ld -shared -soname foo.so.1 udp-preload.o -o udp-preload.so -ldl -lc

produces a library with which

  LD_PRELOAD=test/udp-preload.so test/invoke

produces various complaints like

  udp (test/tmp/outside.conf:19): setsockopt(,IPV6_V6ONLY,&1,): Operation not supported
  udp (test/tmp/inside.conf:19): socket [::]:16913 experiencing some trouble transmitting IPv6 (to [::1]:16900): Bad file descriptor

This is progress.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Proof of concept wrapping (2)

  gcc -D_REENTRANT -fPIC -c udp-preload.c && \
  ld -shared -soname foo.so.1 udp-preload.o -o udp-preload.so -ldl -lc

produces a library with which

  LD_PRELOAD=test/udp-preload.so test/invoke

still works.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Proof of concept wrapping

  gcc -D_REENTRANT -fPIC -c udp-preload.c -ldl -lc && \
  ld -shared -soname foo.so.1 udp-preload.o -o udp-preload.so

produces a library which makes secnet go

  secnet fatal error: Failed to initialise ADNS: Message too long

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: udp-preload: Import libauthbind.c

This is from authbind 2.1.2 64b7841344fcc3cc.  It is GPLv2+ and
my own copyright so no licence problem.  I'm going to hack it up into
what is needed here.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Use dedicated tmp directory in variable

We are going to want to run multiple tests at once, so we'll need
this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: if things aren't working time out rather than waiting forever

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: expect initial ping packet to come back on inside, for now

We are pinging the remote secnet, which responds without sending the
packet to its host (ie, "outside.r" in our terms).

We can improve this later if we care.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: send an initial ping packet

The hex data came from
  ping -I secnet-test-i 172.18.232.1
  tcpdump -wt -isecnet-test-i
  tcpdump -rt -x

We send it twice because the first causes a key exchange and then gets
lost.

The result is that the script crashes with
  inside rx'd!

This is because we are pinging the remote secnet, which responds
without sending the packet to its host (ie, "outside.r" in our terms).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: use chiark-tcl-hbytes rather than ad-hoc \x quoting

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Fix fd plumbing

In a background process, stdin defaults to /dev/null, so we need to do
a little dance.  <&0 looks like it ought to work according to the bash
manual and does indeed work in bash, but not in dash.  This
construction seems to work in both.

Tested with:
  date | sh -exc 'cat <&0 >t &'

SuS says:
  If job control is disabled (see set, -m), the standard input for an
  asynchronous list, before any explicit redirections are performed,
  shall be considered to be assigned to a file that has the same
  properties as /dev/null. This shall not happen if job control is
  enabled. In all cases, explicit redirection of standard input shall
  override this activity.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Provide bgerror

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Send initial confirmation byte, and wait for some rx

Now this hangs indefinitely because we don't send a packet.  That
comes next.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Fix names of stunt pipes

"t" = transmit, ie packets written by us and read by secnet
"r" = receive, ie packets writeen by secnet and read by us

secnet is the network; we are the host.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Use stunt pipes and ipif shell script for netlink

The overall effect is that now secnet crashes as soon as invoke
completes (ie, immediately) because there's no writer for the netlink
pipe.  This is good.  Also we no longer need "really".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Move common config into test script

No functional change.  This duplicates the example, and we are going
to modify/parameterise the copy in "invoke".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Generate configs from pieces

The resulting files are semantically equivalent to
test-example/{in,out}side.conf.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test: Provide first cut of invoke script

This is going to be reorganised and improved a lot.  Right now it
doesn't do much and doesn't exit.  If ^C'd it leaks the secnet
processes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

alg_msg_data: Remove "sig" from member names

We are going to want to use this for other algorithms too.

Suggested-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

site: hex_init: Call from enter_new_state

This seems more logical.  It also more clearly separates it from the
hacky_par system.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

site: kex_init: Have it return a boolean

At some point this will become capable of failing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

site: Break out kex_init

We are going to want to put some more things here.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

hash: Provide and use hash_hash connvenience function

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>