regen.mk.in: Debugging, controlled by SUBDIRMK_REGEN_NDEBUG

And enable this in test/example/check.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Test Perdir.sd.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README: Do not assert that all directive arguments are &-expanded

Since
  Syntax: only some directives' RHS should be &-expanded
this has not been true for &:changequote, and it is not true in
the general case.  It depends on the directive.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README: Document NEWQUOTE\$ in changequote

This also adds a doctest.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Line joining with &\

make has a weird rule for line joining in recipes, where the \ is
retained.  This is very annoying espcially when writing little scripts
in other languages in recipes.  The GNU make documentation suggests
using a `define' block, in which the \ are still necessary, but
vanish.  But that separates out the recipe from the rule, and also
demands that the recipe text be given a name.

(Note that .ONESHELL offers an alternative but sadly is not available
on a per-rule basis.  This is even less sensible with non-recursive
make, where you might want different directories to have different
settings.)

So, provide &\ which does the other thing, swallowing the newline and
discarding the &\, even in rules.

Add a test case too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Test &TARGETS feature

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Refine &TARGETS_things, mostly to avoid # wrinkle

We will add a test case in a moment, since it is quite a noisy diff.

We have to upate clean.sd.mk, which uses this feature.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: only some directives' RHS should be &-expanded

In principle this is an incompatible change but only for strange input
files and only if they use &:changequote.

We need an ad-hoc test for this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Provide &$VARIABLE

Testing is done by the doctests.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Test in sub/dir too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Add missing & from include

This makes no difference in this file but we'll want the same in
sub/dir/ and they should be the same.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: check: Define i and o vars

This makes bash -x debugging easier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Fail on extraction if we don't understand the README

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests/filter/check: Enable pipefail

So we must use bash.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Extract and use

Include the doctests from the toplevel Subdir.sd.mk.

In check, extract the doctests, and seddery the expected output into
the combined expected output file.  This involves adding a comment in
Subdir.mk.expected to indicate where the doctests will appear.

In update-expected, contrive to strip the doctest output back out of
the actual output so we can update only the manually-edited part.

Consequential changes to MAKEFILE_TEMPLATES in main.mk.expected
and .gitignore.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Fix newline handling in changequote mode section

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README, doctests: Use NEWQUOTE instead of STUFF

STUFF is rather generic.  Also we want to seddery it in the rubric for
the doctest to avoid it triggering.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

doctests: Provide extract-doctest

This is not called yet.  There are a few things that need fixing
first.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README: Do not use => inappropriately

Our notation is that => introduces the ouput string.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Use &\ for escaping; abolish \&

"\&" causes lots of trouble, including difficulty specifying certain
resulting strings.

Previously I proposed a scheme using &$@ but this is only useful to
provide a way to write &s.

The $-doubling regime which will come later wants a way to produce a
single literal $, so introduce that now using the same syntax.
And &\ feels quite natural to me.

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Introduce &. and friends

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

generate: Provide $var_prefix_name to process_input_mk

Nothing uses this yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Syntax: Introduce &:changequote

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

generate: Track output Makefiles properly

This fixes a bug where the intermediate directory Makefile was not
included whe it should be.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests/filter: Add .../sub/dir

This will let us test output which depends on the subdirectory,
which is kind of the whole point...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Shuffle, make tests/check run all the tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests/filter: New test framework for generate

This test case bypasses autoconf and runs generate directly.
It also gives us somewhere to write examples of each of the
&-expansions without further cluttering test-example.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

generate: Handle nodes with no own Subdir.sd.mk

If a directory is not mentioned on the command line, we would assume
it had a Subdir.sd.mk anyway.  Instead, expect it not to have one.

We do still generate a per-subdirectory fragment for such unmentioned
nodes: this serves to wire its children ino the parent, make running
`make' in that directory possible, etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

generate: Add missing final newline to main.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Add some section comments.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

example: Add a comment next to for-test bits

This may help unconfuse people using this as an example.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README: Fix a mistake in the file reference table

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge from secnet

&TARGETS*: make autogenerated targets .PHONY

Suggested-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

docs: Minor fix to README

Reported-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

docs: Minor fixes to regen.mk.in comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-example: Fix a dependency on Subdir.mk

This should be *this* directory's Subdir.mk.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test-example: Replace a literal with $@

No functional change.  This makes the command a bit more general.
At some much later point this will make a future commit simpler...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

comprehensive-test: Test that clean target works

We have to have a list of exceptions for the output of configure.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Add some missing clean and cdeps hooks

Tested with
  git-clean -nX | grep -v \*~ | grep -v '~$' | grep -v Makefile | grep -v '\.mk$'

We now clean, additionally,
  .msgcode-test.d
  .version.d
  ipaddrset.confirm

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Switch to using subdirmk's clean and cdeps

Even in the toplevel.  This cleans some more things but doesn't fix
all the lack of clean bugs.  Right now the effect is to remove
Subdir.mk.tmp and main.mk.tmp, if they exist.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Fix breakage if nproc is not installed

This needs to echo 1, not try to run it.  The effect is a message on
stderr and passing `make -j'.  That is actually mostly OK since we
have few tests right now...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system, tests: Marginally speed some things up with -f

This avoids an additional redirection through the stub Makefiles.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

comprehensive-test: Test the stest/ and mtest/ directories

Running these separately after make clean will help spot missing
dependencies.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pretest-to-tested: Script for helping run comprehensive-test

This runs it on every commit between tested and pretest, advancing
tested as it goes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Suppress MAKEFILE_TEMPLATES when running `make clean'

See the README hunk for the reasoning.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

comprehensive-test: new script

This does an in-tree build, and two out-of-tree builds, one absolute,
and one relative.  It runs `make check' (not fullcheck, which is very
slow indeed).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: break out subdirmk_target in generated Makefile

We are going to want to reuse this.  Make it a simply-expanded make
variable rather than a perl variable, so that it only gets computed
once.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest/udp-preload: Be more relaxed about paths

We would give ENAMETOOLONG if the specified path plus a maximal IPv6
address string wouldn't fit.  But in practice we don't use IPv6
addresses which stringify to anything that long, and this imposes an
annoyingly short restriction on the length of the emulation socket
directory path (a restriction which is going to be not met in my own
working tree when I add more sophisticated tests).

Instead, pass the suffix string to sun_prep and do the length check on
the actual length.  This means that addresses with short strings can
work, even if in principle addresses with long strings might fail.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest/udp-preload.c: Fix error handling of sun_prep

If sun_prep returned an error (eg, path too long), this would
segfault.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Cope if HEAD refers to a packed ref

In this case, there may be no corresponding .git/refs/<whatever>.
Simply tolerate this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Suppress MAKEFILE_TEMPLATES when running `make clean'

See the README hunk for the reasoning.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: break out subdirmk_target in generated Makefile

We are going to want to reuse this.  Make it a simply-expanded make
variable rather than a perl variable, so that it only gets computed
once.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: stub Makefiles: filter out `all'

Otherwise `make all' generates this:
  Makefile:3: target 'all' given more than once in the same rule
which is harmless but mildly irritating.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: stub Makefiles: filter out `all'

Otherwise `make all' generates this:
  Makefile:3: target 'all' given more than once in the same rule
which is harmless but mildly irritating.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge commit '0428556765a15326025e9fe4809936c6cafd3c85'

subdirmk: clean.sd.mk: Delete .*~ too (eg .gitignore~)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: clean.sd.mk: Delete .*~ too (eg .gitignore~)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: cdeps.sd.mk: Honour &CDEPS_OBJECTS

This makes this facility more flexible.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: cdeps.sd.mk: Honour &CDEPS_OBJECTS

This makes this facility more flexible.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge commit 'a1e52614e600bed8e51b461e6569f2cd263e82bb'

subdirmk: Fix bug in regen.mk.in

Apparently, if you just write `;', make knows that the command can't
have done anything.  It therefore concludes tha the target file cannot
have been updated and doesn't bother rereading it.

The result is that you end up using last run's makefiles.

Writing @: fixes this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Fix bug in regen.mk.in

Apparently, if you just write `;', make knows that the command can't
have done anything.  It therefore concludes tha the target file cannot
have been updated and doesn't bother rereading it.

The result is that you end up using last run's makefiles.

Writing @: fixes this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge commit '6fce06f32c1dd21b5c2d7a142d54b7153d650574'

subdirmk: Change stub force target to `run-main.mk'

This produces significantly better error messages:
  [some error from actual makefiles]
  make[1]: Leaving directory '/home/ian/wherever'
  Makefile:6: recipe for target 'run-main.mk' failed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Change stub force target to `run-main.mk'

This produces significantly better error messages:
  [some error from actual makefiles]
  make[1]: Leaving directory '/home/ian/wherever'
  Makefile:6: recipe for target 'run-main.mk' failed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Fix bug when target in subdir exists

If you try to make a file that exists, the % rule doesn't seem to
match.  But providing the dependency for all of $(MAKCMDGOALS)
works.

We have to mention `all' explicitly because it might not appear in
$(MAKECMDGOALS).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Fix bug when target in subdir exists

If you try to make a file that exists, the % rule doesn't seem to
match.  But providing the dependency for all of $(MAKCMDGOALS)
works.

We have to mention `all' explicitly because it might not appear in
$(MAKECMDGOALS).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 0.5.2~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

stest: Suppress a GCC warning (for stretch)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

finalise 0.5.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

make-secnet-sites: Add test for -P option

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

make-secnet-sites: Actually fix -P option (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: Provide for sedderyb

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Release checklist: mention Subdir.sd.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Prepare 0.5.1 (more)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Prepare 0.5.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge commit 'd817b49007e60b0e7392f23caf08dc0328eb32de' into HEAD

build system: Add warning suppressions for GCC9

We suppress 4 instances like:

  udp.c:113:45: error: increment of a boolean expression [-Werror=bool-operation]
    113 |     if (us->experienced[!!dest][af][success]++)
|                                             ^~

This is a very convenient idiom (using the saturating property of
boolean addition) and there is no sensible replacement.

And we suppress 6 instances like:

  tun.c:322:6: error: 'strncpy' specified bound 16 equals destination size [-Werror=stringop-truncation]
    322 |      strncpy(ifr.ifr_name,st->interface_name,IFNAMSIZ);
|      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All write to ifr_name.  strncpy is precisely right for this, since the
API supports non-null-terminated names of length IFNAMSIZ.  That is
why I used strncpy.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

polypath.c: Fix a compiler warning

  polypath.c:520:20: error: '*' in boolean context, suggest '&&' instead [-Werror=int-in-bool-context]
    520 |     *allreasonable *= reasonable;
        |                    ^~

This is not really a very helpful suggstion because there is no &&=.
Happily the compiler doesn't (currently) complain about &= which is
the same when both arguments are bools.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

make-secnet-sites: sort the properties in level.output_props

This makes the test suite less sensitive to python version.
Now it works with 2.7.13, 3.5.3,, 3.7.5.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

make-secnet-sites: sort the properties

This makes the test suite less sensitive to python version.
Now it works with 2.7.13, 3.5.3.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

mtest: diff: Write output files and seddery rune

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tests: Introduce `diff' proc

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: tests: Add missing standard clean

We want (i) recursion from the parent (ii) automatic cleaning of
&CLEAN (which is set by cdeps.sd.mk, amongst other things).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Drop `check' from `all'

Otherwise there's no way to make everything but not run tets.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Do always regenerate version.o

We make two changes which affect only the litle recursive make of
version.o.  We only build version.o in the little sub-make when
relinking and then we want to always do it.  So the setting of OBJECTS
is pointless, and instead we make version.c depend on FORCE.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build system: Fix nits following switch to subdirmk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Docs: Fix some leftover ; and ,

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Docs, go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Docs, go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Implement new syntax

But I plan to go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Implement new syntax

But I plan to go back to ^ and ~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge commit 'c0b216c79763f720df20e002fc14c9348a0e05c2'

docs: Discuss variable settings and inclusion order

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

subdirmk: Actually clear right variable

"subdirmk: main.mk: clear SUBDIRMK_MAKEFILES MAKEFILE_TEMPLATES"
cleared MAKEFILES by mistake (this was a rebase transposition error -
this commit originally came before "subdirmk: Rename MAKEFILES
variable".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

docs: Abolish all absolute pathname syntaxes

These are rare and make things much more confusing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

docs: Improve docs.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>