chiark / gitweb /
subdirmk.git
5 months agoWarning reporting: Add a test of the warning suppression system
Ian Jackson [Sat, 28 Dec 2019 13:55:59 +0000 (13:55 +0000)]
Warning reporting: Add a test of the warning suppression system

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarning reporting: Warning suppression system
Ian Jackson [Sat, 28 Dec 2019 13:55:12 +0000 (13:55 +0000)]
Warning reporting: Warning suppression system

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotest/filter; Cause some warnings, to check they appear
Ian Jackson [Sat, 28 Dec 2019 12:58:03 +0000 (12:58 +0000)]
test/filter; Cause some warnings, to check they appear

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Sort occurrences properly in local+global warnings
Ian Jackson [Sat, 28 Dec 2019 12:52:01 +0000 (12:52 +0000)]
Warnings: Sort occurrences properly in local+global warnings

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Warn for confusing single-char $ expansions
Ian Jackson [Sat, 28 Dec 2019 14:05:19 +0000 (14:05 +0000)]
Warnings: Warn for confusing single-char $ expansions

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Track variable references in &-expansions
Ian Jackson [Sat, 28 Dec 2019 12:31:40 +0000 (12:31 +0000)]
Warnings: Track variable references in &-expansions

For &-escapes which match the whole variable name, we can conveniently
track variable expansions as part of the processing.  (We don't change
any of the matching regexps.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Track basic $-references
Ian Jackson [Sat, 28 Dec 2019 12:31:49 +0000 (12:31 +0000)]
Warnings: Track basic $-references

We must always stop at $'s in the input now.

Having stopped at $, we normally just output it and carry on.  Ie, we
only stop so we can do some inspection: as before, we do this variable
tracking as inspection before processing, rather entangled with
processing.

We can deal reasonably properly with ${ } and $( ).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Track settings of variables
Ian Jackson [Sat, 28 Dec 2019 12:27:40 +0000 (12:27 +0000)]
Warnings: Track settings of variables

Where the variable tracking does not involve &-escapes, or only
involves &-escapes which work like a prefix so the processor does not
have the whole variable name, we do this separately from processing.

This is simpler because we can look ahead more.  It also avoids
disturbing the processing logic (which needs to be precisely accurate,
unlike this variable tracking).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: Infrastructure for tracking and warning about variables
Ian Jackson [Sat, 28 Dec 2019 12:59:11 +0000 (12:59 +0000)]
Warnings: Infrastructure for tracking and warning about variables

We are going to track when we see FOO or &FOO.  If we see both, we
issue a warning, as that might mean the programmer forgot a &.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoWarnings: generate: Basic `wrn' function
Ian Jackson [Sat, 28 Dec 2019 12:58:46 +0000 (12:58 +0000)]
Warnings: generate: Basic `wrn' function

This is sufficient to add warnning calls to the code.  We'll soup it
up later.  No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter/check: Capture generate's stderr
Ian Jackson [Sat, 28 Dec 2019 12:55:17 +0000 (12:55 +0000)]
tests/filter/check: Capture generate's stderr

We are going to cause `generate' to emit warnings, which we want to
check are working.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotest/filter: Test &${ ... } rather better
Ian Jackson [Sat, 28 Dec 2019 23:07:01 +0000 (23:07 +0000)]
test/filter: Test &${ ... } rather better

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Fix &${ } end condition
Ian Jackson [Sat, 28 Dec 2019 23:01:49 +0000 (23:01 +0000)]
generate: Fix &${ } end condition

It is quite wrong to use $ddbl, which might be adjusted by &$- &$+.
We must maintain a separate counter.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Fix { } in &${ to actually output the { }
Ian Jackson [Sat, 28 Dec 2019 12:17:14 +0000 (12:17 +0000)]
generate: Fix { } in &${ to actually output the { }

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter: Rename `doctest' -> `doctests' everywhere
Ian Jackson [Sat, 28 Dec 2019 11:34:56 +0000 (11:34 +0000)]
tests/filter: Rename `doctest' -> `doctests' everywhere

This is more uniform.  A lot of churn, though.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Move &:changequote further down
Ian Jackson [Sat, 28 Dec 2019 18:19:17 +0000 (18:19 +0000)]
README: Move &:changequote further down

This is a bit of a minority interest.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Explain a wrinkle in &:include filename semantics
Ian Jackson [Sat, 28 Dec 2019 00:24:15 +0000 (00:24 +0000)]
README: Explain a wrinkle in &:include filename semantics

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: use oraw in a few more places
Ian Jackson [Thu, 26 Dec 2019 02:10:03 +0000 (02:10 +0000)]
generate: use oraw in a few more places

This gets rid of some open-coded `print O'.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Miscellaneous fixes and clarifications
Ian Jackson [Thu, 26 Dec 2019 01:49:49 +0000 (01:49 +0000)]
README: Miscellaneous fixes and clarifications

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Document restriction on $-doubling and lack of -quadrupling
Ian Jackson [Thu, 26 Dec 2019 02:03:07 +0000 (02:03 +0000)]
README: Document restriction on $-doubling and lack of -quadrupling

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Move `Tables of file reference syntaxes' down
Ian Jackson [Thu, 26 Dec 2019 01:48:38 +0000 (01:48 +0000)]
README: Move `Tables of file reference syntaxes' down

This is more of an appendix or addendum.  It should come after the
complete description of the substitution syntax.

Pure motion.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoBig incompatible change: Rename `Subdir' to `Dir'
Ian Jackson [Thu, 26 Dec 2019 01:40:09 +0000 (01:40 +0000)]
Big incompatible change: Rename `Subdir' to `Dir'

We want completion to be easy, and we have Suffix now.  `Dir' is as
good as `Subdir' I think (and a bit shorter).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Fix messages resulting from bad directive arguments
Ian Jackson [Thu, 26 Dec 2019 01:32:43 +0000 (01:32 +0000)]
Error handling: Fix messages resulting from bad directive arguments

"Unknown directive" is not necessarily accurate; another possibility
is that the directive is known but didn't match the parsing regexp
because of problems with the arguments.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Nested scope: Change `Eval' to `eval'
Ian Jackson [Thu, 26 Dec 2019 01:33:01 +0000 (01:33 +0000)]
generate: Nested scope: Change `Eval' to `eval'

This is the nesting kind name.  But it also appears in error messages
where it ought not to be capitalised.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Nested scope: Change `Macro' to `macro'
Ian Jackson [Thu, 26 Dec 2019 01:32:36 +0000 (01:32 +0000)]
generate: Nested scope: Change `Macro' to `macro'

This is the nesting kind name.  But it also appears in error messages
where it ought not to be capitalised.

We'll change `Eval' in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Trim the `bad escape' message.e
Ian Jackson [Sat, 28 Dec 2019 00:28:03 +0000 (00:28 +0000)]
Error handling: Trim the `bad escape' message.e

Do not print the whole line, only the next 5 chars (say) - and, also,
not any newlines.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Better reporting of nest-related errors
Ian Jackson [Sat, 28 Dec 2019 01:04:43 +0000 (01:04 +0000)]
Error handling: Better reporting of nest-related errors

Track `$what' which is a kind of example for the error message.
Use it in error messages.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Often say `subdirmk' rather than `$0'
Ian Jackson [Sat, 28 Dec 2019 00:27:58 +0000 (00:27 +0000)]
Error handling: Often say `subdirmk' rather than `$0'

The latter is generally a path (`.../generate') and may be confusing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Replace or update many calls to `die'
Ian Jackson [Sat, 28 Dec 2019 00:27:13 +0000 (00:27 +0000)]
Error handling: Replace or update many calls to `die'

To conform to new approach.

The `ending wrong kind of nest' error will be dealt with in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoError handling: Provide err and ddbl_only, and an internal spec
Ian Jackson [Sat, 28 Dec 2019 00:26:16 +0000 (00:26 +0000)]
Error handling: Provide err and ddbl_only, and an internal spec

No callers of these yet.  The spec is honoured in the breach.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Make @nest always be nonempty
Ian Jackson [Thu, 26 Dec 2019 01:04:58 +0000 (01:04 +0000)]
generate: Make @nest always be nonempty

No outward functional change, but makes the error handling
etc. slightly simpler.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Do not tolerate unclosed nested scopes at EOF
Ian Jackson [Sat, 28 Dec 2019 00:51:43 +0000 (00:51 +0000)]
generate: Do not tolerate unclosed nested scopes at EOF

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoMacro assistance part 2 - syntax for $(eval $(call...))
Ian Jackson [Tue, 24 Dec 2019 23:04:58 +0000 (23:04 +0000)]
Macro assistance part 2 - syntax for $(eval $(call...))

This can be used to call a macro defined with &:macro.  In combination
these provide a more reasonable macro facility than make's.

Specifically:

* Document the new facility.
* Motivate it in the README.
* Provide the new @nest kind for &${  }, which uses $ddbl as
  a counter to count { and }.
* Implement the {}-counting and nesting end in the main parsing
  loop.  (We must now search for things besides $esc.)
* In extract-doctest, provide a slightly adhoc special case for
  understanding ${eval ...} as an expansion text in the README.
  And do not attempt to test &${ } when already $-doubling.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoMacro assistance part 1 - macro directive
Ian Jackson [Sat, 28 Dec 2019 00:36:50 +0000 (00:36 +0000)]
Macro assistance part 1 - macro directive

Define the &:macro directive, which is a dollar-doubled version of
make's own `define'.

We must introuce a new concept of `nesting', for when "the content of
the construct is $-doubled" in the words of the README.  The
implementation is in the main process_input_mk function: it is a
multi-line scope of some kind with an ad-hoc ending condition, and its
own dollar-doubling setting.

We will also want a convenient syntax for $(eval $(call...)), which
we're going to introduce in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Rename `desc' etc. in extract-doctest from `rubric'
Ian Jackson [Fri, 27 Dec 2019 18:26:11 +0000 (18:26 +0000)]
doctests: Rename `desc' etc. in extract-doctest from `rubric'

This isn't a rubric, it is a description of a particular test entry.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Improve entry description filter
Ian Jackson [Tue, 24 Dec 2019 16:05:49 +0000 (16:05 +0000)]
doctests: Improve entry description filter

This makes the decriptive text more aposite in some situations.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoDollar doubling feature
Ian Jackson [Wed, 25 Dec 2019 23:29:40 +0000 (23:29 +0000)]
Dollar doubling feature

This makes it easier to write certain rules commands, and macros
involving eval.  Further enhancements will follow, but for now:

* Document the new facility, including the &-escapes to turn it on and
  off, its effect, and the three within-doubling don't-double escapes.

* Implementation.  Specifically:
   - Rename od to oud (`undoubled').
   - Provide od which does double things if $ddbl (a global) is set.
   - Implement the escapes, and add some error checks.
   - Change `od' to `oud' in `&\$', so that that always produces an
     undoubled $.

* Enhance extract-doctest:
   - Set $e->{DD} to say if it's in `while dollar doubling'.
   - Filter out such entries from the `normal' test - they are not
     legal outside dollar-doubling.
   - Add a dollar-doubling test which includes versions of all
     the normal tests as well as the dollar-doubling ones, and
     which dollar-doubles the outputs when appropriate.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter/.gitignore: Ignore all .tmp
Ian Jackson [Thu, 26 Dec 2019 00:26:55 +0000 (00:26 +0000)]
tests/filter/.gitignore: Ignore all .tmp

Rather than trying to list them all.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Introduce oraw
Ian Jackson [Wed, 25 Dec 2019 23:45:32 +0000 (23:45 +0000)]
generate: Introduce oraw

Replace all other calls to `o' with `oraw'.  This ensures we
have no stray calls.

It elminates an anomaly, namely that calls outside process_input_mk
would rely on the $buffering_output variable which is set only there.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Introduce od
Ian Jackson [Wed, 25 Dec 2019 23:31:13 +0000 (23:31 +0000)]
generate: Introduce od

Replace every call to `o' in process_input_mk (the main filtering
routine) with a call to `od', which is going to be the dollar-doubled
output variant when we add the dollar doubling feature.

For now it just does the same as `o'.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoexample: Introduce a Final.sd.mk and test it
Ian Jackson [Sat, 28 Dec 2019 22:53:47 +0000 (22:53 +0000)]
example: Introduce a Final.sd.mk and test it

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter: Test Final.sd.mk
Ian Jackson [Wed, 25 Dec 2019 20:08:53 +0000 (20:08 +0000)]
tests/filter: Test Final.sd.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoInterface: Read Final.sd.mk
Ian Jackson [Wed, 25 Dec 2019 19:14:57 +0000 (19:14 +0000)]
Interface: Read Final.sd.mk

Tests will come in a moment, although we already see the new output
file mentioned in main.mk.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: fix TARGETS text for new tidier semantics
Ian Jackson [Thu, 26 Dec 2019 01:43:50 +0000 (01:43 +0000)]
README: fix TARGETS text for new tidier semantics

In
  Syntax: Refine &TARGETS_things, mostly to avoid # wrinkle
we updated the spec for &TARGETS_things in the list of expansions,
but not in the discussion of per-directory recursive targets.

Fix that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoInterface: Rename Perdir to Suffix; provide Prefix
Ian Jackson [Sat, 28 Dec 2019 22:45:50 +0000 (22:45 +0000)]
Interface: Rename Perdir to Suffix; provide Prefix

We want to be able to specify something that appears at the top of
every per-directory segment, principally to provide
for-every-directory variable settings.

A more sophisticated scheme akin to m4 diversions would be possible in
principle, but it would make organising the output files (which must
be processed by autoconf) really awkward - let alone the depenency
management for regenerating the makefiles.

So instead, simply provide a Prefix too.  Additionally, soon we are
going to introduce a Final.sd.mk which is processed right at the end.

I think this new scheme will be sufficient in principle to do anything
that might be needed.  Sadly it will continue to require that
&:include's are done in the right order.

"Suffix" and "Subdir" tab-complete really badly.  In a forthcoming
commit we will rename Subdir.sd.mk to Dir.sd.mk.

Incompatible change.  We must update the tests right away.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Make many dir vars globals
Ian Jackson [Wed, 25 Dec 2019 19:35:55 +0000 (19:35 +0000)]
generate: Make many dir vars globals

This makes the code a bit more fragile but a lot less repetitious.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Fix an erroneous comment
Ian Jackson [Wed, 25 Dec 2019 20:02:48 +0000 (20:02 +0000)]
generate: Fix an erroneous comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Drop comment about input files
Ian Jackson [Wed, 25 Dec 2019 19:19:04 +0000 (19:19 +0000)]
generate: Drop comment about input files

This is all documented in README now.
And it's about to become out of date.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/check: Dynamic parallism choice
Ian Jackson [Sun, 29 Dec 2019 13:29:18 +0000 (13:29 +0000)]
tests/check: Dynamic parallism choice

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/check: Show parallel make rune we run
Ian Jackson [Sun, 29 Dec 2019 13:29:05 +0000 (13:29 +0000)]
tests/check: Show parallel make rune we run

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests: Run in parallel
Ian Jackson [Sun, 29 Dec 2019 13:25:42 +0000 (13:25 +0000)]
tests: Run in parallel

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/example: Reset times of (copied) source files too
Ian Jackson [Sun, 29 Dec 2019 13:07:53 +0000 (13:07 +0000)]
tests/example: Reset times of (copied) source files too

This eliminates a problem with our test case: if the build takes less
than 1s the files were probably copied more recently than the 1s ago,
resulting in too much rebuilding (and possibly missing
failed-to-rebuild bugs).

We can now change the timestamp for the build tree to something
earlier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/example: Build in a copy
Ian Jackson [Sun, 29 Dec 2019 13:05:18 +0000 (13:05 +0000)]
tests/example: Build in a copy

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/build-common: Copy subdirmk, rather than symlinking
Ian Jackson [Sun, 29 Dec 2019 13:02:34 +0000 (13:02 +0000)]
tests/build-common: Copy subdirmk, rather than symlinking

This is more like it will be in someone's project.  Also it prevents
us from accidentally touching anything in the source tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/intree: Make copy and break out from tests/example
Ian Jackson [Sun, 29 Dec 2019 12:59:14 +0000 (12:59 +0000)]
tests/intree: Make copy and break out from tests/example

We introduce make_copy, which makes a copy of tests/example
suitable for testing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agobuild tests: Break out build-common
Ian Jackson [Sun, 29 Dec 2019 12:48:55 +0000 (12:48 +0000)]
build tests: Break out build-common

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoregen.mk.in: Clarify documentation about realclean
Ian Jackson [Sun, 29 Dec 2019 12:34:39 +0000 (12:34 +0000)]
regen.mk.in: Clarify documentation about realclean

It's not sensible to `make [the provided] realclean:: into a recursive
target' in that sense since the one we provide is effective, and
present, only in the toplevel.  What we mean is that you can make your
own realclean target and regen.mk's will fit in nicely.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoregen.mk.in: Document CONFIG_STATUS_OUTPUTS
Ian Jackson [Sun, 29 Dec 2019 11:00:47 +0000 (11:00 +0000)]
regen.mk.in: Document CONFIG_STATUS_OUTPUTS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotest/example/check: Reset times for regen tests
Ian Jackson [Sun, 29 Dec 2019 12:27:00 +0000 (12:27 +0000)]
test/example/check: Reset times for regen tests

In principle, on a filesystem with poor time resolution, the updates
to the input files might have the same timestamp as the output files.
Things would then not be regenerated.

Work around this by resetting all timestamps in the build tree to
1s ago.  This should ensure that exactly the one file whose edit we
are simulating seems, to make, to have been changed.

There is a remaining possible problem: if our last actual change to
the subdirmk source tree is less than 1s ago, this might rebuild too
much.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotest/example/check: Make sections appear in stderr output
Ian Jackson [Sun, 29 Dec 2019 12:26:25 +0000 (12:26 +0000)]
test/example/check: Make sections appear in stderr output

This makes the output easier to navigate.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/example/check: Better handling of makefile updates
Ian Jackson [Sun, 29 Dec 2019 11:01:28 +0000 (11:01 +0000)]
tests/example/check: Better handling of makefile updates

Rather than checking just for presence of a string in a templated
output file, also have the input file define a target which we try to
run.  This checks that make actually reread the file.

Also add ||false to the grep lines, which makes the stderr output
more obviously contain a failure when the test fails.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/example: Include non-.sd.mk files in SUBDIRMK_MAKEFILES
Ian Jackson [Sun, 29 Dec 2019 11:01:18 +0000 (11:01 +0000)]
tests/example: Include non-.sd.mk files in SUBDIRMK_MAKEFILES

subdirmk.ac provides @_SUBDIRMK_MAKEFILES@ which is a list of the
*.mk makefiles (corresponding to *.mk.in makefiles) passed to
SUBDIRMK_MAKEFILES.  These need to be included in the
identically-named make variable for regen.mk to rerun config.status
when the .in files are edited.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/example: Correct filename for for-test.mk
Ian Jackson [Sun, 29 Dec 2019 11:00:39 +0000 (11:00 +0000)]
tests/example: Correct filename for for-test.mk

&^/ means "this source directory" so &^/lib/ in lib/ is nonsense and
produces lib/lib/.  Also we didn't want the source directory:
anyway: config.status produces outputs in the build tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoregen.mk.in: Debugging, controlled by SUBDIRMK_REGEN_NDEBUG
Ian Jackson [Sat, 28 Dec 2019 23:17:07 +0000 (23:17 +0000)]
regen.mk.in: Debugging, controlled by SUBDIRMK_REGEN_NDEBUG

And enable this in test/example/check.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests: Test Perdir.sd.mk
Ian Jackson [Wed, 25 Dec 2019 19:21:51 +0000 (19:21 +0000)]
tests: Test Perdir.sd.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Do not assert that all directive arguments are &-expanded
Ian Jackson [Sat, 28 Dec 2019 00:16:58 +0000 (00:16 +0000)]
README: Do not assert that all directive arguments are &-expanded

Since
  Syntax: only some directives' RHS should be &-expanded
this has not been true for &:changequote, and it is not true in
the general case.  It depends on the directive.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Document NEWQUOTE\$ in changequote
Ian Jackson [Sun, 22 Dec 2019 18:25:55 +0000 (18:25 +0000)]
README: Document NEWQUOTE\$ in changequote

This also adds a doctest.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Line joining with &\
Ian Jackson [Wed, 18 Dec 2019 21:44:15 +0000 (21:44 +0000)]
Syntax: Line joining with &\

make has a weird rule for line joining in recipes, where the \ is
retained.  This is very annoying espcially when writing little scripts
in other languages in recipes.  The GNU make documentation suggests
using a `define' block, in which the \ are still necessary, but
vanish.  But that separates out the recipe from the rule, and also
demands that the recipe text be given a name.

(Note that .ONESHELL offers an alternative but sadly is not available
on a per-rule basis.  This is even less sensible with non-recursive
make, where you might want different directories to have different
settings.)

So, provide &\ which does the other thing, swallowing the newline and
discarding the &\, even in rules.

Add a test case too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests: Test &TARGETS feature
Ian Jackson [Wed, 25 Dec 2019 17:14:40 +0000 (17:14 +0000)]
tests: Test &TARGETS feature

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Refine &TARGETS_things, mostly to avoid # wrinkle
Ian Jackson [Mon, 9 Dec 2019 15:05:22 +0000 (15:05 +0000)]
Syntax: Refine &TARGETS_things, mostly to avoid # wrinkle

We will add a test case in a moment, since it is quite a noisy diff.

We have to upate clean.sd.mk, which uses this feature.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: only some directives' RHS should be &-expanded
Ian Jackson [Wed, 25 Dec 2019 17:08:19 +0000 (17:08 +0000)]
Syntax: only some directives' RHS should be &-expanded

In principle this is an incompatible change but only for strange input
files and only if they use &:changequote.

We need an ad-hoc test for this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Provide &$VARIABLE
Ian Jackson [Mon, 9 Dec 2019 16:31:59 +0000 (16:31 +0000)]
Syntax: Provide &$VARIABLE

Testing is done by the doctests.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Test in sub/dir too
Ian Jackson [Wed, 25 Dec 2019 15:43:38 +0000 (15:43 +0000)]
doctests: Test in sub/dir too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Add missing & from include
Ian Jackson [Wed, 25 Dec 2019 15:43:00 +0000 (15:43 +0000)]
doctests: Add missing & from include

This makes no difference in this file but we'll want the same in
sub/dir/ and they should be the same.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: check: Define i and o vars
Ian Jackson [Wed, 25 Dec 2019 15:42:16 +0000 (15:42 +0000)]
doctests: check: Define i and o vars

This makes bash -x debugging easier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Fail on extraction if we don't understand the README
Ian Jackson [Tue, 24 Dec 2019 23:04:07 +0000 (23:04 +0000)]
doctests: Fail on extraction if we don't understand the README

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter/check: Enable pipefail
Ian Jackson [Wed, 25 Dec 2019 01:21:54 +0000 (01:21 +0000)]
tests/filter/check: Enable pipefail

So we must use bash.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Extract and use
Ian Jackson [Tue, 24 Dec 2019 00:49:00 +0000 (00:49 +0000)]
doctests: Extract and use

Include the doctests from the toplevel Subdir.sd.mk.

In check, extract the doctests, and seddery the expected output into
the combined expected output file.  This involves adding a comment in
Subdir.mk.expected to indicate where the doctests will appear.

In update-expected, contrive to strip the doctest output back out of
the actual output so we can update only the manually-edited part.

Consequential changes to MAKEFILE_TEMPLATES in main.mk.expected
and .gitignore.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Fix newline handling in changequote mode section
Ian Jackson [Wed, 25 Dec 2019 12:55:48 +0000 (12:55 +0000)]
doctests: Fix newline handling in changequote mode section

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME, doctests: Use NEWQUOTE instead of STUFF
Ian Jackson [Wed, 25 Dec 2019 12:55:02 +0000 (12:55 +0000)]
README, doctests: Use NEWQUOTE instead of STUFF

STUFF is rather generic.  Also we want to seddery it in the rubric for
the doctest to avoid it triggering.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodoctests: Provide extract-doctest
Ian Jackson [Sun, 22 Dec 2019 18:27:10 +0000 (18:27 +0000)]
doctests: Provide extract-doctest

This is not called yet.  There are a few things that need fixing
first.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Do not use => inappropriately
Ian Jackson [Sun, 22 Dec 2019 18:25:08 +0000 (18:25 +0000)]
README: Do not use => inappropriately

Our notation is that => introduces the ouput string.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Use &\ for escaping; abolish \&
Ian Jackson [Mon, 9 Dec 2019 14:49:43 +0000 (14:49 +0000)]
Syntax: Use &\ for escaping; abolish \&

"\&" causes lots of trouble, including difficulty specifying certain
resulting strings.

Previously I proposed a scheme using &$@ but this is only useful to
provide a way to write &s.

The $-doubling regime which will come later wants a way to produce a
single literal $, so introduce that now using the same syntax.
And &\ feels quite natural to me.

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Introduce &. and friends
Ian Jackson [Tue, 10 Dec 2019 01:21:43 +0000 (01:21 +0000)]
Syntax: Introduce &. and friends

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Provide $var_prefix_name to process_input_mk
Ian Jackson [Wed, 25 Dec 2019 01:09:18 +0000 (01:09 +0000)]
generate: Provide $var_prefix_name to process_input_mk

Nothing uses this yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoSyntax: Introduce &:changequote
Ian Jackson [Wed, 11 Dec 2019 13:24:47 +0000 (13:24 +0000)]
Syntax: Introduce &:changequote

Incompatible change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Track output Makefiles properly
Ian Jackson [Tue, 24 Dec 2019 00:28:30 +0000 (00:28 +0000)]
generate: Track output Makefiles properly

This fixes a bug where the intermediate directory Makefile was not
included whe it should be.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter: Add .../sub/dir
Ian Jackson [Tue, 24 Dec 2019 00:18:46 +0000 (00:18 +0000)]
tests/filter: Add .../sub/dir

This will let us test output which depends on the subdirectory,
which is kind of the whole point...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests: Shuffle, make tests/check run all the tests
Ian Jackson [Wed, 25 Dec 2019 17:28:34 +0000 (17:28 +0000)]
tests: Shuffle, make tests/check run all the tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests/filter: New test framework for generate
Ian Jackson [Sat, 21 Dec 2019 11:40:54 +0000 (11:40 +0000)]
tests/filter: New test framework for generate

This test case bypasses autoconf and runs generate directly.
It also gives us somewhere to write examples of each of the
&-expansions without further cluttering test-example.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Handle nodes with no own Subdir.sd.mk
Ian Jackson [Sun, 22 Dec 2019 19:52:10 +0000 (19:52 +0000)]
generate: Handle nodes with no own Subdir.sd.mk

If a directory is not mentioned on the command line, we would assume
it had a Subdir.sd.mk anyway.  Instead, expect it not to have one.

We do still generate a per-subdirectory fragment for such unmentioned
nodes: this serves to wire its children ino the parent, make running
`make' in that directory possible, etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agogenerate: Add missing final newline to main.mk
Ian Jackson [Sat, 21 Dec 2019 11:34:12 +0000 (11:34 +0000)]
generate: Add missing final newline to main.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agotests: Add some section comments.
Ian Jackson [Sat, 21 Dec 2019 11:17:41 +0000 (11:17 +0000)]
tests: Add some section comments.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoexample: Add a comment next to for-test bits
Ian Jackson [Sat, 21 Dec 2019 11:15:23 +0000 (11:15 +0000)]
example: Add a comment next to for-test bits

This may help unconfuse people using this as an example.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoREADME: Fix a mistake in the file reference table
Ian Jackson [Tue, 17 Dec 2019 22:19:37 +0000 (22:19 +0000)]
README: Fix a mistake in the file reference table

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agoMerge from secnet
Ian Jackson [Sat, 28 Dec 2019 22:18:19 +0000 (22:18 +0000)]
Merge from secnet

5 months ago&TARGETS*: make autogenerated targets .PHONY
Ian Jackson [Sun, 8 Dec 2019 23:12:25 +0000 (23:12 +0000)]
&TARGETS*: make autogenerated targets .PHONY

Suggested-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodocs: Minor fix to README
Ian Jackson [Sun, 8 Dec 2019 18:06:50 +0000 (18:06 +0000)]
docs: Minor fix to README

Reported-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agodocs: Minor fixes to regen.mk.in comment
Ian Jackson [Sun, 8 Dec 2019 18:06:37 +0000 (18:06 +0000)]
docs: Minor fixes to regen.mk.in comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 months agosubdirmk: Suppress MAKEFILE_TEMPLATES when running `make clean'
Ian Jackson [Sat, 7 Dec 2019 15:28:29 +0000 (15:28 +0000)]
subdirmk: Suppress MAKEFILE_TEMPLATES when running `make clean'

See the README hunk for the reasoning.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>