chiark / gitweb /
subdirmk.git
10 days agoREADME: Update copyright date master subdirmk/0.4
Ian Jackson [Sun, 16 Feb 2020 18:39:51 +0000 (18:39 +0000)]
README: Update copyright date

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
10 days agoREADME: Fix documentation errors relating to &{ etc.
Ian Jackson [Sun, 16 Feb 2020 17:29:52 +0000 (17:29 +0000)]
README: Fix documentation errors relating to &{ etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
4 weeks agosyntax: Support &: literal (for make `grouped' multiple targets)
Ian Jackson [Sun, 26 Jan 2020 12:37:14 +0000 (12:37 +0000)]
syntax: Support &: literal (for make `grouped' multiple targets)

Very recent versions of GNU make support this syntax:

  foo bar biz &: baz boz
        echo $^ > foo
        echo $^ > bar
        echo $^ > biz

(See (make)Multiple Targets, subheading `Rules with Grouped Targets')

This use of & is nonoverlapping with everything we care about, so we
can just pass it straight through.  Passing through &: means we pass
through &:: too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
4 weeks agoREADME: Add subheading `Directives'
Ian Jackson [Sun, 26 Jan 2020 12:36:53 +0000 (12:36 +0000)]
README: Add subheading `Directives'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agoSyntax: Incompatible change: &$( does not do daft { thing
Ian Jackson [Sat, 11 Jan 2020 14:13:03 +0000 (14:13 +0000)]
Syntax: Incompatible change: &$( does not do daft { thing

It is silly for &$( to expand to ${.  That requires the input to
contain non-matched kinds of brackets.

The shell compatibility is not needed.  In a shell rune, you can write
&\$thing since the { } are not normally needed there at all.  If they
are then now &\${thing} is necessary.  It would be possible to steal
"&${" (or allow only "&$(") but that seems less good.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agoSyntax: Provide convenience syntax &( for $(eval
Ian Jackson [Sat, 11 Jan 2020 13:58:34 +0000 (13:58 +0000)]
Syntax: Provide convenience syntax &( for $(eval

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agotests/filter/extract-doctests: Use $withspcs more
Ian Jackson [Sat, 11 Jan 2020 13:51:21 +0000 (13:51 +0000)]
tests/filter/extract-doctests: Use $withspcs more

This does away with a special case and will allow general use of
spaces in before and after expansions.

No change with current input.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agotests/filter/extract-doctests: Improve an error message
Ian Jackson [Sat, 11 Jan 2020 13:50:32 +0000 (13:50 +0000)]
tests/filter/extract-doctests: Improve an error message

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agotests/filter/extract-doctests: Recognise directives first
Ian Jackson [Sat, 11 Jan 2020 13:49:01 +0000 (13:49 +0000)]
tests/filter/extract-doctests: Recognise directives first

We are going to expand the rules for => expansions and they would
match directives too, but we don't want them to.

No functional change with existing input.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agoSyntax: Incompatible change: Use &{ not &${ for macros
Ian Jackson [Sat, 11 Jan 2020 13:26:26 +0000 (13:26 +0000)]
Syntax: Incompatible change: Use &{ not &${ for macros

This will be more orthogonal with $( which will come in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
6 weeks agoREADME: Adjust indent of &${..$..} expansion
Ian Jackson [Sat, 11 Jan 2020 13:25:42 +0000 (13:25 +0000)]
README: Adjust indent of &${..$..} expansion

No change to the meaning.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests: Provide make-release script subdirmk/0.3
Ian Jackson [Mon, 30 Dec 2019 12:59:57 +0000 (12:59 +0000)]
tests: Provide make-release script

Not really `tests' but I don't want to add a whole new directory just
for this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests: Provide advance-tested script
Ian Jackson [Mon, 30 Dec 2019 12:50:30 +0000 (12:50 +0000)]
tests: Provide advance-tested script

Useful for our own testing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoLegal: Update tests/filter/ inputs and outputs
Ian Jackson [Mon, 30 Dec 2019 11:58:11 +0000 (11:58 +0000)]
Legal: Update tests/filter/ inputs and outputs

We judiciously use # vs. &# for the legal comment so that each
expected output file contains one notice.

The following files, which are wholly constructed by running
generate and reviewing the diffs, do not have a legal notice:
  tests/filter/main.mk.expected
  tests/filter/stderr.expected
  tests/filter/sub/Dir.mk.expected
This is OK I think.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoLegal: Add copyright, licence and warranty notice to many files
Ian Jackson [Mon, 30 Dec 2019 11:17:17 +0000 (11:17 +0000)]
Legal: Add copyright, licence and warranty notice to many files

The .sd.mk and .expected files in tests/filter/ are more complicated.
We'll do those in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoLegal: Add NO WARRANTY everywhere
Ian Jackson [Mon, 30 Dec 2019 11:09:36 +0000 (11:09 +0000)]
Legal: Add NO WARRANTY everywhere

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Avoid $err_file undefined warning during startup
Ian Jackson [Mon, 30 Dec 2019 11:02:59 +0000 (11:02 +0000)]
generate: Avoid $err_file undefined warning during startup

If, for example, one of the input files implied by the command line
arguments is missing, err is called with $err_file not yet set.

Handle this case.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Mention suppressions when warning re VAR vs &VAR
Ian Jackson [Sun, 29 Dec 2019 15:56:44 +0000 (15:56 +0000)]
generate: Mention suppressions when warning re VAR vs &VAR

When we are actually printing the warning, report all the locations,
including the suppressed ones.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoSyntax: Support &:local+global !...
Ian Jackson [Sun, 29 Dec 2019 15:50:28 +0000 (15:50 +0000)]
Syntax: Support &:local+global !...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter: Test &:local+global & vs non-&
Ian Jackson [Sun, 29 Dec 2019 15:46:54 +0000 (15:46 +0000)]
tests/filter: Test &:local+global & vs non-&

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoSyntax: Rescope effect of &:local+global
Ian Jackson [Sun, 29 Dec 2019 15:18:39 +0000 (15:18 +0000)]
Syntax: Rescope effect of &:local+global

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agosubdirmk: Fix a Subdir left over in README
Ian Jackson [Sun, 29 Dec 2019 14:32:49 +0000 (14:32 +0000)]
subdirmk: Fix a Subdir left over in README

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoSyntax: Expand &$( and &$NN to use ${ } rather than $( )
Ian Jackson [Sun, 29 Dec 2019 14:17:00 +0000 (14:17 +0000)]
Syntax: Expand &$( and &$NN to use ${ } rather than $( )

And now document the use of $&+ for recipes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agodoctests: Allow parenthetical comments as an expansion RHS
Ian Jackson [Sun, 29 Dec 2019 14:18:49 +0000 (14:18 +0000)]
doctests: Allow parenthetical comments as an expansion RHS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Swap order of &$( and &$NN
Ian Jackson [Sun, 29 Dec 2019 14:15:15 +0000 (14:15 +0000)]
README: Swap order of &$( and &$NN

We are going to add a note to &$( which applies to &$NN too, and this
makes it nicer.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoSyntax: Use ${top_srcdir} rather than $(top_srcdir)
Ian Jackson [Sun, 29 Dec 2019 14:13:52 +0000 (14:13 +0000)]
Syntax: Use ${top_srcdir} rather than $(top_srcdir)

Again, this makes it possible to use in $-doubled shell runes as well
as in make syntax.  (Assuming you have made top_srcdir be a shell
variable as well as a make variable.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoSyntax: expand &$... to ${....} rather than $(....)
Ian Jackson [Sun, 29 Dec 2019 14:03:28 +0000 (14:03 +0000)]
Syntax: expand &$... to ${....} rather than $(....)

This makes it possible to use in $-doubled shell runes as well as in
make syntax.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Introduce new `broken-var-ref' warning
Ian Jackson [Sun, 29 Dec 2019 14:01:22 +0000 (14:01 +0000)]
Warnings: Introduce new `broken-var-ref' warning

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter: Print all the diffs, not just the first
Ian Jackson [Sun, 29 Dec 2019 14:00:58 +0000 (14:00 +0000)]
tests/filter: Print all the diffs, not just the first

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Refer to example/
Ian Jackson [Sat, 28 Dec 2019 19:07:17 +0000 (19:07 +0000)]
README: Refer to example/

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Move the "how to use" up
Ian Jackson [Sat, 28 Dec 2019 19:06:17 +0000 (19:06 +0000)]
README: Move the "how to use" up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Add another layer of structure
Ian Jackson [Sat, 28 Dec 2019 19:04:26 +0000 (19:04 +0000)]
README: Add another layer of structure

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Move the Warnings section up into what is going to be spec
Ian Jackson [Sat, 28 Dec 2019 19:04:10 +0000 (19:04 +0000)]
README: Move the Warnings section up into what is going to be spec

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: deinent the macro explaation
Ian Jackson [Sat, 28 Dec 2019 18:55:45 +0000 (18:55 +0000)]
README: deinent the macro explaation

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Further miscellaneous fixes and clarifications
Ian Jackson [Sat, 28 Dec 2019 18:55:25 +0000 (18:55 +0000)]
README: Further miscellaneous fixes and clarifications

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Document that local+global is not 100% accurate
Ian Jackson [Sat, 28 Dec 2019 18:46:58 +0000 (18:46 +0000)]
README: Document that local+global is not 100% accurate

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: More syntax further up
Ian Jackson [Sat, 28 Dec 2019 18:43:18 +0000 (18:43 +0000)]
README: More syntax further up

This is primary.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarning reporting: Report each warning only once
Ian Jackson [Sat, 28 Dec 2019 18:36:31 +0000 (18:36 +0000)]
Warning reporting: Report each warning only once

In particular, Prefix and Suffix, or conventional &:include's can
generate a lot of repeated warnings.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarning reporting: Provide &:local+global directive
Ian Jackson [Sat, 28 Dec 2019 18:26:09 +0000 (18:26 +0000)]
Warning reporting: Provide &:local+global directive

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarning reporting: Add a test of the warning suppression system
Ian Jackson [Sat, 28 Dec 2019 13:55:59 +0000 (13:55 +0000)]
Warning reporting: Add a test of the warning suppression system

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarning reporting: Warning suppression system
Ian Jackson [Sat, 28 Dec 2019 13:55:12 +0000 (13:55 +0000)]
Warning reporting: Warning suppression system

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotest/filter; Cause some warnings, to check they appear
Ian Jackson [Sat, 28 Dec 2019 12:58:03 +0000 (12:58 +0000)]
test/filter; Cause some warnings, to check they appear

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Sort occurrences properly in local+global warnings
Ian Jackson [Sat, 28 Dec 2019 12:52:01 +0000 (12:52 +0000)]
Warnings: Sort occurrences properly in local+global warnings

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Warn for confusing single-char $ expansions
Ian Jackson [Sat, 28 Dec 2019 14:05:19 +0000 (14:05 +0000)]
Warnings: Warn for confusing single-char $ expansions

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Track variable references in &-expansions
Ian Jackson [Sat, 28 Dec 2019 12:31:40 +0000 (12:31 +0000)]
Warnings: Track variable references in &-expansions

For &-escapes which match the whole variable name, we can conveniently
track variable expansions as part of the processing.  (We don't change
any of the matching regexps.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Track basic $-references
Ian Jackson [Sat, 28 Dec 2019 12:31:49 +0000 (12:31 +0000)]
Warnings: Track basic $-references

We must always stop at $'s in the input now.

Having stopped at $, we normally just output it and carry on.  Ie, we
only stop so we can do some inspection: as before, we do this variable
tracking as inspection before processing, rather entangled with
processing.

We can deal reasonably properly with ${ } and $( ).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Track settings of variables
Ian Jackson [Sat, 28 Dec 2019 12:27:40 +0000 (12:27 +0000)]
Warnings: Track settings of variables

Where the variable tracking does not involve &-escapes, or only
involves &-escapes which work like a prefix so the processor does not
have the whole variable name, we do this separately from processing.

This is simpler because we can look ahead more.  It also avoids
disturbing the processing logic (which needs to be precisely accurate,
unlike this variable tracking).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: Infrastructure for tracking and warning about variables
Ian Jackson [Sat, 28 Dec 2019 12:59:11 +0000 (12:59 +0000)]
Warnings: Infrastructure for tracking and warning about variables

We are going to track when we see FOO or &FOO.  If we see both, we
issue a warning, as that might mean the programmer forgot a &.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoWarnings: generate: Basic `wrn' function
Ian Jackson [Sat, 28 Dec 2019 12:58:46 +0000 (12:58 +0000)]
Warnings: generate: Basic `wrn' function

This is sufficient to add warnning calls to the code.  We'll soup it
up later.  No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter/check: Capture generate's stderr
Ian Jackson [Sat, 28 Dec 2019 12:55:17 +0000 (12:55 +0000)]
tests/filter/check: Capture generate's stderr

We are going to cause `generate' to emit warnings, which we want to
check are working.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotest/filter: Test &${ ... } rather better
Ian Jackson [Sat, 28 Dec 2019 23:07:01 +0000 (23:07 +0000)]
test/filter: Test &${ ... } rather better

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Fix &${ } end condition
Ian Jackson [Sat, 28 Dec 2019 23:01:49 +0000 (23:01 +0000)]
generate: Fix &${ } end condition

It is quite wrong to use $ddbl, which might be adjusted by &$- &$+.
We must maintain a separate counter.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Fix { } in &${ to actually output the { }
Ian Jackson [Sat, 28 Dec 2019 12:17:14 +0000 (12:17 +0000)]
generate: Fix { } in &${ to actually output the { }

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter: Rename `doctest' -> `doctests' everywhere
Ian Jackson [Sat, 28 Dec 2019 11:34:56 +0000 (11:34 +0000)]
tests/filter: Rename `doctest' -> `doctests' everywhere

This is more uniform.  A lot of churn, though.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Move &:changequote further down
Ian Jackson [Sat, 28 Dec 2019 18:19:17 +0000 (18:19 +0000)]
README: Move &:changequote further down

This is a bit of a minority interest.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Explain a wrinkle in &:include filename semantics
Ian Jackson [Sat, 28 Dec 2019 00:24:15 +0000 (00:24 +0000)]
README: Explain a wrinkle in &:include filename semantics

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: use oraw in a few more places
Ian Jackson [Thu, 26 Dec 2019 02:10:03 +0000 (02:10 +0000)]
generate: use oraw in a few more places

This gets rid of some open-coded `print O'.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Miscellaneous fixes and clarifications
Ian Jackson [Thu, 26 Dec 2019 01:49:49 +0000 (01:49 +0000)]
README: Miscellaneous fixes and clarifications

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Document restriction on $-doubling and lack of -quadrupling
Ian Jackson [Thu, 26 Dec 2019 02:03:07 +0000 (02:03 +0000)]
README: Document restriction on $-doubling and lack of -quadrupling

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: Move `Tables of file reference syntaxes' down
Ian Jackson [Thu, 26 Dec 2019 01:48:38 +0000 (01:48 +0000)]
README: Move `Tables of file reference syntaxes' down

This is more of an appendix or addendum.  It should come after the
complete description of the substitution syntax.

Pure motion.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoBig incompatible change: Rename `Subdir' to `Dir'
Ian Jackson [Thu, 26 Dec 2019 01:40:09 +0000 (01:40 +0000)]
Big incompatible change: Rename `Subdir' to `Dir'

We want completion to be easy, and we have Suffix now.  `Dir' is as
good as `Subdir' I think (and a bit shorter).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Fix messages resulting from bad directive arguments
Ian Jackson [Thu, 26 Dec 2019 01:32:43 +0000 (01:32 +0000)]
Error handling: Fix messages resulting from bad directive arguments

"Unknown directive" is not necessarily accurate; another possibility
is that the directive is known but didn't match the parsing regexp
because of problems with the arguments.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Nested scope: Change `Eval' to `eval'
Ian Jackson [Thu, 26 Dec 2019 01:33:01 +0000 (01:33 +0000)]
generate: Nested scope: Change `Eval' to `eval'

This is the nesting kind name.  But it also appears in error messages
where it ought not to be capitalised.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Nested scope: Change `Macro' to `macro'
Ian Jackson [Thu, 26 Dec 2019 01:32:36 +0000 (01:32 +0000)]
generate: Nested scope: Change `Macro' to `macro'

This is the nesting kind name.  But it also appears in error messages
where it ought not to be capitalised.

We'll change `Eval' in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Trim the `bad escape' message.e
Ian Jackson [Sat, 28 Dec 2019 00:28:03 +0000 (00:28 +0000)]
Error handling: Trim the `bad escape' message.e

Do not print the whole line, only the next 5 chars (say) - and, also,
not any newlines.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Better reporting of nest-related errors
Ian Jackson [Sat, 28 Dec 2019 01:04:43 +0000 (01:04 +0000)]
Error handling: Better reporting of nest-related errors

Track `$what' which is a kind of example for the error message.
Use it in error messages.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Often say `subdirmk' rather than `$0'
Ian Jackson [Sat, 28 Dec 2019 00:27:58 +0000 (00:27 +0000)]
Error handling: Often say `subdirmk' rather than `$0'

The latter is generally a path (`.../generate') and may be confusing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Replace or update many calls to `die'
Ian Jackson [Sat, 28 Dec 2019 00:27:13 +0000 (00:27 +0000)]
Error handling: Replace or update many calls to `die'

To conform to new approach.

The `ending wrong kind of nest' error will be dealt with in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoError handling: Provide err and ddbl_only, and an internal spec
Ian Jackson [Sat, 28 Dec 2019 00:26:16 +0000 (00:26 +0000)]
Error handling: Provide err and ddbl_only, and an internal spec

No callers of these yet.  The spec is honoured in the breach.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Make @nest always be nonempty
Ian Jackson [Thu, 26 Dec 2019 01:04:58 +0000 (01:04 +0000)]
generate: Make @nest always be nonempty

No outward functional change, but makes the error handling
etc. slightly simpler.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Do not tolerate unclosed nested scopes at EOF
Ian Jackson [Sat, 28 Dec 2019 00:51:43 +0000 (00:51 +0000)]
generate: Do not tolerate unclosed nested scopes at EOF

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoMacro assistance part 2 - syntax for $(eval $(call...))
Ian Jackson [Tue, 24 Dec 2019 23:04:58 +0000 (23:04 +0000)]
Macro assistance part 2 - syntax for $(eval $(call...))

This can be used to call a macro defined with &:macro.  In combination
these provide a more reasonable macro facility than make's.

Specifically:

* Document the new facility.
* Motivate it in the README.
* Provide the new @nest kind for &${  }, which uses $ddbl as
  a counter to count { and }.
* Implement the {}-counting and nesting end in the main parsing
  loop.  (We must now search for things besides $esc.)
* In extract-doctest, provide a slightly adhoc special case for
  understanding ${eval ...} as an expansion text in the README.
  And do not attempt to test &${ } when already $-doubling.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoMacro assistance part 1 - macro directive
Ian Jackson [Sat, 28 Dec 2019 00:36:50 +0000 (00:36 +0000)]
Macro assistance part 1 - macro directive

Define the &:macro directive, which is a dollar-doubled version of
make's own `define'.

We must introuce a new concept of `nesting', for when "the content of
the construct is $-doubled" in the words of the README.  The
implementation is in the main process_input_mk function: it is a
multi-line scope of some kind with an ad-hoc ending condition, and its
own dollar-doubling setting.

We will also want a convenient syntax for $(eval $(call...)), which
we're going to introduce in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agodoctests: Rename `desc' etc. in extract-doctest from `rubric'
Ian Jackson [Fri, 27 Dec 2019 18:26:11 +0000 (18:26 +0000)]
doctests: Rename `desc' etc. in extract-doctest from `rubric'

This isn't a rubric, it is a description of a particular test entry.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agodoctests: Improve entry description filter
Ian Jackson [Tue, 24 Dec 2019 16:05:49 +0000 (16:05 +0000)]
doctests: Improve entry description filter

This makes the decriptive text more aposite in some situations.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoDollar doubling feature
Ian Jackson [Wed, 25 Dec 2019 23:29:40 +0000 (23:29 +0000)]
Dollar doubling feature

This makes it easier to write certain rules commands, and macros
involving eval.  Further enhancements will follow, but for now:

* Document the new facility, including the &-escapes to turn it on and
  off, its effect, and the three within-doubling don't-double escapes.

* Implementation.  Specifically:
   - Rename od to oud (`undoubled').
   - Provide od which does double things if $ddbl (a global) is set.
   - Implement the escapes, and add some error checks.
   - Change `od' to `oud' in `&\$', so that that always produces an
     undoubled $.

* Enhance extract-doctest:
   - Set $e->{DD} to say if it's in `while dollar doubling'.
   - Filter out such entries from the `normal' test - they are not
     legal outside dollar-doubling.
   - Add a dollar-doubling test which includes versions of all
     the normal tests as well as the dollar-doubling ones, and
     which dollar-doubles the outputs when appropriate.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter/.gitignore: Ignore all .tmp
Ian Jackson [Thu, 26 Dec 2019 00:26:55 +0000 (00:26 +0000)]
tests/filter/.gitignore: Ignore all .tmp

Rather than trying to list them all.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Introduce oraw
Ian Jackson [Wed, 25 Dec 2019 23:45:32 +0000 (23:45 +0000)]
generate: Introduce oraw

Replace all other calls to `o' with `oraw'.  This ensures we
have no stray calls.

It elminates an anomaly, namely that calls outside process_input_mk
would rely on the $buffering_output variable which is set only there.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Introduce od
Ian Jackson [Wed, 25 Dec 2019 23:31:13 +0000 (23:31 +0000)]
generate: Introduce od

Replace every call to `o' in process_input_mk (the main filtering
routine) with a call to `od', which is going to be the dollar-doubled
output variant when we add the dollar doubling feature.

For now it just does the same as `o'.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoexample: Introduce a Final.sd.mk and test it
Ian Jackson [Sat, 28 Dec 2019 22:53:47 +0000 (22:53 +0000)]
example: Introduce a Final.sd.mk and test it

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/filter: Test Final.sd.mk
Ian Jackson [Wed, 25 Dec 2019 20:08:53 +0000 (20:08 +0000)]
tests/filter: Test Final.sd.mk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoInterface: Read Final.sd.mk
Ian Jackson [Wed, 25 Dec 2019 19:14:57 +0000 (19:14 +0000)]
Interface: Read Final.sd.mk

Tests will come in a moment, although we already see the new output
file mentioned in main.mk.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoREADME: fix TARGETS text for new tidier semantics
Ian Jackson [Thu, 26 Dec 2019 01:43:50 +0000 (01:43 +0000)]
README: fix TARGETS text for new tidier semantics

In
  Syntax: Refine &TARGETS_things, mostly to avoid # wrinkle
we updated the spec for &TARGETS_things in the list of expansions,
but not in the discussion of per-directory recursive targets.

Fix that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoInterface: Rename Perdir to Suffix; provide Prefix
Ian Jackson [Sat, 28 Dec 2019 22:45:50 +0000 (22:45 +0000)]
Interface: Rename Perdir to Suffix; provide Prefix

We want to be able to specify something that appears at the top of
every per-directory segment, principally to provide
for-every-directory variable settings.

A more sophisticated scheme akin to m4 diversions would be possible in
principle, but it would make organising the output files (which must
be processed by autoconf) really awkward - let alone the depenency
management for regenerating the makefiles.

So instead, simply provide a Prefix too.  Additionally, soon we are
going to introduce a Final.sd.mk which is processed right at the end.

I think this new scheme will be sufficient in principle to do anything
that might be needed.  Sadly it will continue to require that
&:include's are done in the right order.

"Suffix" and "Subdir" tab-complete really badly.  In a forthcoming
commit we will rename Subdir.sd.mk to Dir.sd.mk.

Incompatible change.  We must update the tests right away.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Make many dir vars globals
Ian Jackson [Wed, 25 Dec 2019 19:35:55 +0000 (19:35 +0000)]
generate: Make many dir vars globals

This makes the code a bit more fragile but a lot less repetitious.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Fix an erroneous comment
Ian Jackson [Wed, 25 Dec 2019 20:02:48 +0000 (20:02 +0000)]
generate: Fix an erroneous comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agogenerate: Drop comment about input files
Ian Jackson [Wed, 25 Dec 2019 19:19:04 +0000 (19:19 +0000)]
generate: Drop comment about input files

This is all documented in README now.
And it's about to become out of date.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/check: Dynamic parallism choice
Ian Jackson [Sun, 29 Dec 2019 13:29:18 +0000 (13:29 +0000)]
tests/check: Dynamic parallism choice

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/check: Show parallel make rune we run
Ian Jackson [Sun, 29 Dec 2019 13:29:05 +0000 (13:29 +0000)]
tests/check: Show parallel make rune we run

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests: Run in parallel
Ian Jackson [Sun, 29 Dec 2019 13:25:42 +0000 (13:25 +0000)]
tests: Run in parallel

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/example: Reset times of (copied) source files too
Ian Jackson [Sun, 29 Dec 2019 13:07:53 +0000 (13:07 +0000)]
tests/example: Reset times of (copied) source files too

This eliminates a problem with our test case: if the build takes less
than 1s the files were probably copied more recently than the 1s ago,
resulting in too much rebuilding (and possibly missing
failed-to-rebuild bugs).

We can now change the timestamp for the build tree to something
earlier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/example: Build in a copy
Ian Jackson [Sun, 29 Dec 2019 13:05:18 +0000 (13:05 +0000)]
tests/example: Build in a copy

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/build-common: Copy subdirmk, rather than symlinking
Ian Jackson [Sun, 29 Dec 2019 13:02:34 +0000 (13:02 +0000)]
tests/build-common: Copy subdirmk, rather than symlinking

This is more like it will be in someone's project.  Also it prevents
us from accidentally touching anything in the source tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/intree: Make copy and break out from tests/example
Ian Jackson [Sun, 29 Dec 2019 12:59:14 +0000 (12:59 +0000)]
tests/intree: Make copy and break out from tests/example

We introduce make_copy, which makes a copy of tests/example
suitable for testing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agobuild tests: Break out build-common
Ian Jackson [Sun, 29 Dec 2019 12:48:55 +0000 (12:48 +0000)]
build tests: Break out build-common

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoregen.mk.in: Clarify documentation about realclean
Ian Jackson [Sun, 29 Dec 2019 12:34:39 +0000 (12:34 +0000)]
regen.mk.in: Clarify documentation about realclean

It's not sensible to `make [the provided] realclean:: into a recursive
target' in that sense since the one we provide is effective, and
present, only in the toplevel.  What we mean is that you can make your
own realclean target and regen.mk's will fit in nicely.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agoregen.mk.in: Document CONFIG_STATUS_OUTPUTS
Ian Jackson [Sun, 29 Dec 2019 11:00:47 +0000 (11:00 +0000)]
regen.mk.in: Document CONFIG_STATUS_OUTPUTS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotest/example/check: Reset times for regen tests
Ian Jackson [Sun, 29 Dec 2019 12:27:00 +0000 (12:27 +0000)]
test/example/check: Reset times for regen tests

In principle, on a filesystem with poor time resolution, the updates
to the input files might have the same timestamp as the output files.
Things would then not be regenerated.

Work around this by resetting all timestamps in the build tree to
1s ago.  This should ensure that exactly the one file whose edit we
are simulating seems, to make, to have been changed.

There is a remaining possible problem: if our last actual change to
the subdirmk source tree is less than 1s ago, this might rebuild too
much.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotest/example/check: Make sections appear in stderr output
Ian Jackson [Sun, 29 Dec 2019 12:26:25 +0000 (12:26 +0000)]
test/example/check: Make sections appear in stderr output

This makes the output easier to navigate.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
8 weeks agotests/example/check: Better handling of makefile updates
Ian Jackson [Sun, 29 Dec 2019 11:01:28 +0000 (11:01 +0000)]
tests/example/check: Better handling of makefile updates

Rather than checking just for presence of a string in a templated
output file, also have the input file define a target which we try to
run.  This checks that make actually reread the file.

Also add ||false to the grep lines, which makes the stderr output
more obviously contain a failure when the test fails.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>