`subdirmk' is an attempt to solve these problems (and it also slightly
alleviates some of the boilerplate needed to support out-of-tree
-builds well).
+builds well, and helps a bit with metaprogramming and rule writing).
Basic approach
--------------
The developer is expected to write a makefile fragment, in each
-relevant subdirectory, called `Subdir.sd.mk'.
+relevant subdirectory, called `Dir.sd.mk'.
These fragments may contain ordinary make language. Unqualified
filenames are relative to the build toplevel, and all commands all run
`the build directory corresponding to this .sd.mk file', etc.
There are a variety of convenient constructions.
-The result is that to a large extent, the Subdir.sd.mk has an easy way
+The result is that to a large extent, the Dir.sd.mk has an easy way
to namespace its "local" make variables, and an easy way to refer to
its "local" filenames (and filenames in general).
-The Subdir.sd.mk's are filtered, fed through autoconf in the usual way
+The Dir.sd.mk's are filtered, fed through autoconf in the usual way
(for @..@-substitutions) and included by one autogenerated toplevel
makefile.
which exists purely to capture ordinary make invocations and arrange
for something suitable to happen.
-Where there are dependencies between subdirectories, each Subdir.sd.mk
+Where there are dependencies between subdirectories, each Dir.sd.mk
can simply refer to files in other subdirectories directly.
-Invocation, "recursive" per-directory targets
----------------------------------------------
-
-Arrangements are made so that when you run `make foo' in a
-subdirectory, it is like running the whole toplevel makefile, from the
-toplevel, as `make subdir/foo'. If `subdir/foo' is a file that might
-be built, that builds it.
-
-But `foo' can also be a conventional target like `all'.
-
-Each subdirectory has its own `all' target. For example a
-subdirectory `src' has a target `src/all'. The rules for these are
-automatically generated from the settings of the per-directory
-&TARGETS variables. &TARGETS is magic in this way. (In
-src/Subdir.sd.mk, &TARGETS of course refers to a make variable called
-src_TARGETS.)
-
-The `all' target in a parent directory is taken to imply the `all'
-targets in all of its subdirectories, recursively. And in the
-autogenerated stub Makefiles, `all' is the default target. So if you
-just type `make' in the toplevel, you are asking for `&all'
-(<subdir>/all) for every directory in the project.
-
-In a parallel build, the rules for all these various subdirectory
-targets may be in run in parallel: there is only one `make' invocation
-at a time. There is no sequencing between subdirectories, only been
-individual targets (as specified according to their dependencies).
-
-You can define other per-directory recursive targets too: set the
-variable &TARGETS_zonk, or whatever (being sure to write &TARGETS_zonk
-at the start of a line). This will create a src/zonk target (for
-appropriate value of src/). Unlike `all', these other targets only
-exist in areas of the project where at least something mentions them.
-So for example, if &TARGETS_zonk is set in src but not lib, `make
-zonk' in lib will fail. If you want to make a target exist
-everywhere, += it with nothing in Prefix.sd.mk or Suffix.sd.mk (see
-below).
-
-Prefix.sd.mk, Suffix.sd.mk, Final.sd.mk, inclusion
---------------------------------------------------
-
-The files Prefix.sd.mk and Suffix.sd.mk in the toplevel of the source
-are automatically processed before and after each individual
-directory's Subdir.sd.mk, and the &-substituted contents therefore
-appear once for each subdirectory.
-
-This lets you do per-directory boilerplate. Some useful boilerplate
-is already provided in subdirmk, for you to reference like this:
- &:include subdirmk/cdeps.sd.mk
- &:include subdirmk/clean.sd.mk
-For example you could put that in Suffix.sd.mk.
-
-The top-level Subdir.sd.mk is the first makefile included after the
-autogenerated `main.mk' which merely has some basic settings and
-includes. So if you want to get in early and set global variables,
-put them near the top of Subdir.sd.mk.
-
-The file Final.sd.mk in the toplevel directory is processed and
-included after all the other files.
-
-subdirmk's filter script itself sets (only) these variables:
- top_srcdir
- abs_top_srcdir
- SUBDIRMK_MAKEFILES
- MAKEFILE_TEMPLATES
-You are likely to want to define $(PWD), and shorter names for
-top_srdir and abs_top_srcdir (we suggest $(src) and $(abs_src)).
-
-Global definitions
-------------------
-
-If want to set global variables, such as CC, that should only be done
-once. You can put them in your top-level Subdir.sd.mk, or a separate
-file you `include' and declare using SUBDIRMK_MAKEFILES.
-
-If you need different settings of variables like CC for different
-subdirectories, you should probably do that with target-specific
-variable settings. See the info node `(make) Target-specific'.
-
-Subdirectory templates `.sd.mk' vs plain autoconf templates `.mk.in'
---------------------------------------------------------------------
-
-There are two kinds of template files.
-
- Filename .sd.mk .mk.in
-
- Processed by &-substitution, autoconf only
- then autoconf
-
- Instantiated Usu. once per subdir Once only
-
- Need to be mentioned No, but Subdir.sd.mk All not in subdirmk/
- in configure.ac? via SUBDIRMK_SUBDIRS via SUBDIRMK_MAKEFILES
-
- How to include `&:include foo.sd.mk' `include foo.mk'
- in all relevant .sd.mk in only one
- (but not needed for Subdir.sd.mk
- Prefix, Suffix, Final)
-
-If you `include subdirmk/regen.mk', dependency management and
-automatic regeneration for all of this template substitution, and for
-config.status etc. is done for you.
-
-Tables of file reference syntaxes
----------------------------------
-
-In a nonrecursive makefile supporting out of tree builds there are
-three separate important distinctions between different file
-locations:
-
- (i) In the build tree, or in the source tree ?
-
- (ii) In (or relative to) the subdirectory to which this Subdir.sd.mk
- relates, or relative to the project's top level ?
-
- (iii) Absolute or relative pathname ? Usually relative pathnames
- suffice. Where an absolute pathname is needed, it can be built
- out of &/ and an appropriate make variable such as $(PWD).
-
-Path construction &-expansions are built from the following:
-
- Relative paths in...
- build source
-
- This directory & &^
- Top level . &~
-
-In more detail, with all the various options laid out:
-
- Recommended Relative paths in... Absolute paths in...
- for build source build source
-
- This lc &file &^file $(PWD)/&file $(abs_src)/&file
- directory any &/file &^/file $(PWD)/&/file $(abs_src)/&/file
- several & f g h &^ f g h $(addprefix...)
-
- Top lc file &~file
- level any file &~/file $(PWD)/file $(abs_src)/file
- .mk.in file $(src)/file $(PWD)/file $(abs_src)/file
- several f g h &~ f g h $(addprefix...)
-
-(This assumes you have appropriate make variables src, PWD and
-abs_src.)
-
Substitution syntax
-------------------
In general & expands to the subdirectory name when used for a
filename, and to the subdirectory name with / replaced with _ for
-variable names.
+variable names. (If your variables start with capital letters and
+your filenames with lowercase. Otherwise, use &/ or &_.)
Note that & is processed *even in makefile comments*. The substitutor
does not understand make syntax, or shell syntax, at all. However,
&= => sub_dir or TOP
-&^lc => $(top_srcdir)/sub/dir/lc
-&^/ => $(top_srcdir)/sub/dir/
-&^. => $(top_srcdir)/sub/dir
+&^lc => ${top_srcdir}/sub/dir/lc
+&^/ => ${top_srcdir}/sub/dir/
+&^. => ${top_srcdir}/sub/dir
-&~lc => $(top_srcdir)/lc
-&~/ => $(top_srcdir)/
-&~. => $(top_srcdir)
+&~lc => ${top_srcdir}/lc
+&~/ => ${top_srcdir}/
+&~. => ${top_srcdir}
In general:
^ pathname of this subdirectory in source tree
~ pathname of top level of source tree
/ terminates the path escape } needed if next is
- _ terminates the var escape } not lwsp or space)
+ _ terminates the var escape } not letter or space)
. terminates path escape giving dir name (excluding /)
= terminates var escape giving only prefix part (rarely needed)
lwsp starts multi-word processing (see below)
So pathname syntax is a subset of:
- '&' [ '^' | '~' ] [ lc | '/' | '.' | '=' ]
+ '&' [ '^' | '~' ] [ lc | '/' | '.' ]
&& => && for convenience in shell runes
&\$ => $ provided for $-doubling regimes
&\NEWLINE eats the newline and vanishes
-&$VARIABLE => $(sub_dir_VARIABLE) or $(TOP_VARIABLE)
+&$VARIABLE => ${sub_dir_VARIABLE} or ${TOP_VARIABLE}
VARIABLE is ASCII starting with a letter and matching \w+
& thing thing... &
&:include filename filename should usually be [&]foo.sd.mk
&:-include filename tolerate nonexistent file
- filenames are relative to $(top_srcdir)
- RHS is &-expanded
+ RHS is &-expanded but filenames are relative to the top
+ srcdir. This implies that unqualified names are like &~/
+ whereas &/ is like &^/. &^ and &~ do not work here because
+ they expand to constructions involving literally
+ `$(top_srcdir)', but the RHS is not make-expanded.
&!<lwsp> disables & until EOL (and then disappears)
&# delete everything to end of line
(useful if the RHS contains unrecognised & constructions)
-&:changequote NEWQUOTE
- changes the escape sequence from & to literally NEWQUOTE
- NEWQUOTE may be any series of of non-whitespace characters,
- and is terminated by EOL or lwsp. The whole line is
- discarded.
-
- After this, write NEWQUOTE instead of &, everywhere.
- The effect is global and lasts until the next setting.
- It takes effect on &:include'd files too, so maybe set
- it back before using &:include.
-
- Notably
- NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE
- NEWQUOTE\NEWQUOTE => NEWQUOTE
- NEWQUOTE\$ => $
- NEWQUOTE:changequote & set escape back to &
-
&TARGETS_things
Handled specially. If mentioned at the start of a line
(possibly following whitespace), declares that this
`all' is extra special: every directory has an `all'
target, which corresponds to &TARGETS.
+&:warn [!]WARNTAG ...
+ Suppress (with !) or re-enable (without !) warnings tagged
+ WARNTAG (see section `Warnings', below). The suppression list
+ is reset at the start of processing in each subdirectory.
+ Warnings that appear at the end of processing are controlled
+ by the final warning state after processing all the toplevel
+ input files (including Final.sd.mk).
+
+&:local+global [&]VARIABLE ...
+ Suppresses the warning about seeing both VARIABLE and
+ &VARIABLE. Any & specified in the RHS is redundant: this
+ always affects both versions identically.
+
+&:changequote NEWQUOTE
+ changes the escape sequence from & to literally NEWQUOTE
+ NEWQUOTE may be any series of of non-whitespace characters,
+ and is terminated by EOL or lwsp. The whole line is
+ discarded.
+
+ After this, write NEWQUOTE instead of &, everywhere.
+ The effect is unscoped and lasts until the next setting,
+ or until the end of the current directory's Suffix.sd.mk.
+ It takes effect on &:include'd files too, so maybe set
+ it back before using &:include.
+
+ Notably
+ NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE
+ NEWQUOTE\NEWQUOTE => NEWQUOTE
+ NEWQUOTE\$ => $
+ NEWQUOTE:changequote & set escape back to &
+
Dollar doubling and macro assistance
------------------------------------
&$- Stops dollar-doubling
Both are idempotent and local to the file or context.
+This is useful both for make macrology involving $(eval ...), and
+possibly for helping write complicated recipes involving shell
+variables, inline Perl code, etc.
+
Sometimes we will show $'s being doubled inside another construct.
This means the content of the construct is $-doubled: $-doubling is
locally enabled, and restored afterwards.
&${..$..} => ${eval ${call ..$$..}}
(matches { } pairs to find the end)
- content is $-doubled (unless it contains $- to turn that off)
+ content is $-doubled (unless it contains &$- to turn that off)
- Together &:macro and &${...} provide a more reasonable macro
- facility than raw make. They solve the problem that make
- expansions cannot directly generate multiple rules, variable,
- etc.; instead, `$(eval )' must be used, but that re-expands
- the argument, meaning that all the literal text must be
- $-doubled. This applies to the macro text and to the
- arguments. Also `$(eval $(call ...))' is an unfortunate syntax.
- Hence &:macro and &${...}.
+Together &:macro and &${...} provide a more reasonable macro facility
+than raw make. They solve the problem that make expansions cannot
+directly generate multiple rules, variable, etc.; instead, `$(eval )'
+must be used, but that re-expands the argument, meaning that all the
+literal text must be $-doubled. This applies to the macro text and to
+the arguments. Also `$(eval $(call ...))' is an unfortunate syntax.
+Hence &:macro and &${...}.
While dollar-doubling:
- - - - - - - - - - -
&-expansions not mentioned here
&\$ => $
-&$NN => $(NN) where N are digits
-&$( => $(
+&$( => ${ (expands to { } so it is useable for shell too)
+&$NN => ${NN} where N are digits
+A few contexts do not support $-doubling, such as directive arguments
+or places where this might imply $-quadrupling. (There is no way to
+get $-quadrupling.)
-Subdirectory and variable naming
---------------------------------
-The simple variable decoration scheme does not enforce a strict
-namespace distinction between parts of variable names which come from
-subdirectory names, and parts that mean something else.
+Invocation, "recursive" per-directory targets
+---------------------------------------------
-So it is a good idea to be a bit careful with your directory naming.
-`TOP', names that contain `_', and names that are similar to parts of
-make variables (whether conventional ones, or ones used in your
-project) are best avoided.
+Arrangements are made so that when you run `make foo' in a
+subdirectory, it is like running the whole toplevel makefile, from the
+toplevel, as `make subdir/foo'. If `subdir/foo' is a file that might
+be built, that builds it.
-If you name your variables in ALL CAPS and your subdirectories in
-lower case with `-' rather than `_', there will be no confusion.
+But `foo' can also be a conventional target like `all'.
+
+Each subdirectory has its own `all' target. For example a
+subdirectory `src' has a target `src/all'. The rules for these are
+automatically generated from the settings of the per-directory
+&TARGETS variables. &TARGETS is magic in this way. (In
+src/Dir.sd.mk, &TARGETS of course refers to a make variable called
+src_TARGETS.)
+
+The `all' target in a parent directory is taken to imply the `all'
+targets in all of its subdirectories, recursively. And in the
+autogenerated stub Makefiles, `all' is the default target. So if you
+just type `make' in the toplevel, you are asking for `&all'
+(<subdir>/all) for every directory in the project.
+
+In a parallel build, the rules for all these various subdirectory
+targets may be in run in parallel: there is only one `make' invocation
+at a time. There is no sequencing between subdirectories, only been
+individual targets (as specified according to their dependencies).
+
+You can define other per-directory recursive targets too: set the
+variable &TARGETS_zonk, or whatever (being sure to write &TARGETS_zonk
+at the start of a line). This will create a src/zonk target (for
+appropriate value of src/). Unlike `all', these other targets only
+exist in areas of the project where at least something mentions them.
+So for example, if &TARGETS_zonk is set in src but not lib, `make
+zonk' in lib will fail. If you want to make a target exist
+everywhere, += it with nothing in Prefix.sd.mk or Suffix.sd.mk (see
+below).
+
+Prefix.sd.mk, Suffix.sd.mk, Final.sd.mk, inclusion
+--------------------------------------------------
+
+The files Prefix.sd.mk and Suffix.sd.mk in the toplevel of the source
+are automatically processed before and after each individual
+directory's Dir.sd.mk, and the &-substituted contents therefore
+appear once for each subdirectory.
+
+This lets you do per-directory boilerplate. Some useful boilerplate
+is already provided in subdirmk, for you to reference like this:
+ &:include subdirmk/cdeps.sd.mk
+ &:include subdirmk/clean.sd.mk
+For example you could put that in Suffix.sd.mk.
+
+The top-level Dir.sd.mk is the first makefile included after the
+autogenerated `main.mk' which merely has some basic settings and
+includes. So if you want to get in early and set global variables,
+put them near the top of Dir.sd.mk.
+
+The file Final.sd.mk in the toplevel directory is processed and
+the result included after all the other files. Its subdirmk
+filtering context inherits warning suppressions from the toplevel's
+Dir.sd.mk etc., but not anything else.
+
+subdirmk's filter script itself sets (only) these variables:
+ top_srcdir
+ abs_top_srcdir
+ SUBDIRMK_MAKEFILES
+ MAKEFILE_TEMPLATES
+You are likely to want to define $(PWD), and shorter names for
+top_srdir and abs_top_srcdir (we suggest $(src) and $(abs_src)).
+
+Warnings
+--------
+
+subdirmk's `generate' program, which does the acual &-substitution,
+can produce some warnings about your .sd.mk files. These can be
+suppressed with the &:warn directive. The warning tags are:
+
+ local+global
+ The same VARNAME was used both with and without an & prefix.
+ This can be confusing. Also, if you avoid this then you will
+ get a warning iff you accidentally leave off a needed &.
+ The generation of this warning depends on scanning your
+ makefile for things that look like variable references, which
+ subdirmk does not do completely perfectly. Exciting make
+ syntax may evade this warning, or require suppressions.
+ (You can suppress this warning for a particular VARNAME with
+ the &:local+global directive.)
+
+ single-char-var
+ A variable expansion like $FBAR. make's expansion rules
+ interpret this as $(F)BAR. It's normally better to write
+ it this way, at least if the variable expansion is followed
+ by more letters. Note that &$FOO works differently to
+ raw make: it expands to ${sub_dir_FOO}.
+
+ broken-var-ref
+ An attempt at variable expansion looking like $&...
+ You probably expected this to mean $(TOP_F)BAR but it
+ expands to $TOP_FBAR which make thinks means $(T)OP_FBAR.
+
+ unknown-warning
+ &:warn was used to try to enable a warning that this version
+ of subdirmk does not understand. (Note that an attempt to
+ *dis*able an unknown warning is only reported if some other
+ warning was issued which might have been disabled.)
+
+
+Guides, hints, and further explanations
+=======================================
Incorporating this into your project
------------------------------------
Use `git-subtree' to merge the subdirmk/ directory. You may find it
useful to symlink the DEVELOPER-CERTIFICATE file (git can store
symlinks as symlinks - just `git add' the link). And you probably
-want to mention the situation in your top-level COPYING.
+want to mention the situation in your top-level COPYING and HACKING.
Symlink autogen.sh into your project toplevel.
m4_include([subdirmk/subdirmk.ac])
SUBDIRMK_SUBDIRS([...list of subdirectories in relative syntax...])
-Write a Subdir.sd.mk in each directory. The toplevel one should
-probably contain:
+Write a Dir.sd.mk in each directory. See the substitution syntax
+reference, above, and the example/ directory here. The toplevel
+Dir.sd.mk should probably contain:
include subdirmk/usual.mk
include subdirmk/regen.mk
&:include subdirmk/cdeps.sd.mk
&:include subdirmk/clean.sd.mk
+
Hints
-----
You can convert your project incrementally. Start with the top-level
-Makefile.in and rename it to Subdir.sd.mk, and add the appropriate
+Makefile.in and rename it to Dir.sd.mk, and add the appropriate
stuff to configure.ac, and fix everything up. Leave the existing
$(MAKE) -C for your existing subdirectories alone. Then you can
convert individual subdirectories, or classes of subdirectories, at
-your leisure. (You must be /sure/ that each subdirectory will be
-entered only once at a time, but your existing recursive make descent
-system should already do that or you already have concurrency bugs.)
+your leisure. (You must be /sure/ that each recursive (non-subdirmk)
+subdirectory will be entered only once at a time, but your existing
+recursive make descent system should already do that or you already
+have concurrency bugs.)
Aside from this, be very wary of any invocation of $(MAKE) anywhere.
This is a frequent source of concurrency bugs in recursive make build
will clear up the problem.
+Global definitions
+------------------
+
+If want to set global variables, such as CC, that should only be done
+once. You can put them in your top-level Dir.sd.mk, or a separate
+file you `include' and declare using SUBDIRMK_MAKEFILES.
+
+If you need different settings of variables like CC for different
+subdirectories, you should probably do that with target-specific
+variable settings. See the info node `(make) Target-specific'.
+
+
+Directory templates `.sd.mk' vs plain autoconf templates `.mk.in'
+--------------------------------------------------------------------
+
+There are two kinds of template files.
+
+ Filename .sd.mk .mk.in
+
+ Processed by &-substitution, autoconf only
+ then autoconf
+
+ Instantiated Usu. once per subdir Once only
+
+ Need to be mentioned No, but Dir.sd.mk All not in subdirmk/
+ in configure.ac? via SUBDIRMK_SUBDIRS via SUBDIRMK_MAKEFILES
+
+ How to include `&:include foo.sd.mk' `include foo.mk'
+ in all relevant .sd.mk in only one
+ (but not needed for Dir.sd.mk
+ Prefix, Suffix, Final)
+
+If you `include subdirmk/regen.mk', dependency management and
+automatic regeneration for all of this template substitution, and for
+config.status etc. is done for you.
+
+
+Tables of file reference syntaxes
+---------------------------------
+
+In a nonrecursive makefile supporting out of tree builds there are
+three separate important distinctions between different file
+locations:
+
+ (i) In the build tree, or in the source tree ?
+
+ (ii) In (or relative to) the subdirectory to which this Dir.sd.mk
+ relates, or relative to the project's top level ?
+
+ (iii) Absolute or relative pathname ? Usually relative pathnames
+ suffice. Where an absolute pathname is needed, it can be built
+ out of &/ and an appropriate make variable such as $(PWD).
+
+Path construction &-expansions are built from the following:
+
+ Relative paths in...
+ build source
+
+ This directory & &^
+ Top level . &~
+
+In more detail, with all the various options laid out:
+
+ Recommended Relative paths in... Absolute paths in...
+ for build source build source
+
+ This lc &file &^file $(PWD)/&file $(abs_src)/&file
+ directory any &/file &^/file $(PWD)/&/file $(abs_src)/&/file
+ several & f g h &^ f g h $(addprefix...)
+
+ Top lc file &~file
+ level any file &~/file $(PWD)/file $(abs_src)/file
+ .mk.in file $(src)/file $(PWD)/file $(abs_src)/file
+ several f g h &~ f g h $(addprefix...)
+
+(This assumes you have appropriate make variables src, PWD and
+abs_src.)
+
+
+Subdirectory and variable naming
+--------------------------------
+
+The simple variable decoration scheme does not enforce a strict
+namespace distinction between parts of variable names which come from
+subdirectory names, and parts that mean something else.
+
+So it is a good idea to be a bit careful with your directory naming.
+`TOP', names that contain `_', and names that are similar to parts of
+make variables (whether conventional ones, or ones used in your
+project) are best avoided.
+
+If you name your variables in ALL CAPS and your subdirectories in
+lower case with `-' rather than `_', there will be no confusion.
+
+
Legal information
------------------
+=================
subdirmk is
Copyright 2019 Mark Wooding
~ian / subdirmk.git / blobdiff