`subdirmk' is an attempt to solve these problems (and it also slightly
alleviates some of the boilerplate needed to support out-of-tree
-builds well).
+builds well, and helps a bit with metaprogramming and rule writing).
Basic approach
--------------
The developer is expected to write a makefile fragment, in each
-relevant subdirectory, called `Subdir.sd.mk'.
+relevant subdirectory, called `Dir.sd.mk'.
-These fragments may contain ordinary make language.
+These fragments may contain ordinary make language. Unqualified
+filenames are relative to the build toplevel, and all commands all run
+there.
However, the sigil & is treated specially. By and large, it refers to
-`the current directory'. There are a variety of convenient
-constructions.
+`the build directory corresponding to this .sd.mk file', etc.
+There are a variety of convenient constructions.
-The result is that to a large extent, the Subdir.sd.mk has an easy way
+The result is that to a large extent, the Dir.sd.mk has an easy way
to namespace its "local" make variables, and an easy way to refer to
-its "local" filenames.
+its "local" filenames (and filenames in general).
-The Subdir.sd.mk's are filtered, fed through autoconf in the usual way
+The Dir.sd.mk's are filtered, fed through autoconf in the usual way
(for @..@-substitutions) and included by one autogenerated toplevel
makefile.
which exists purely to capture ordinary make invocations and arrange
for something suitable to happen.
-Where there are dependencies between subdirectories, each Subdir.sd.mk
+Where there are dependencies between subdirectories, each Dir.sd.mk
can simply refer to files in other subdirectories directly.
Invocation, "recursive" per-directory targets
subdirectory `src' has a target `src/all'. The rules for these are
automatically generated from the settings of the per-directory
&TARGETS variables. &TARGETS is magic in this way. (In
-src/Subdir.sd.mk, &TARGETS of course refers to a make variable called
+src/Dir.sd.mk, &TARGETS of course refers to a make variable called
src_TARGETS.)
The `all' target in a parent directory is taken to imply the `all'
at a time. There is no sequencing between subdirectories, only been
individual targets (as specified according to their dependencies).
-You can define other per-directory recursive targets too: simply
-mention (usually, by setting) the variable &TARGETS_zonk, or whatever.
-This will create a src/zonk target (for appropriate value of src/).
-Unlike `all', these other targets only exist in areas of the project
-where at least something mentions them. So for example, if
-&TARGETS_zonk is mentioned in src but not lib, `make zonk' in
-lib will fail. If you want to make a target exist everywhere,
-mention its name in Perdir.sd.mk (see below).
+You can define other per-directory recursive targets too: set the
+variable &TARGETS_zonk, or whatever (being sure to write &TARGETS_zonk
+at the start of a line). This will create a src/zonk target (for
+appropriate value of src/). Unlike `all', these other targets only
+exist in areas of the project where at least something mentions them.
+So for example, if &TARGETS_zonk is set in src but not lib, `make
+zonk' in lib will fail. If you want to make a target exist
+everywhere, += it with nothing in Prefix.sd.mk or Suffix.sd.mk (see
+below).
-Perdir.sd.mk, inclusion
------------------------
+Prefix.sd.mk, Suffix.sd.mk, Final.sd.mk, inclusion
+--------------------------------------------------
-The file Perdir.sd.mk in the toplevel of the source is automatically
-processed after each individual directory's Subdir.sd.mk, and the
-&-substituted contents therefore appear once for each subdirectory.
+The files Prefix.sd.mk and Suffix.sd.mk in the toplevel of the source
+are automatically processed before and after each individual
+directory's Dir.sd.mk, and the &-substituted contents therefore
+appear once for each subdirectory.
This lets you do per-directory boilerplate. Some useful boilerplate
is already provided in subdirmk, for you to reference like this:
&:include subdirmk/cdeps.sd.mk
&:include subdirmk/clean.sd.mk
-For example you could put that in Perdir.sd.mk.
+For example you could put that in Suffix.sd.mk.
+
+The top-level Dir.sd.mk is the first makefile included after the
+autogenerated `main.mk' which merely has some basic settings and
+includes. So if you want to get in early and set global variables,
+put them near the top of Dir.sd.mk.
+
+The file Final.sd.mk in the toplevel directory is processed and
+included after all the other files.
+
+subdirmk's filter script itself sets (only) these variables:
+ top_srcdir
+ abs_top_srcdir
+ SUBDIRMK_MAKEFILES
+ MAKEFILE_TEMPLATES
+You are likely to want to define $(PWD), and shorter names for
+top_srdir and abs_top_srcdir (we suggest $(src) and $(abs_src)).
Global definitions
------------------
If want to set global variables, such as CC, that should only be done
-once. You can put them in your top-level Subdir.sd.mk, or a separate
+once. You can put them in your top-level Dir.sd.mk, or a separate
file you `include' and declare using SUBDIRMK_MAKEFILES.
If you need different settings of variables like CC for different
subdirectories, you should probably do that with target-specific
variable settings. See the info node `(make) Target-specific'.
-Subdirectory templates `.sd.mk' vs plain autoconf templates `.mk.in'
+Directory templates `.sd.mk' vs plain autoconf templates `.mk.in'
--------------------------------------------------------------------
There are two kinds of template files.
Instantiated Usu. once per subdir Once only
- Need to be mentioned No, but Subdir.sd.mk All not in subdirmk/
+ Need to be mentioned No, but Dir.sd.mk All not in subdirmk/
in configure.ac? via SUBDIRMK_SUBDIRS via SUBDIRMK_MAKEFILES
How to include `&:include foo.sd.mk' `include foo.mk'
in all relevant .sd.mk in only one
- (but not needed for Subdir.sd.mk
- Subdir and Perdir)
+ (but not needed for Dir.sd.mk
+ Prefix, Suffix, Final)
If you `include subdirmk/regen.mk', dependency management and
automatic regeneration for all of this template substitution, and for
config.status etc. is done for you.
-Summary of recommended directory reference syntaxes
----------------------------------------------------
-
-Path construction &-expansions, meanings summary:
-
- In build tree In source tree
- This directory just & &,
- Top level &. implies absolute &;
-
-Adding `@' means "absolute path". This is not needed with &. because
-there is never any need to use &. since it would expand to nothing.
-`/' terminates the escape (needed if the next thing is not a lowercase
-character, or space). `=' means "just the value, no /". Space starts
-multi-word processing.
-
- Recommended In build tree In source tree
- when Relative Absolute Relative Absolute
-
- This lc &file &@file &,file &@,file
- directory any &/file &@/file &,/file &@,/file
- several & f g h &@ f g h &, f g h &@, f g h
-
- Top lc &.file &;file &@;file
- level any file &./file &;/file &@;/file
- several f g h &. f g h &; f g h &@; f g h
- .mk.in file $(abs)/file $(src)/file $(abs_src)/file
-
Substitution syntax
-------------------
In general & expands to the subdirectory name when used for a
filename, and to the subdirectory name with / replaced with _ for
-variable names.
+variable names. (If your variables start with capital letters and
+your filenames with lowercase. Otherwise, use &/ or &_.)
Note that & is processed *even in makefile comments*. The substitutor
does not understand make syntax, or shell syntax, at all. However,
The assumption is that filenames are usually lowercase and
variables usually uppercase. Otherwise, use another syntax:
+&/ => sub/dir/ or nothing
&_ => sub_dir_ or TOP_
-&=_ => sub_dir or TOP
+&. => sub/dir or .
+ (This implies that `&./' works roughly like `&/', although
+ it can produce a needless `./')
-&/ => sub/dir/ or nothing
-&=/ => sub/dir or .
+&= => sub_dir or TOP
-&,lc => $(top_srcdir)/sub/dir/lc &,/ => $(top_srcdir)/sub/dir/
-&;lc => $(top_srcdir)/lc &;/ => $(top_srcdir)/
+&^lc => $(top_srcdir)/sub/dir/lc
+&^/ => $(top_srcdir)/sub/dir/
+&^. => $(top_srcdir)/sub/dir
-&@lc => $(PWD)/sub/dir/lc &@/ => $(PWD)/sub/dir/
-&.lc => $(PWD)/lc &./ => $(PWD)/
-&@,lc => $(abs_top_srcdir)/sub/dir/lc &@,/ => $(abs_top_srcdir)/sub/dir/
-&@;lc => $(abs_top_srcdir)/lc &@;/ => $(abs_top_srcdir)/
+&~lc => $(top_srcdir)/lc
+&~/ => $(top_srcdir)/
+&~. => $(top_srcdir)
In general:
- = return subdir without delimiter (not allowed with `,' `;' `@')
- , pathname of this subdirectory in source tree
- ; pathname of top level of source tree
- . pathname of this directory in build tree, implies absolute pathnames
- @ absolute pathnames (forbidden with `.', must come first)
+ ^ pathname of this subdirectory in source tree
+ ~ pathname of top level of source tree
+ / terminates the path escape } needed if next is
+ _ terminates the var escape } not letter or space)
+ . terminates path escape giving dir name (excluding /)
+ = terminates var escape giving only prefix part (rarely needed)
+ lwsp starts multi-word processing (see below)
So pathname syntax is a subset of:
- '&' [ '@' ] [ ',' | ';' | '.' ] [ lc | '/' ]
-
- To avoid incomprehensible .sd.mk files, some combinations are not
- allowed. For example `&@=./' would mean `$(PWD)/sub/dir' but can
- be spelled `$(PWD)/&=/', but more normally the trailing / can be
- tolerated, so use `&@/'.
+ '&' [ '^' | '~' ] [ lc | '/' | '.' ]
&& => && for convenience in shell runes
-\& => & general escaping mechanism
-& thing thing... & &@ thing thing... &
- &. thing thing... &
-&, thing thing... & &@, thing thing... &
-&; thing thing... & &@; thing thing... &
+&\& => & general escaping mechanism
+&\$ => $ provided for $-doubling regimes
+&\NEWLINE eats the newline and vanishes
+
+&$VARIABLE => $(sub_dir_VARIABLE) or $(TOP_VARIABLE)
+ VARIABLE is ASCII starting with a letter and matching \w+
+
+& thing thing... &
+&^ thing thing... &
+&~ thing thing... &
Convenience syntax for prefixing multiple filenames.
Introduced by & followed by lwsp where lc could go.
Each lwsp-separated non-ws word is prefixed by &/ etc.
&:<directive> <args>....
recognised at start of line only (possibly after lwsp)
- args are processed for &
-&:include filename filename should usually be foo.sd.mk
+&:include filename filename should usually be [&]foo.sd.mk
&:-include filename tolerate nonexistent file
- filenames are relative to $(top_srcdir)
+ RHS is &-expanded but filenames are relative to the top
+ srcdir. This implies that unqualified names are like &~/
+ whereas &/ is like &^/. &^ and &~ do not work here because
+ they expand to constructions involving literally
+ `$(top_srcdir)', but the RHS is not make-expanded.
&!<lwsp> disables & until EOL (and then disappears)
&# delete everything to end of line
(useful if the RHS contains unrecognised & constructions)
-&!STUFF
- changes the escape sequence from & to literally STUFF
- STUFF may be any series of of non-whitespace characters,
- and is terminated by EOL or lwsp. &!STUFF and the lwsp
- are discarded.
-
- After this, write STUFF instead of &, everywhere.
- The effect is global and lasts until the next setting.
- It takes effect on &:include'd files too, so maybe set
- it back before using &:include.
-
- Notably
- STUFFSTUFF => STUFFSTUFF
- \STUFF => STUFF
- STUFF!& set escape back to &
-
&TARGETS_things
- Handled specially. If mentioned, declares that this
+ Handled specially. If mentioned at the start of a line
+ (possibly following whitespace), declares that this
subdir ought to have a target `things'. The rule will be
&/things:: $(&TARGETS_things)
which recursively implies this directory's `things'.
Must be spelled exactly &TARGETS_things. &_TARGETS_things,
- for example, is not magic. But mentioning &TARGETS_things in
- a #-comment *does* work because the & filter does not care
- about comments.
+ for example, is not magic. To make the target exist
+ without providing any prerequisites for it, write a line
+ containing just `&TARGETS_things +='.
`all' is extra special: every directory has an `all'
target, which corresponds to &TARGETS.
+&:changequote NEWQUOTE
+ changes the escape sequence from & to literally NEWQUOTE
+ NEWQUOTE may be any series of of non-whitespace characters,
+ and is terminated by EOL or lwsp. The whole line is
+ discarded.
+
+ After this, write NEWQUOTE instead of &, everywhere.
+ The effect is unscoped and lasts until the next setting,
+ or until the end of the current directory's Suffix.sd.mk.
+ It takes effect on &:include'd files too, so maybe set
+ it back before using &:include.
+
+ Notably
+ NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE
+ NEWQUOTE\NEWQUOTE => NEWQUOTE
+ NEWQUOTE\$ => $
+ NEWQUOTE:changequote & set escape back to &
+
+
+Dollar doubling and macro assistance
+------------------------------------
+
+&$+ Starts dollar-doubling
+&$- Stops dollar-doubling
+ Both are idempotent and local to the file or context.
+
+Sometimes we will show $'s being doubled inside another construct.
+This means the content of the construct is $-doubled: $-doubling is
+locally enabled, and restored afterwards.
+
+&:macro NAME => define NAME
+STUFF $ THINGS .. STUFF $$ THINGS
+&:endm .. endef
+ NAME is processed for &
+
+&${..$..} => ${eval ${call ..$$..}}
+ (matches { } pairs to find the end)
+ content is $-doubled (unless it contains $- to turn that off)
+
+ Together &:macro and &${...} provide a more reasonable macro
+ facility than raw make. They solve the problem that make
+ expansions cannot directly generate multiple rules, variable,
+ etc.; instead, `$(eval )' must be used, but that re-expands
+ the argument, meaning that all the literal text must be
+ $-doubled. This applies to the macro text and to the
+ arguments. Also `$(eval $(call ...))' is an unfortunate syntax.
+ Hence &:macro and &${...}.
+
+While dollar-doubling:
+- - - - - - - - - - -
+
+$ => $$ including $'s produced by other
+ &-expansions not mentioned here
+
+&\$ => $
+&$NN => $(NN) where N are digits
+&$( => $(
+
+A few contexts do not support $-doubling, such as directive arguments
+or places where this might imply $-quadrupling. (There is no way to
+get $-quadrupling.)
+
+
+Tables of file reference syntaxes
+---------------------------------
+
+In a nonrecursive makefile supporting out of tree builds there are
+three separate important distinctions between different file
+locations:
+
+ (i) In the build tree, or in the source tree ?
+
+ (ii) In (or relative to) the subdirectory to which this Dir.sd.mk
+ relates, or relative to the project's top level ?
+
+ (iii) Absolute or relative pathname ? Usually relative pathnames
+ suffice. Where an absolute pathname is needed, it can be built
+ out of &/ and an appropriate make variable such as $(PWD).
+
+Path construction &-expansions are built from the following:
+
+ Relative paths in...
+ build source
+
+ This directory & &^
+ Top level . &~
+
+In more detail, with all the various options laid out:
+
+ Recommended Relative paths in... Absolute paths in...
+ for build source build source
+
+ This lc &file &^file $(PWD)/&file $(abs_src)/&file
+ directory any &/file &^/file $(PWD)/&/file $(abs_src)/&/file
+ several & f g h &^ f g h $(addprefix...)
+
+ Top lc file &~file
+ level any file &~/file $(PWD)/file $(abs_src)/file
+ .mk.in file $(src)/file $(PWD)/file $(abs_src)/file
+ several f g h &~ f g h $(addprefix...)
+
+(This assumes you have appropriate make variables src, PWD and
+abs_src.)
+
Subdirectory and variable naming
--------------------------------
Use `git-subtree' to merge the subdirmk/ directory. You may find it
useful to symlink the DEVELOPER-CERTIFICATE file (git can store
symlinks as symlinks - just `git add' the link). And you probably
-want to mention the situation in your top-level COPYING.
+want to mention the situation in your top-level COPYING and HACKING.
Symlink autogen.sh into your project toplevel.
m4_include([subdirmk/subdirmk.ac])
SUBDIRMK_SUBDIRS([...list of subdirectories in relative syntax...])
-Write a Subdir.sd.mk in each directory. The toplevel one should
+Write a Dir.sd.mk in each directory. The toplevel one should
probably contain:
include subdirmk/usual.mk
include subdirmk/regen.mk
-Write a Perdir.sd.mk in the toplevel, if you want. It should probably
+Write a Suffix.sd.mk in the toplevel, if you want. It should probably
have:
&:include subdirmk/cdeps.sd.mk
-----
You can convert your project incrementally. Start with the top-level
-Makefile.in and rename it to Subdir.sd.mk, and add the appropriate
+Makefile.in and rename it to Dir.sd.mk, and add the appropriate
stuff to configure.ac, and fix everything up. Leave the existing
$(MAKE) -C for your existing subdirectories alone. Then you can
convert individual subdirectories, or classes of subdirectories, at
-your leisure. (You must be /sure/ that each subdirectory will be
-entered only once at a time, but your existing recursive make descent
-system should already do that or you already have concurrency bugs.)
+your leisure. (You must be /sure/ that each recursive (non-subdirmk)
+subdirectory will be entered only once at a time, but your existing
+recursive make descent system should already do that or you already
+have concurrency bugs.)
Aside from this, be very wary of any invocation of $(MAKE) anywhere.
This is a frequent source of concurrency bugs in recursive make build
far enough to regenerate a working set of makefiles. If this happens
just rerun ./config.status by hand.
+If you go back and forth between different versions of your code you
+can sometimes find that `make' complains that one of your Subdir.sd.mk
+files is missing: typically, if iot was used and therefore a
+dependency in some other version of your code. If you run `make
+clean' (or `make realclean') these dependencies are suppressed, which
+will clear up the problem.
+
Legal information
-----------------