X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~ian/git?p=subdirmk.git;a=blobdiff_plain;f=README;h=41110ce410ac8ca098cff5e28cbd182951fcc4ee;hp=518dd087d853308b13e07aab805349714288d26e;hb=08e825fede28f22ce3fbaf494c3e06360ed74d7a;hpb=d817b49007e60b0e7392f23caf08dc0328eb32de diff --git a/README b/README index 518dd08..41110ce 100644 --- a/README +++ b/README @@ -20,13 +20,13 @@ style is not very ergonomic. The main difficulties are: `subdirmk' is an attempt to solve these problems (and it also slightly alleviates some of the boilerplate needed to support out-of-tree -builds well). +builds well, and helps a bit with metaprogramming and rule writing). Basic approach -------------- The developer is expected to write a makefile fragment, in each -relevant subdirectory, called `Subdir.sd.mk'. +relevant subdirectory, called `Dir.sd.mk'. These fragments may contain ordinary make language. Unqualified filenames are relative to the build toplevel, and all commands all run @@ -36,11 +36,11 @@ However, the sigil & is treated specially. By and large, it refers to `the build directory corresponding to this .sd.mk file', etc. There are a variety of convenient constructions. -The result is that to a large extent, the Subdir.sd.mk has an easy way +The result is that to a large extent, the Dir.sd.mk has an easy way to namespace its "local" make variables, and an easy way to refer to its "local" filenames (and filenames in general). -The Subdir.sd.mk's are filtered, fed through autoconf in the usual way +The Dir.sd.mk's are filtered, fed through autoconf in the usual way (for @..@-substitutions) and included by one autogenerated toplevel makefile. @@ -52,7 +52,7 @@ Each subdirectory is also provided with an autogenerated `Makefile' which exists purely to capture ordinary make invocations and arrange for something suitable to happen. -Where there are dependencies between subdirectories, each Subdir.sd.mk +Where there are dependencies between subdirectories, each Dir.sd.mk can simply refer to files in other subdirectories directly. Invocation, "recursive" per-directory targets @@ -69,7 +69,7 @@ Each subdirectory has its own `all' target. For example a subdirectory `src' has a target `src/all'. The rules for these are automatically generated from the settings of the per-directory &TARGETS variables. &TARGETS is magic in this way. (In -src/Subdir.sd.mk, &TARGETS of course refers to a make variable called +src/Dir.sd.mk, &TARGETS of course refers to a make variable called src_TARGETS.) The `all' target in a parent directory is taken to imply the `all' @@ -83,32 +83,37 @@ targets may be in run in parallel: there is only one `make' invocation at a time. There is no sequencing between subdirectories, only been individual targets (as specified according to their dependencies). -You can define other per-directory recursive targets too: simply -mention (usually, by setting) the variable &TARGETS_zonk, or whatever. -This will create a src/zonk target (for appropriate value of src/). -Unlike `all', these other targets only exist in areas of the project -where at least something mentions them. So for example, if -&TARGETS_zonk is mentioned in src but not lib, `make zonk' in -lib will fail. If you want to make a target exist everywhere, -mention its name in Perdir.sd.mk (see below). +You can define other per-directory recursive targets too: set the +variable &TARGETS_zonk, or whatever (being sure to write &TARGETS_zonk +at the start of a line). This will create a src/zonk target (for +appropriate value of src/). Unlike `all', these other targets only +exist in areas of the project where at least something mentions them. +So for example, if &TARGETS_zonk is set in src but not lib, `make +zonk' in lib will fail. If you want to make a target exist +everywhere, += it with nothing in Prefix.sd.mk or Suffix.sd.mk (see +below). -Perdir.sd.mk, inclusion ------------------------ +Prefix.sd.mk, Suffix.sd.mk, Final.sd.mk, inclusion +-------------------------------------------------- -The file Perdir.sd.mk in the toplevel of the source is automatically -processed after each individual directory's Subdir.sd.mk, and the -&-substituted contents therefore appear once for each subdirectory. +The files Prefix.sd.mk and Suffix.sd.mk in the toplevel of the source +are automatically processed before and after each individual +directory's Dir.sd.mk, and the &-substituted contents therefore +appear once for each subdirectory. This lets you do per-directory boilerplate. Some useful boilerplate is already provided in subdirmk, for you to reference like this: &:include subdirmk/cdeps.sd.mk &:include subdirmk/clean.sd.mk -For example you could put that in Perdir.sd.mk. +For example you could put that in Suffix.sd.mk. -The top-level Subdir.sd.mk is the first makefile included after the +The top-level Dir.sd.mk is the first makefile included after the autogenerated `main.mk' which merely has some basic settings and includes. So if you want to get in early and set global variables, -put them near the top of Subdir.sd.mk. +put them near the top of Dir.sd.mk. + +The file Final.sd.mk in the toplevel directory is processed and +included after all the other files. subdirmk's filter script itself sets (only) these variables: top_srcdir @@ -122,14 +127,14 @@ Global definitions ------------------ If want to set global variables, such as CC, that should only be done -once. You can put them in your top-level Subdir.sd.mk, or a separate +once. You can put them in your top-level Dir.sd.mk, or a separate file you `include' and declare using SUBDIRMK_MAKEFILES. If you need different settings of variables like CC for different subdirectories, you should probably do that with target-specific variable settings. See the info node `(make) Target-specific'. -Subdirectory templates `.sd.mk' vs plain autoconf templates `.mk.in' +Directory templates `.sd.mk' vs plain autoconf templates `.mk.in' -------------------------------------------------------------------- There are two kinds of template files. @@ -141,65 +146,25 @@ There are two kinds of template files. Instantiated Usu. once per subdir Once only - Need to be mentioned No, but Subdir.sd.mk All not in subdirmk/ + Need to be mentioned No, but Dir.sd.mk All not in subdirmk/ in configure.ac? via SUBDIRMK_SUBDIRS via SUBDIRMK_MAKEFILES How to include `&:include foo.sd.mk' `include foo.mk' in all relevant .sd.mk in only one - (but not needed for Subdir.sd.mk - Subdir and Perdir) + (but not needed for Dir.sd.mk + Prefix, Suffix, Final) If you `include subdirmk/regen.mk', dependency management and automatic regeneration for all of this template substitution, and for config.status etc. is done for you. -Tables of file reference syntaxes ---------------------------------- - -In a nonrecursive makefile supporting out of tree builds there are -three separate important distinctions between different file -locations: - - (i) In the build tree, or in the source tree ? - - (ii) In (or relative to) the subdirectory to which this Subdir.sd.mk - relates, or relative to the project's top level ? - - (iii) Absolute or relative pathname ? Usually relative pathnames - suffice. Where an absolute pathname is needed, it can be built - out of &/ and an appropriate make variable such as $(PWD). - -Path construction &-expansions are built from the following: - - Relative paths in... - build source - - This directory & &^ - Top level . &~ - -In more detail, with all the various options laid out: - - Recommended Relative paths in... Absolute paths in... - for build source build source - - This lc &file &^file $(PWD)/&file $(abs_src)/&file - directory any &/file &^/file $(PWD)&/file $(abs_src)/&/file - several & f g h &^ f g h $(addprefix...) - - Top lc file &~file - level any file &~/file $(PWD)/file $(abs_src)/file - .mk.in file $(src)/file $(PWD)/file $(abs_src)/file - several f g h &~ f g h $(addprefix...) - -(This assumes you have appropriate make variables src, PWD and -abs_src.) - Substitution syntax ------------------- In general & expands to the subdirectory name when used for a filename, and to the subdirectory name with / replaced with _ for -variable names. +variable names. (If your variables start with capital letters and +your filenames with lowercase. Otherwise, use &/ or &_.) Note that & is processed *even in makefile comments*. The substitutor does not understand make syntax, or shell syntax, at all. However, @@ -218,30 +183,42 @@ empty string). The assumption is that filenames are usually lowercase and variables usually uppercase. Otherwise, use another syntax: +&/ => sub/dir/ or nothing &_ => sub_dir_ or TOP_ -&=_ => sub_dir or TOP +&. => sub/dir or . + (This implies that `&./' works roughly like `&/', although + it can produce a needless `./') -&/ => sub/dir/ or nothing -&=/ => sub/dir or . +&= => sub_dir or TOP &^lc => $(top_srcdir)/sub/dir/lc &^/ => $(top_srcdir)/sub/dir/ +&^. => $(top_srcdir)/sub/dir &~lc => $(top_srcdir)/lc &~/ => $(top_srcdir)/ +&~. => $(top_srcdir) In general: - = return subdir without delimiter (not allowed with `^' `~') ^ pathname of this subdirectory in source tree ~ pathname of top level of source tree - / terminates the escape (needed if next is not lwsp or space) + / terminates the path escape } needed if next is + _ terminates the var escape } not letter or space) + . terminates path escape giving dir name (excluding /) + = terminates var escape giving only prefix part (rarely needed) lwsp starts multi-word processing (see below) So pathname syntax is a subset of: - '&' [ '^' | '~' ] [ lc | '/' ] + '&' [ '^' | '~' ] [ lc | '/' | '.' ] && => && for convenience in shell runes -\& => & general escaping mechanism + +&\& => & general escaping mechanism +&\$ => $ provided for $-doubling regimes +&\NEWLINE eats the newline and vanishes + +&$VARIABLE => $(sub_dir_VARIABLE) or $(TOP_VARIABLE) + VARIABLE is ASCII starting with a letter and matching \w+ & thing thing... & &^ thing thing... & @@ -255,35 +232,23 @@ So pathname syntax is a subset of: &: .... recognised at start of line only (possibly after lwsp) - args are processed for & -&:include filename filename should usually be foo.sd.mk +&:include filename filename should usually be [&]foo.sd.mk &:-include filename tolerate nonexistent file - filenames are relative to $(top_srcdir) + RHS is &-expanded but filenames are relative to the top + srcdir. This implies that unqualified names are like &~/ + whereas &/ is like &^/. &^ and &~ do not work here because + they expand to constructions involving literally + `$(top_srcdir)', but the RHS is not make-expanded. &! disables & until EOL (and then disappears) &# delete everything to end of line (useful if the RHS contains unrecognised & constructions) -&!STUFF - changes the escape sequence from & to literally STUFF - STUFF may be any series of of non-whitespace characters, - and is terminated by EOL or lwsp. &!STUFF and the lwsp - are discarded. - - After this, write STUFF instead of &, everywhere. - The effect is global and lasts until the next setting. - It takes effect on &:include'd files too, so maybe set - it back before using &:include. - - Notably - STUFFSTUFF => STUFFSTUFF - \STUFF => STUFF - STUFF!& set escape back to & - &TARGETS_things - Handled specially. If mentioned, declares that this + Handled specially. If mentioned at the start of a line + (possibly following whitespace), declares that this subdir ought to have a target `things'. The rule will be &/things:: $(&TARGETS_things) @@ -297,13 +262,157 @@ So pathname syntax is a subset of: which recursively implies this directory's `things'. Must be spelled exactly &TARGETS_things. &_TARGETS_things, - for example, is not magic. But mentioning &TARGETS_things in - a #-comment *does* work because the & filter does not care - about comments. + for example, is not magic. To make the target exist + without providing any prerequisites for it, write a line + containing just `&TARGETS_things +='. `all' is extra special: every directory has an `all' target, which corresponds to &TARGETS. +&:warn [!]WARNTAG ... + Suppress (with !) or re-enable (without !) warnings tagged + WARNTAG (see section `Warnings', below). The suppression list + is reset at the start of processing in each subdirectory. + Warnings that appear at the end of processing are controlled + by the final warning state after processing all the toplevel + input files (including Final.sd.mk). + +&:local+global [&]VARIABLE ... + Suppresses the warning about seeing both VARIABLE and + &VARIABLE. Any & specified in the RHS is redundant: this + always affects both versions identically. + +&:changequote NEWQUOTE + changes the escape sequence from & to literally NEWQUOTE + NEWQUOTE may be any series of of non-whitespace characters, + and is terminated by EOL or lwsp. The whole line is + discarded. + + After this, write NEWQUOTE instead of &, everywhere. + The effect is unscoped and lasts until the next setting, + or until the end of the current directory's Suffix.sd.mk. + It takes effect on &:include'd files too, so maybe set + it back before using &:include. + + Notably + NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE + NEWQUOTE\NEWQUOTE => NEWQUOTE + NEWQUOTE\$ => $ + NEWQUOTE:changequote & set escape back to & + + +Dollar doubling and macro assistance +------------------------------------ + +&$+ Starts dollar-doubling +&$- Stops dollar-doubling + Both are idempotent and local to the file or context. + +Sometimes we will show $'s being doubled inside another construct. +This means the content of the construct is $-doubled: $-doubling is +locally enabled, and restored afterwards. + +&:macro NAME => define NAME +STUFF $ THINGS .. STUFF $$ THINGS +&:endm .. endef + NAME is processed for & + +&${..$..} => ${eval ${call ..$$..}} + (matches { } pairs to find the end) + content is $-doubled (unless it contains $- to turn that off) + + Together &:macro and &${...} provide a more reasonable macro + facility than raw make. They solve the problem that make + expansions cannot directly generate multiple rules, variable, + etc.; instead, `$(eval )' must be used, but that re-expands + the argument, meaning that all the literal text must be + $-doubled. This applies to the macro text and to the + arguments. Also `$(eval $(call ...))' is an unfortunate syntax. + Hence &:macro and &${...}. + +While dollar-doubling: +- - - - - - - - - - - + +$ => $$ including $'s produced by other + &-expansions not mentioned here + +&\$ => $ +&$NN => $(NN) where N are digits +&$( => $( + +A few contexts do not support $-doubling, such as directive arguments +or places where this might imply $-quadrupling. (There is no way to +get $-quadrupling.) + + +Tables of file reference syntaxes +--------------------------------- + +In a nonrecursive makefile supporting out of tree builds there are +three separate important distinctions between different file +locations: + + (i) In the build tree, or in the source tree ? + + (ii) In (or relative to) the subdirectory to which this Dir.sd.mk + relates, or relative to the project's top level ? + + (iii) Absolute or relative pathname ? Usually relative pathnames + suffice. Where an absolute pathname is needed, it can be built + out of &/ and an appropriate make variable such as $(PWD). + +Path construction &-expansions are built from the following: + + Relative paths in... + build source + + This directory & &^ + Top level . &~ + +In more detail, with all the various options laid out: + + Recommended Relative paths in... Absolute paths in... + for build source build source + + This lc &file &^file $(PWD)/&file $(abs_src)/&file + directory any &/file &^/file $(PWD)/&/file $(abs_src)/&/file + several & f g h &^ f g h $(addprefix...) + + Top lc file &~file + level any file &~/file $(PWD)/file $(abs_src)/file + .mk.in file $(src)/file $(PWD)/file $(abs_src)/file + several f g h &~ f g h $(addprefix...) + +(This assumes you have appropriate make variables src, PWD and +abs_src.) + +Warnings +-------- + +subdirmk's `generate' program, which does the acual &-substitution, +can produce some warnings about your .sd.mk files. These can be +suppressed with the &:warn directive. The warning tags are: + + local+global + The same VARNAME was used both with and without an & prefix. + This can be confusing. Also, if you avoid this then you will + get a warning iff you accidentally leave off a needed &. + (You can suppress this warning for a particular VARNAME with + the &:local+global directive.) + + single-char-var + A variable expansion like $FBAR. make's expansion rules + interpret this as $(F)BAR. It's normally better to write + it this way, at least if the variable expansion is followed + by more letters. Note that &$FOO works differently to + raw make: it expands to $(sub_dir_FOO). + + unknown-warning + &:warn was used to try to enable a warning that this version + of subdirmk does not understand. (Note that an attempt to + *dis*able an unknown warning is only reported if some other + warning was issued which might have been disabled.) + Subdirectory and variable naming -------------------------------- @@ -325,7 +434,7 @@ Incorporating this into your project Use `git-subtree' to merge the subdirmk/ directory. You may find it useful to symlink the DEVELOPER-CERTIFICATE file (git can store symlinks as symlinks - just `git add' the link). And you probably -want to mention the situation in your top-level COPYING. +want to mention the situation in your top-level COPYING and HACKING. Symlink autogen.sh into your project toplevel. @@ -334,13 +443,13 @@ In your configure.ac, say m4_include([subdirmk/subdirmk.ac]) SUBDIRMK_SUBDIRS([...list of subdirectories in relative syntax...]) -Write a Subdir.sd.mk in each directory. The toplevel one should +Write a Dir.sd.mk in each directory. The toplevel one should probably contain: include subdirmk/usual.mk include subdirmk/regen.mk -Write a Perdir.sd.mk in the toplevel, if you want. It should probably +Write a Suffix.sd.mk in the toplevel, if you want. It should probably have: &:include subdirmk/cdeps.sd.mk @@ -350,13 +459,14 @@ Hints ----- You can convert your project incrementally. Start with the top-level -Makefile.in and rename it to Subdir.sd.mk, and add the appropriate +Makefile.in and rename it to Dir.sd.mk, and add the appropriate stuff to configure.ac, and fix everything up. Leave the existing $(MAKE) -C for your existing subdirectories alone. Then you can convert individual subdirectories, or classes of subdirectories, at -your leisure. (You must be /sure/ that each subdirectory will be -entered only once at a time, but your existing recursive make descent -system should already do that or you already have concurrency bugs.) +your leisure. (You must be /sure/ that each recursive (non-subdirmk) +subdirectory will be entered only once at a time, but your existing +recursive make descent system should already do that or you already +have concurrency bugs.) Aside from this, be very wary of any invocation of $(MAKE) anywhere. This is a frequent source of concurrency bugs in recursive make build @@ -371,6 +481,13 @@ makefiles, you may find that just `make' is broken now and cannot get far enough to regenerate a working set of makefiles. If this happens just rerun ./config.status by hand. +If you go back and forth between different versions of your code you +can sometimes find that `make' complains that one of your Subdir.sd.mk +files is missing: typically, if iot was used and therefore a +dependency in some other version of your code. If you run `make +clean' (or `make realclean') these dependencies are suppressed, which +will clear up the problem. + Legal information -----------------