X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~ian/git?p=subdirmk.git;a=blobdiff_plain;f=README;h=50bd4d3261898d8d96ff03bed8d6774212051f84;hp=be1ece7c3d09e40d89170e7d0cb919e343f1369b;hb=c4859585c93ccf44d33e657dec83eec9cfde0ab6;hpb=cf4ae9ff56d9313777f95d53092412023f806571 diff --git a/README b/README index be1ece7..50bd4d3 100644 --- a/README +++ b/README @@ -20,7 +20,7 @@ style is not very ergonomic. The main difficulties are: `subdirmk' is an attempt to solve these problems (and it also slightly alleviates some of the boilerplate needed to support out-of-tree -builds well). +builds well, and helps a bit with metaprogramming and rule writing). Basic approach -------------- @@ -55,6 +55,192 @@ for something suitable to happen. Where there are dependencies between subdirectories, each Dir.sd.mk can simply refer to files in other subdirectories directly. +Substitution syntax +------------------- + +In general & expands to the subdirectory name when used for a +filename, and to the subdirectory name with / replaced with _ for +variable names. (If your variables start with capital letters and +your filenames with lowercase. Otherwise, use &/ or &_.) + +Note that & is processed *even in makefile comments*. The substitutor +does not understand make syntax, or shell syntax, at all. However, +the substitution rules are chosen to work well with constructs which +are common in makefiles. + +In the notation below, we suppose that the substitution is being in +done in a subdirectory sub/dir of the source tree. In the RH column +we describe the expansion at the top level, which is often a special +case (in general in variable names we call that TOP rather than the +empty string). + +&CAPS => sub_dir_CAPS or TOP_CAPS +&lc => sub/dir/lc or lc + Here CAPS is any ASCII letter A-Z and lc is a-z. + The assumption is that filenames are usually lowercase and + variables usually uppercase. Otherwise, use another syntax: + +&/ => sub/dir/ or nothing +&_ => sub_dir_ or TOP_ +&. => sub/dir or . + (This implies that `&./' works roughly like `&/', although + it can produce a needless `./') + +&= => sub_dir or TOP + +&^lc => $(top_srcdir)/sub/dir/lc +&^/ => $(top_srcdir)/sub/dir/ +&^. => $(top_srcdir)/sub/dir + +&~lc => $(top_srcdir)/lc +&~/ => $(top_srcdir)/ +&~. => $(top_srcdir) + +In general: + ^ pathname of this subdirectory in source tree + ~ pathname of top level of source tree + / terminates the path escape } needed if next is + _ terminates the var escape } not letter or space) + . terminates path escape giving dir name (excluding /) + = terminates var escape giving only prefix part (rarely needed) + lwsp starts multi-word processing (see below) + +So pathname syntax is a subset of: + '&' [ '^' | '~' ] [ lc | '/' | '.' ] + +&& => && for convenience in shell runes + +&\& => & general escaping mechanism +&\$ => $ provided for $-doubling regimes +&\NEWLINE eats the newline and vanishes + +&$VARIABLE => $(sub_dir_VARIABLE) or $(TOP_VARIABLE) + VARIABLE is ASCII starting with a letter and matching \w+ + +& thing thing... & +&^ thing thing... & +&~ thing thing... & + Convenience syntax for prefixing multiple filenames. + Introduced by & followed by lwsp where lc could go. + Each lwsp-separated non-ws word is prefixed by &/ etc. + etc. respectively. No other & escapes are recognised. + This processing continues until & preceded by lwsp, + or until EOL (the end of the line), or \ then EOL. + +&: .... + recognised at start of line only (possibly after lwsp) + +&:include filename filename should usually be [&]foo.sd.mk +&:-include filename tolerate nonexistent file + RHS is &-expanded but filenames are relative to the top + srcdir. This implies that unqualified names are like &~/ + whereas &/ is like &^/. &^ and &~ do not work here because + they expand to constructions involving literally + `$(top_srcdir)', but the RHS is not make-expanded. + +&! disables & until EOL (and then disappears) + +&# delete everything to end of line + (useful if the RHS contains unrecognised & constructions) + +&TARGETS_things + Handled specially. If mentioned at the start of a line + (possibly following whitespace), declares that this + subdir ought to have a target `things'. The rule will be + &/things:: $(&TARGETS_things) + + You may extend it by adding more :: rules for the target, + but the preferred style is to do things like this: + &TARGETS_check += & test-passed.stamp + + It is important to mention &TARGETS_things at least once in + the context of each applicable directory, because doing so + arranges that the *parent* will also have a `things' target + which recursively implies this directory's `things'. + + Must be spelled exactly &TARGETS_things. &_TARGETS_things, + for example, is not magic. To make the target exist + without providing any prerequisites for it, write a line + containing just `&TARGETS_things +='. + + `all' is extra special: every directory has an `all' + target, which corresponds to &TARGETS. + +&:warn [!]WARNTAG ... + Suppress (with !) or re-enable (without !) warnings tagged + WARNTAG (see section `Warnings', below). The suppression list + is reset at the start of processing in each subdirectory. + Warnings that appear at the end of processing are controlled + by the final warning state after processing all the toplevel + input files (including Final.sd.mk). + +&:local+global [&]VARIABLE ... + Suppresses the warning about seeing both VARIABLE and + &VARIABLE. Any & specified in the RHS is redundant: this + always affects both versions identically. + +&:changequote NEWQUOTE + changes the escape sequence from & to literally NEWQUOTE + NEWQUOTE may be any series of of non-whitespace characters, + and is terminated by EOL or lwsp. The whole line is + discarded. + + After this, write NEWQUOTE instead of &, everywhere. + The effect is unscoped and lasts until the next setting, + or until the end of the current directory's Suffix.sd.mk. + It takes effect on &:include'd files too, so maybe set + it back before using &:include. + + Notably + NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE + NEWQUOTE\NEWQUOTE => NEWQUOTE + NEWQUOTE\$ => $ + NEWQUOTE:changequote & set escape back to & + + +Dollar doubling and macro assistance +------------------------------------ + +&$+ Starts dollar-doubling +&$- Stops dollar-doubling + Both are idempotent and local to the file or context. + +Sometimes we will show $'s being doubled inside another construct. +This means the content of the construct is $-doubled: $-doubling is +locally enabled, and restored afterwards. + +&:macro NAME => define NAME +STUFF $ THINGS .. STUFF $$ THINGS +&:endm .. endef + NAME is processed for & + +&${..$..} => ${eval ${call ..$$..}} + (matches { } pairs to find the end) + content is $-doubled (unless it contains &$- to turn that off) + +Together &:macro and &${...} provide a more reasonable macro facility +than raw make. They solve the problem that make expansions cannot +directly generate multiple rules, variable, etc.; instead, `$(eval )' +must be used, but that re-expands the argument, meaning that all the +literal text must be $-doubled. This applies to the macro text and to +the arguments. Also `$(eval $(call ...))' is an unfortunate syntax. +Hence &:macro and &${...}. + +While dollar-doubling: +- - - - - - - - - - - + +$ => $$ including $'s produced by other + &-expansions not mentioned here + +&\$ => $ +&$NN => $(NN) where N are digits +&$( => $( + +A few contexts do not support $-doubling, such as directive arguments +or places where this might imply $-quadrupling. (There is no way to +get $-quadrupling.) + + Invocation, "recursive" per-directory targets --------------------------------------------- @@ -113,7 +299,9 @@ includes. So if you want to get in early and set global variables, put them near the top of Dir.sd.mk. The file Final.sd.mk in the toplevel directory is processed and -included after all the other files. +the result included after all the other files. Its subdirmk +filtering context inherits warning suppressions from the toplevel's +Dir.sd.mk etc., but not anything else. subdirmk's filter script itself sets (only) these variables: top_srcdir @@ -123,6 +311,37 @@ subdirmk's filter script itself sets (only) these variables: You are likely to want to define $(PWD), and shorter names for top_srdir and abs_top_srcdir (we suggest $(src) and $(abs_src)). +Warnings +-------- + +subdirmk's `generate' program, which does the acual &-substitution, +can produce some warnings about your .sd.mk files. These can be +suppressed with the &:warn directive. The warning tags are: + + local+global + The same VARNAME was used both with and without an & prefix. + This can be confusing. Also, if you avoid this then you will + get a warning iff you accidentally leave off a needed &. + The generation of this warning depends on scanning your + makefile for things that look like variable references, which + subdirmk does not do completely perfectly. Exciting make + syntax may evade this warning, or require suppressions. + (You can suppress this warning for a particular VARNAME with + the &:local+global directive.) + + single-char-var + A variable expansion like $FBAR. make's expansion rules + interpret this as $(F)BAR. It's normally better to write + it this way, at least if the variable expansion is followed + by more letters. Note that &$FOO works differently to + raw make: it expands to $(sub_dir_FOO). + + unknown-warning + &:warn was used to try to enable a warning that this version + of subdirmk does not understand. (Note that an attempt to + *dis*able an unknown warning is only reported if some other + warning was issued which might have been disabled.) + Global definitions ------------------ @@ -158,6 +377,7 @@ If you `include subdirmk/regen.mk', dependency management and automatic regeneration for all of this template substitution, and for config.status etc. is done for you. + Tables of file reference syntaxes --------------------------------- @@ -199,170 +419,6 @@ In more detail, with all the various options laid out: (This assumes you have appropriate make variables src, PWD and abs_src.) -Substitution syntax -------------------- - -In general & expands to the subdirectory name when used for a -filename, and to the subdirectory name with / replaced with _ for -variable names. - -Note that & is processed *even in makefile comments*. The substitutor -does not understand make syntax, or shell syntax, at all. However, -the substitution rules are chosen to work well with constructs which -are common in makefiles. - -In the notation below, we suppose that the substitution is being in -done in a subdirectory sub/dir of the source tree. In the RH column -we describe the expansion at the top level, which is often a special -case (in general in variable names we call that TOP rather than the -empty string). - -&CAPS => sub_dir_CAPS or TOP_CAPS -&lc => sub/dir/lc or lc - Here CAPS is any ASCII letter A-Z and lc is a-z. - The assumption is that filenames are usually lowercase and - variables usually uppercase. Otherwise, use another syntax: - -&/ => sub/dir/ or nothing -&_ => sub_dir_ or TOP_ -&. => sub/dir or . - (This implies that `&./' works roughly like `&/', although - it can produce a needless `./') - -&= => sub_dir or TOP - -&^lc => $(top_srcdir)/sub/dir/lc -&^/ => $(top_srcdir)/sub/dir/ -&^. => $(top_srcdir)/sub/dir - -&~lc => $(top_srcdir)/lc -&~/ => $(top_srcdir)/ -&~. => $(top_srcdir) - -In general: - ^ pathname of this subdirectory in source tree - ~ pathname of top level of source tree - / terminates the path escape } needed if next is - _ terminates the var escape } not lwsp or space) - . terminates path escape giving dir name (excluding /) - = terminates var escape giving only prefix part (rarely needed) - lwsp starts multi-word processing (see below) - -So pathname syntax is a subset of: - '&' [ '^' | '~' ] [ lc | '/' | '.' | '=' ] - -&& => && for convenience in shell runes - -&\& => & general escaping mechanism -&\$ => $ provided for $-doubling regimes -&\NEWLINE eats the newline and vanishes - -&$VARIABLE => $(sub_dir_VARIABLE) or $(TOP_VARIABLE) - VARIABLE is ASCII starting with a letter and matching \w+ - -& thing thing... & -&^ thing thing... & -&~ thing thing... & - Convenience syntax for prefixing multiple filenames. - Introduced by & followed by lwsp where lc could go. - Each lwsp-separated non-ws word is prefixed by &/ etc. - etc. respectively. No other & escapes are recognised. - This processing continues until & preceded by lwsp, - or until EOL (the end of the line), or \ then EOL. - -&: .... - recognised at start of line only (possibly after lwsp) - -&:include filename filename should usually be [&]foo.sd.mk -&:-include filename tolerate nonexistent file - filenames are relative to $(top_srcdir) - RHS is &-expanded - -&! disables & until EOL (and then disappears) - -&# delete everything to end of line - (useful if the RHS contains unrecognised & constructions) - -&:changequote NEWQUOTE - changes the escape sequence from & to literally NEWQUOTE - NEWQUOTE may be any series of of non-whitespace characters, - and is terminated by EOL or lwsp. The whole line is - discarded. - - After this, write NEWQUOTE instead of &, everywhere. - The effect is global and lasts until the next setting. - It takes effect on &:include'd files too, so maybe set - it back before using &:include. - - Notably - NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE - NEWQUOTE\NEWQUOTE => NEWQUOTE - NEWQUOTE\$ => $ - NEWQUOTE:changequote & set escape back to & - -&TARGETS_things - Handled specially. If mentioned at the start of a line - (possibly following whitespace), declares that this - subdir ought to have a target `things'. The rule will be - &/things:: $(&TARGETS_things) - - You may extend it by adding more :: rules for the target, - but the preferred style is to do things like this: - &TARGETS_check += & test-passed.stamp - - It is important to mention &TARGETS_things at least once in - the context of each applicable directory, because doing so - arranges that the *parent* will also have a `things' target - which recursively implies this directory's `things'. - - Must be spelled exactly &TARGETS_things. &_TARGETS_things, - for example, is not magic. To make the target exist - without providing any prerequisites for it, write a line - containing just `&TARGETS_things +='. - - `all' is extra special: every directory has an `all' - target, which corresponds to &TARGETS. - - -Dollar doubling and macro assistance ------------------------------------- - -&$+ Starts dollar-doubling -&$- Stops dollar-doubling - Both are idempotent and local to the file or context. - -Sometimes we will show $'s being doubled inside another construct. -This means the content of the construct is $-doubled: $-doubling is -locally enabled, and restored afterwards. - -&:macro NAME => define NAME -STUFF $ THINGS .. STUFF $$ THINGS -&:endm .. endef - NAME is processed for & - -&${..$..} => ${eval ${call ..$$..}} - (matches { } pairs to find the end) - content is $-doubled (unless it contains $- to turn that off) - - Together &:macro and &${...} provide a more reasonable macro - facility than raw make. They solve the problem that make - expansions cannot directly generate multiple rules, variable, - etc.; instead, `$(eval )' must be used, but that re-expands - the argument, meaning that all the literal text must be - $-doubled. This applies to the macro text and to the - arguments. Also `$(eval $(call ...))' is an unfortunate syntax. - Hence &:macro and &${...}. - -While dollar-doubling: -- - - - - - - - - - - - -$ => $$ including $'s produced by other - &-expansions not mentioned here - -&\$ => $ -&$NN => $(NN) where N are digits -&$( => $( - Subdirectory and variable naming -------------------------------- @@ -385,7 +441,7 @@ Incorporating this into your project Use `git-subtree' to merge the subdirmk/ directory. You may find it useful to symlink the DEVELOPER-CERTIFICATE file (git can store symlinks as symlinks - just `git add' the link). And you probably -want to mention the situation in your top-level COPYING. +want to mention the situation in your top-level COPYING and HACKING. Symlink autogen.sh into your project toplevel. @@ -414,9 +470,10 @@ Makefile.in and rename it to Dir.sd.mk, and add the appropriate stuff to configure.ac, and fix everything up. Leave the existing $(MAKE) -C for your existing subdirectories alone. Then you can convert individual subdirectories, or classes of subdirectories, at -your leisure. (You must be /sure/ that each subdirectory will be -entered only once at a time, but your existing recursive make descent -system should already do that or you already have concurrency bugs.) +your leisure. (You must be /sure/ that each recursive (non-subdirmk) +subdirectory will be entered only once at a time, but your existing +recursive make descent system should already do that or you already +have concurrency bugs.) Aside from this, be very wary of any invocation of $(MAKE) anywhere. This is a frequent source of concurrency bugs in recursive make build