X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~ianmdlvl/git?a=blobdiff_plain;f=README;h=200da3742ba9ef91d7b4c81273c22184e02883ca;hb=564022994befb8f71b89ae015751b22c34ae3ee8;hp=e5a415f8196c8088ee74d9c312e5ae6e0defad62;hpb=37a3299b4ba1aea1509b187cbd67b0bc8990653e;p=secnet.git diff --git a/README b/README index e5a415f..200da37 100644 --- a/README +++ b/README @@ -1,564 +1,648 @@ -subdirmk - assistance for non-recursive use of make -=================================================== - -Introduction ------------- - -Peter Miller's 1997 essay _Recursive Make Considered Harmful_ -persuasively argues that it is better to arrange to have a single -make invocation with the project's complete dependency tree, rather -than the currently conventional `$(MAKE) -C subdirectory' approach. - -However, actually writing a project's build system in a non-recursive -style is not very ergonomic. The main difficulties are: - - constantly having to write out long file and directory names - - the lack of a per-directory make variable namespace means - long make variables (or namespace clashes) - - it is difficult to arrange that one can cd to a subdirectory - and say `make all' and have something reasonable happen - (to wit, build an appropriate subset) - -`subdirmk' is an attempt to solve these problems (and it also slightly -alleviates some of the boilerplate needed to support out-of-tree -builds well, and helps a bit with metaprogramming and rule writing). - -Basic approach --------------- - -The developer is expected to write a makefile fragment, in each -relevant subdirectory, called `Dir.sd.mk'. - -These fragments may contain ordinary make language. Unqualified -filenames are relative to the build toplevel, and all commands all run -there. - -However, the sigil & is treated specially. By and large, it refers to -`the build directory corresponding to this .sd.mk file', etc. -There are a variety of convenient constructions. - -The result is that to a large extent, the Dir.sd.mk has an easy way -to namespace its "local" make variables, and an easy way to refer to -its "local" filenames (and filenames in general). - -The Dir.sd.mk's are filtered, fed through autoconf in the usual way -(for @..@-substitutions) and included by one autogenerated toplevel -makefile. - -So all of the input is combined and passed to one make invocation. -(A corollary is that there is no enforcement of the namespacing: -discipline is required to prefix relevant variable names with &, etc.) - -Each subdirectory is also provided with an autogenerated `Makefile' -which exists purely to capture ordinary make invocations and arrange -for something suitable to happen. - -Where there are dependencies between subdirectories, each Dir.sd.mk -can simply refer to files in other subdirectories directly. - -Substitution syntax -------------------- - -In general & expands to the subdirectory name when used for a -filename, and to the subdirectory name with / replaced with _ for -variable names. (If your variables start with capital letters and -your filenames with lowercase. Otherwise, use &/ or &_.) - -Note that & is processed *even in makefile comments*. The substitutor -does not understand make syntax, or shell syntax, at all. However, -the substitution rules are chosen to work well with constructs which -are common in makefiles. - -In the notation below, we suppose that the substitution is being in -done in a subdirectory sub/dir of the source tree. In the RH column -we describe the expansion at the top level, which is often a special -case (in general in variable names we call that TOP rather than the -empty string). - -&CAPS => sub_dir_CAPS or TOP_CAPS -&lc => sub/dir/lc or lc - Here CAPS is any ASCII letter A-Z and lc is a-z. - The assumption is that filenames are usually lowercase and - variables usually uppercase. Otherwise, use another syntax: - -&/ => sub/dir/ or nothing -&_ => sub_dir_ or TOP_ -&. => sub/dir or . - (This implies that `&./' works roughly like `&/', although - it can produce a needless `./') - -&= => sub_dir or TOP - -&^lc => ${top_srcdir}/sub/dir/lc -&^/ => ${top_srcdir}/sub/dir/ -&^. => ${top_srcdir}/sub/dir - -&~lc => ${top_srcdir}/lc -&~/ => ${top_srcdir}/ -&~. => ${top_srcdir} - -In general: - ^ pathname of this subdirectory in source tree - ~ pathname of top level of source tree - / terminates the path escape } needed if next is - _ terminates the var escape } not letter or space) - . terminates path escape giving dir name (excluding /) - = terminates var escape giving only prefix part (rarely needed) - lwsp starts multi-word processing (see below) - -So pathname syntax is a subset of: - '&' [ '^' | '~' ] [ lc | '/' | '.' ] - -&& => && for convenience in shell runes - -&\& => & general escaping mechanism -&\$ => $ provided for $-doubling regimes -&\NEWLINE eats the newline and vanishes - -&$VARIABLE => ${sub_dir_VARIABLE} or ${TOP_VARIABLE} - VARIABLE is ASCII starting with a letter and matching \w+ - -& thing thing... & -&^ thing thing... & -&~ thing thing... & - Convenience syntax for prefixing multiple filenames. - Introduced by & followed by lwsp where lc could go. - Each lwsp-separated non-ws word is prefixed by &/ etc. - etc. respectively. No other & escapes are recognised. - This processing continues until & preceded by lwsp, - or until EOL (the end of the line), or \ then EOL. - -&: .... - recognised at start of line only (possibly after lwsp) - -&:include filename filename should usually be [&]foo.sd.mk -&:-include filename tolerate nonexistent file - RHS is &-expanded but filenames are relative to the top - srcdir. This implies that unqualified names are like &~/ - whereas &/ is like &^/. &^ and &~ do not work here because - they expand to constructions involving literally - `$(top_srcdir)', but the RHS is not make-expanded. - -&! disables & until EOL (and then disappears) - -&# delete everything to end of line - (useful if the RHS contains unrecognised & constructions) - -&TARGETS_things - Handled specially. If mentioned at the start of a line - (possibly following whitespace), declares that this - subdir ought to have a target `things'. The rule will be - &/things:: $(&TARGETS_things) - - You may extend it by adding more :: rules for the target, - but the preferred style is to do things like this: - &TARGETS_check += & test-passed.stamp - - It is important to mention &TARGETS_things at least once in - the context of each applicable directory, because doing so - arranges that the *parent* will also have a `things' target - which recursively implies this directory's `things'. - - Must be spelled exactly &TARGETS_things. &_TARGETS_things, - for example, is not magic. To make the target exist - without providing any prerequisites for it, write a line - containing just `&TARGETS_things +='. - - `all' is extra special: every directory has an `all' - target, which corresponds to &TARGETS. - -&:warn [!]WARNTAG ... - Suppress (with !) or re-enable (without !) warnings tagged - WARNTAG (see section `Warnings', below). The suppression list - is reset at the start of processing in each subdirectory. - Warnings that appear at the end of processing are controlled - by the final warning state after processing all the toplevel - input files (including Final.sd.mk). - -&:local+global [!][&]VARIABLE ... - Suppresses any warnings relating to forthcoming mentions - to VARIABLE or &VARIABLE, as applicable. Scope ends at - the end of the current directory's Suffix.sd.mk. - Prefixing with ! removes [&]VARIABLE from the suppresion list. - -&:changequote NEWQUOTE - changes the escape sequence from & to literally NEWQUOTE - NEWQUOTE may be any series of of non-whitespace characters, - and is terminated by EOL or lwsp. The whole line is - discarded. - - After this, write NEWQUOTE instead of &, everywhere. - The effect is unscoped and lasts until the next setting, - or until the end of the current directory's Suffix.sd.mk. - It takes effect on &:include'd files too, so maybe set - it back before using &:include. - - Notably - NEWQUOTENEWQUOTE => NEWQUOTENEWQUOTE - NEWQUOTE\NEWQUOTE => NEWQUOTE - NEWQUOTE\$ => $ - NEWQUOTE:changequote & set escape back to & - - -Dollar doubling and macro assistance ------------------------------------- - -&$+ Starts dollar-doubling -&$- Stops dollar-doubling - Both are idempotent and local to the file or context. - -This is useful both for make macrology involving $(eval ...), and -possibly for helping write complicated recipes involving shell -variables, inline Perl code, etc. - -Sometimes we will show $'s being doubled inside another construct. -This means the content of the construct is $-doubled: $-doubling is -locally enabled, and restored afterwards. - -&:macro NAME => define NAME -STUFF $ THINGS .. STUFF $$ THINGS -&:endm .. endef - NAME is processed for & - -&{..$..} => ${eval ${call ..$$..}} - (matches { } pairs to find the end) - content is $-doubled (unless it contains &$- to turn that off) - cf &(...), see "Convenience syntax for eval", below. - -Together &:macro and &${...} provide a more reasonable macro facility -than raw make. They solve the problem that make expansions cannot -directly generate multiple rules, variable, etc.; instead, `$(eval )' -must be used, but that re-expands the argument, meaning that all the -literal text must be $-doubled. This applies to the macro text and to -the arguments. Also `$(eval $(call ...))' is an unfortunate syntax. -Hence &:macro and &${...}. - -While dollar-doubling: -- - - - - - - - - - - - -$ => $$ including $'s produced by other - &-expansions not mentioned here - -&\$ => $ -&$( => $( -&$NN => ${NN} where N are digits - -A few contexts do not support $-doubling, such as directive arguments -or places where this might imply $-quadrupling. (There is no way to -get $-quadrupling.) - -Convenience syntax for eval -- - - - - - - - - - - - - - - -&(thing => $(call thing -&( thing => $(call thing - and specially: -&(&lc => $(call sub_dir_lc or $(call TOP_lc -&( &lc => $(call sub_dir_lc or $(call TOP_lc - even though lc would normally be thought a filename - -Unlike &{...}, this does not involve any dollar-doubling. - - -Invocation, "recursive" per-directory targets ---------------------------------------------- - -Arrangements are made so that when you run `make foo' in a -subdirectory, it is like running the whole toplevel makefile, from the -toplevel, as `make subdir/foo'. If `subdir/foo' is a file that might -be built, that builds it. - -But `foo' can also be a conventional target like `all'. - -Each subdirectory has its own `all' target. For example a -subdirectory `src' has a target `src/all'. The rules for these are -automatically generated from the settings of the per-directory -&TARGETS variables. &TARGETS is magic in this way. (In -src/Dir.sd.mk, &TARGETS of course refers to a make variable called -src_TARGETS.) - -The `all' target in a parent directory is taken to imply the `all' -targets in all of its subdirectories, recursively. And in the -autogenerated stub Makefiles, `all' is the default target. So if you -just type `make' in the toplevel, you are asking for `&all' -(/all) for every directory in the project. - -In a parallel build, the rules for all these various subdirectory -targets may be in run in parallel: there is only one `make' invocation -at a time. There is no sequencing between subdirectories, only been -individual targets (as specified according to their dependencies). - -You can define other per-directory recursive targets too: set the -variable &TARGETS_zonk, or whatever (being sure to write &TARGETS_zonk -at the start of a line). This will create a src/zonk target (for -appropriate value of src/). Unlike `all', these other targets only -exist in areas of the project where at least something mentions them. -So for example, if &TARGETS_zonk is set in src but not lib, `make -zonk' in lib will fail. If you want to make a target exist -everywhere, += it with nothing in Prefix.sd.mk or Suffix.sd.mk (see -below). - -Prefix.sd.mk, Suffix.sd.mk, Final.sd.mk, inclusion --------------------------------------------------- - -The files Prefix.sd.mk and Suffix.sd.mk in the toplevel of the source -are automatically processed before and after each individual -directory's Dir.sd.mk, and the &-substituted contents therefore -appear once for each subdirectory. - -This lets you do per-directory boilerplate. Some useful boilerplate -is already provided in subdirmk, for you to reference like this: - &:include subdirmk/cdeps.sd.mk - &:include subdirmk/clean.sd.mk -For example you could put that in Suffix.sd.mk. - -The top-level Dir.sd.mk is the first makefile included after the -autogenerated `main.mk' which merely has some basic settings and -includes. So if you want to get in early and set global variables, -put them near the top of Dir.sd.mk. - -The file Final.sd.mk in the toplevel directory is processed and -the result included after all the other files. Its subdirmk -filtering context inherits warning suppressions from the toplevel's -Dir.sd.mk etc., but not anything else. - -subdirmk's filter script itself sets (only) these variables: - top_srcdir - abs_top_srcdir - SUBDIRMK_MAKEFILES - MAKEFILE_TEMPLATES -You are likely to want to define $(PWD), and shorter names for -top_srdir and abs_top_srcdir (we suggest $(src) and $(abs_src)). - -Warnings --------- - -subdirmk's `generate' program, which does the acual &-substitution, -can produce some warnings about your .sd.mk files. These can be -suppressed with the &:warn directive. The warning tags are: - - local+global - The same VARNAME was used both with and without an & prefix. - This can be confusing. Also, if you avoid this then you will - get a warning iff you accidentally leave off a needed &. - The generation of this warning depends on scanning your - makefile for things that look like variable references, which - subdirmk does not do completely perfectly. Exciting make - syntax may evade this warning, or require suppressions. - (You can suppress this warning for a particular VARNAME with - the &:local+global directive.) - - single-char-var - A variable expansion like $FBAR. make's expansion rules - interpret this as $(F)BAR. It's normally better to write - it this way, at least if the variable expansion is followed - by more letters. Note that &$FOO works differently to - raw make: it expands to ${sub_dir_FOO}. - - broken-var-ref - An attempt at variable expansion looking like $&... - You probably expected this to mean $(TOP_F)BAR but it - expands to $TOP_FBAR which make thinks means $(T)OP_FBAR. - - unknown-warning - &:warn was used to try to enable a warning that this version - of subdirmk does not understand. (Note that an attempt to - *dis*able an unknown warning is only reported if some other - warning was issued which might have been disabled.) - - -Guides, hints, and further explanations -======================================= - -Incorporating this into your project ------------------------------------- - -Use `git-subtree' to merge the subdirmk/ directory. You may find it -useful to symlink the DEVELOPER-CERTIFICATE file (git can store -symlinks as symlinks - just `git add' the link). And you probably -want to mention the situation in your top-level COPYING and HACKING. - -Symlink autogen.sh into your project toplevel. - -In your configure.ac, say +secnet - flexible VPN software - m4_include([subdirmk/subdirmk.ac]) - SUBDIRMK_SUBDIRS([...list of subdirectories in relative syntax...]) - -Write a Dir.sd.mk in each directory. See the substitution syntax -reference, above, and the example/ directory here. The toplevel -Dir.sd.mk should probably contain: +* Copying - include subdirmk/usual.mk - include subdirmk/regen.mk +secnet is + Copyright 1995-2003 Stephen Early + Copyright 2002-2014 Ian Jackson + Copyright 1991 Massachusetts Institute of Technology + Copyright 1998 Ross Anderson, Eli Biham, Lars Knudsen + Copyright 1993 Colin Plumb + Copyright 1998 James H. Brown, Steve Reid + Copyright 2000 Vincent Rijmen, Antoon Bosselaers, Paulo Barreto + Copyright 2001 Saul Kravitz + Copyright 2004 Fabrice Bellard + Copyright 2002 Guido Draheim + Copyright 2005-2010 Free Software Foundation, Inc. + Copyright 1995-2001 Jonathan Amery + Copyright 1995-2003 Peter Benie + Copyright 2011 Richard Kettlewell + Copyright 2012 Matthew Vernon + Copyright 2013-2019 Mark Wooding + Copyright 1995-2013 Simon Tatham -Write a Suffix.sd.mk in the toplevel, if you want. It should probably -have: +secnet is distributed under the terms of the GNU General Public +License, version 3 or later. Some individual files have more +permissive licences; where this is the case, it is documented in the +header comment for the files in question. + +secnet is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +The file COPYING contains a copy of the GNU GPL v3. + + +* Introduction + +secnet allows large virtual private networks to be constructed +spanning multiple separate sites. It is designed for the case where a +private network connecting many hosts is 'hidden' behind a single +globally-routable IP address, but can also be applied in other +circumstances. It communicates entirely using UDP, and works well +with gateways that implement network address translation. + +If you are installing secnet to join an existing VPN, you should read +the 'INSTALL' file and your particular VPN's documentation now. You +may need to refer back to this file for information on the netlink and +comm sections of the configuration file. + +If you are thinking about setting up a new VPN of any size (from one +providing complete links between multiple sites to a simple +laptop-to-host link), read the section in this file on 'Creating a +VPN'. + +* Mailing lists and bug reporting + +There are two mailing lists associated with secnet: an 'announce' list +and a 'discuss' list. Their addresses are: +http://www.chiark.greenend.org.uk/mailman/listinfo/secnet-announce +http://www.chiark.greenend.org.uk/mailman/listinfo/secnet-discuss + +The -announce list receives one message per secnet release. The +-discuss list is for general discussion, including help with +configuration, bug reports, feature requests, etc. + +Bug reports should be sent to ; they will be +forwarded to the -discuss list by me. + +* Creating a VPN + +XXX TODO + +* secnet configuration file format + +By default secnet on linux reads /etc/secnet/secnet.conf. The default +may be different on other platforms. + +This file defines a dictionary (a mapping from keys to values) full of +configuration information for secnet. Two keys must be defined in +this file for secnet to start. One is "system", a dictionary +containing systemwide control parameters. The other is "sites", a +list of all the sites that you intend to communicate with. + +The configuration file has a very simple syntax; keys are defined as +follows: + +key definition; +or +key = definition; + +(the "=" is optional) + +Keys must match the following regular expression: +[[:alpha:]_][[:alnum:]\-_]* + +i.e. the first character must be an alpha or an underscore, and the +remaining characters may be alphanumeric, '-' or '_'. + +Keys can be defined to be a comma-separated list of any of the +following types: + + a boolean + a string, in quotes + a number, in decimal + a dictionary of definitions, enclosed in { } + a "closure", followed by arguments + a path to a key that already exists, to reference that definition + +Note that dictionaries can be nested: a key in one dictionary can +refer to another dictionary. When secnet looks for a key in a +particular directory and can't find it, it looks in the dictionary's +lexical 'parents' in turn until it finds it (or fails to find it at +all and stops with an error). + +Definitions can refer to previous definitions by naming them with a +path. Paths are key1/key2/key3... (starting from wherever we find +key1, i.e. in the current dictionary or any of its parents), or +alternatively /key1/key2/key3... (to start from the root). +Definitions cannot refer to future definitions. - &:include subdirmk/cdeps.sd.mk - &:include subdirmk/clean.sd.mk +Example: +a=1; +b=2; +c={ d=3; e=a; }; +f={ a=4; g=c; }; -Hints ------ +The following paths are valid: +a is 1 +b is 2 +c is a dictionary: + c/d is 3 + c/e is 1 +f is a dictionary: + f/a is 4 + f/g is a dictionary: + f/g/d is 3 + f/g/e is 1 -You can convert your project incrementally. Start with the top-level -Makefile.in and rename it to Dir.sd.mk, and add the appropriate -stuff to configure.ac, and fix everything up. Leave the existing -$(MAKE) -C for your existing subdirectories alone. Then you can -convert individual subdirectories, or classes of subdirectories, at -your leisure. (You must be /sure/ that each recursive (non-subdirmk) -subdirectory will be entered only once at a time, but your existing -recursive make descent system should already do that or you already -have concurrency bugs.) +Note that f/g/e is NOT 4. -Aside from this, be very wary of any invocation of $(MAKE) anywhere. -This is a frequent source of concurrency bugs in recursive make build -systems. When combined with nonrecursive make it's all in the same -directory and there is nothing stopping the different invocations -ending up trying to make the same targets at the same time. That -causes hideous racy lossage. There are ways to get this to work -reliably but it is advanced stuff. +Elements that are lists are inserted into lists in definitions, not +referenced by them (i.e. you can't have lists of lists). -If you make syntax errors, or certain kinds of other errors, in your -makefiles, you may find that just `make' is broken now and cannot get -far enough to regenerate a working set of makefiles. If this happens -just rerun ./config.status by hand. +Some closures may be followed by an argument list in ( ), and may +return any number of whatever type they like (including other +closures). Some types of closure (typically those returned from +invokations of other closures) cannot be invoked. -If you go back and forth between different versions of your code you -can sometimes find that `make' complains that one of your Dir.sd.mk -files is missing: typically, if iot was used and therefore a -dependency in some other version of your code. If you run `make -clean' (or `make realclean') these dependencies are suppressed, which -will clear up the problem. +closure { definitions } is short for closure({definitions}). +The main body of secnet, and all the additional modules, predefine +some keys in the root dictionary. The main ones are: -Global definitions ------------------- + yes, true, True, TRUE, on: the boolean value True + no, false, False, FALSE, off: the boolean value False + makelist: turns a dictionary (arg1) into a list of definitions + (ignoring the keys) + readfile: reads a file (arg1) and returns it as a string + map: applies the closure specified as arg1 to each of the + remaining elements in the list in turn. Returns a list + made up of the outputs of the closure. + +Keys defined by modules are described below, in the module +documentation. + +Other configuration files can be included inline by writing "include +filename" at the start of a line. + +After the configuration file is read, secnet looks for particular keys +in configuration space to tell it what to do: + + system: a dictionary which can contain the following keys: + log (log closure): a destination for system messages + userid (string): the userid for secnet to run as once it drops privileges + pidfile (string): where to store its PID + + sites: a list of closures of type 'site', which define other tunnel + endpoints that secnet will attempt to communicate with + +* secnet command line options + +Usage: secnet [OPTION]... + + -f, --silent, --quiet suppress error messages + -w, --nowarnings suppress warnings + -v, --verbose output extra diagnostics + -c, --config=filename specify a configuration file + -j, --just-check-config stop after reading configfile + -n, --nodetach do not run in background + -d, --debug=item,... set debug options + --help display this help and exit + --version output version information and exit + +* base91s + +secnet defines a variant of the base91 encoding `basE91', from + http://base91.sourceforge.net/ + +base91s is the same as baseE91 except that: + - in the encoded charset, `"' is replaced with `-' + - spaces, newlines etc. and other characters outside the charset + are not permitted (although in some places they may be ignored, + this is not guaranteed). + +* secnet builtin modules + +** resolver + +Defines: + adns (closure => resolver closure) + +adns: dict argument + config (string): optional, a resolv.conf for ADNS to use + +** random + +Defines: + randomsrc (closure => randomsrc closure) + +randomsrc: string[,bool] + arg1: filename of random source + arg2: if True then source is blocking + +** udp + +Defines: + udp (closure => comm closure) + +udp: dict argument + address (string list): IPv6 or IPv4 addresses to listen and send on; + default is all local addresses + port (integer): UDP port to listen and send on; optional if you + don't need to have a stable address for your peers to talk to + (in which case your site ought probably to have `local-mobile true'). + buffer (buffer closure): buffer for incoming packets + authbind (string): optional, path to authbind-helper program + +** polypath + +Defines: + polypath (closure => comm closure) + +polypath: dict argument + port (integer): UDP port to listen and send on + buffer (buffer closure): buffer for incoming packets + authbind (string): optional, path to authbind-helper program + max-interfaces (number): optional, max number of different interfaces to + use (also, maximum steady-state amount of packet multiplication); + interfaces marked with `@' do not count. + interfaces (string list): which interfaces to process; each entry is + optionally `!' or `+' or `@' followed by a glob pattern (which is + applied to a prospective interface using fnmatch with no flags). + `+' or nothing means to process normally. `!' means to ignore; + `@' means to use only in conjunction with dedicated-interface-addr. + If no list is specified, or the list ends with a `!' entry, a + default list is used/appended: + "!tun*","!tap*","!sl*","!userv*","!lo","@hippo*","*". + Patterns which do not start with `*' or an alphanumeric need to be + preceded by `!' or `+' or `@'. + monitor-command (string list): Program to use to monitor appearance + and disappearance of addresses on local network interfaces. Should + produce lines of the form `+|- 4|6 ' where is + an address literal. Each - line should relate to a previously + printed + line. On startup, should produce a + line for each + currently existing address. secnet does filtering so there is no + need to strip out tun interfaces, multicast addresses, and so on. + The command is run as the user secnet is started as (not the one + which secnet may drop privilege to due to the configured `userid'). + The default depends on the operating system. + permit-loopback (boolean): Normally, loopback IPv6 and IPv4 + addresses on local interfaces are disregarded, because such + interfaces are not interesting for communicating with distant + hosts. Setting this option will ignore that check, which can be + useful for testing. Setting this option also removes "!lo*" from + the default interface pattern list. + +When using this comm, packets are sent out of every active interface +on the host (where possible). It is important that interfaces created +by secnet itself are not included! secnet's default filter list tries +to do this. + +This comm only makes sense for sites which are mobile. That is, the +site closures used with this comm should all have the `local-mobile' +parameter set to `true'. When the local site site is not marked +mobile the address selection machinery might fixate on an unsuitable +address. + +polypath takes site-specific informtion as passed to the `comm-info' +site closure parameter. The entries understood in the dictionary +are: + dedicated-interface-addr (string): IPv4 or IPv6 address + literal. Interfaces specified with `@' in `interfaces' will be + used for the corresponding site iff the interface local address + is this address. + +For an interface to work with polypath, it must either have a suitable +default route, or be a point-to-point interface. In the general case +this might mean that the host would have to have multiple default +routes. However in practice the most useful configuration is two +interfaces being (1) wifi (2) mobile internet. + +I have had success on Linux by using network-manager for wifi and +invoking ppp directly for mobile internet. ppp sets up a +point-to-point link, and does not add a default route if there already +is one. network-manager always sets up a default route. The result +is that the wifi always has a default route (so is useable); ppp +(being a point-to-point link) does not need one. + +The use of polypath requires that secnet be started with root +privilege, to make the setsockopt(,,SO_BINDTODEVICE,) calls. If the +configuration specifies that secnet should drop privilege (see +`userid' above), secnet will keep a special process around for this +purpose; that process will handle local network interface changes but +does not deal with any packets, key exchange, etc. + +polypath support is only available when secnet is built against an +IPv6-capable version of adns (because it wants features in the newer +adns). + +** log + +Defines: + logfile (closure => log closure) + syslog (closure => log closure) + +logfile: dict argument + filename (string): where to log to; default is stderr + prefix (string): added to messages [""] + class (string list): what type of messages to log + { "debug-config", M_DEBUG_CONFIG }, + { "debug-phase", M_DEBUG_PHASE }, + { "debug", M_DEBUG }, + { "all-debug", M_DEBUG|M_DEBUG_PHASE|M_DEBUG_CONFIG }, + { "info", M_INFO }, + { "notice", M_NOTICE }, + { "warning", M_WARNING }, + { "error", M_ERROR }, + { "security", M_SECURITY }, + { "fatal", M_FATAL }, + { "default", M_WARNING|M_ERROR|M_SECURITY|M_FATAL }, + { "verbose", M_INFO|M_NOTICE|M_WARNING|M_ERROR|M_SECURITY|M_FATAL }, + { "quiet", M_FATAL } + +logfile will close and reopen its file upon receipt of SIGHUP. + +syslog: dict argument + ident (string): include this string in every log message + facility (string): facility to log as + { "authpriv", LOG_AUTHPRIV }, + { "cron", LOG_CRON }, + { "daemon", LOG_DAEMON }, + { "kern", LOG_KERN }, + { "local0", LOG_LOCAL0 }, + { "local1", LOG_LOCAL1 }, + { "local2", LOG_LOCAL2 }, + { "local3", LOG_LOCAL3 }, + { "local4", LOG_LOCAL4 }, + { "local5", LOG_LOCAL5 }, + { "local6", LOG_LOCAL6 }, + { "local7", LOG_LOCAL7 }, + { "lpr", LOG_LPR }, + { "mail", LOG_MAIL }, + { "news", LOG_NEWS }, + { "syslog", LOG_SYSLOG }, + { "user", LOG_USER }, + { "uucp", LOG_UUCP } + +** util + +Defines: + sysbuffer (closure => buffer closure) + +sysbuffer: integer[,dict] + arg1: buffer length + arg2: options: + lockdown (boolean): if True, mlock() the buffer + +** site + +Defines: + site (closure => site closure) + +site: dict argument + local-name (string): this site's name for itself + name (string): the name of the site's peer + link (netlink closure) + comm (one or more comm closures): if there is more than one, the + first one will be used for any key setups initiated by us using the + configured address. Others are only used if our peer talks to + them. + resolver (resolver closure) + random (randomsrc closure) + key-cache (privcache closure) + local-key (sigprivkey closure): Deprecated; use key-cache instead. + address (string list): optional, DNS name(s) used to find our peer; + address literals are supported too if enclosed in `[' `]'. + port (integer): mandatory if 'address' is specified: the port used + to contact our peer + peer-keys (string): path (prefix) for peer public key set file(s); + see README.make-secnet-sites re `pub' etc. and NOTES.peer-keys. + key (sigpubkey closure): our peer's public key (obsolete) + transform (transform closure): how to mangle packets sent between sites + dh (dh closure) + key-lifetime (integer): max lifetime of a session key, in ms + [one hour; mobile: 2 days] + setup-retries (integer): max number of times to transmit a key negotiation + packet [5; mobile: 30] + setup-timeout (integer): time between retransmissions of key negotiation + packets, in ms [2000; mobile: 1000] + wait-time (integer): after failed key setup, wait roughly this long + (in ms) before allowing another attempt [20000; mobile: 10000] + Actual wait time is randomly chosen between ~0.5x and ~1.5x this. + renegotiate-time (integer): if we see traffic on the link after this time + then renegotiate another session key immediately (in ms) + [half key-lifetime, or key-lifetime minus 5 mins (mobile: 12 hours), + whichever is longer]. + keepalive (bool): if True then attempt always to keep a valid session key. + [false] + log-events (string list): types of events to log for this site + unexpected: unexpected key setup packets (may be late retransmissions) + setup-init: start of attempt to setup a session key + setup-timeout: failure of attempt to setup a session key, through timeout + activate-key: activation of a new session key + timeout-key: deletion of current session key through age + security: anything potentially suspicious + state-change: steps in the key setup protocol + packet-drop: whenever we throw away an outgoing packet + dump-packets: every key setup packet we see + errors: failure of name resolution, internal errors + peer-addrs: changes to sets of peer addresses (interesting for mobile peers) + all: everything (too much!) + mobile (bool): if True then peer is "mobile" ie we assume it may + change its apparent IP address and port number without either it + or us being aware of the change; so, we remember the last several + port/addr pairs we've seen and send packets to all of them + (subject to a timeout). We maintain one set of addresses for key + setup exchanges, and another for data traffic. Two communicating + peers must not each regard the other as mobile, or all the traffic + in each direction will be triplicated (strictly, transmitted + mobile-peers-max times) and anyway two peers whose public contact + address may suddenly change couldn't communicate reliably because + their contact addresses might both change at once. [false] + mobile-peers-max (integer): Maximum number of peer port/addr pairs we + remember and send to. Must be at least 1 and no more than 5. + [4 if any address is configured, otherwise 3] + static-peers-max (integer): Maximum number of peer port/addr pairs + we can try for a static site. Must be at least 1 and no more + than 5. [4 or 3, as above] + mobile-peer-expiry (integer): For "mobile" peers only, the length + of time (in seconds) for which we will keep sending to multiple + address/ports from which we have not seen incoming traffic. [120] + local-mobile (bool): if True then other peers have been told we are + "mobile". This should be True iff the peers' site configurations + for us have "mobile True" (and if we find a site configuration for + ourselves in the config, we insist on this). The effect is to + check that there are no links both ends of which are allegedly + mobile (which is not supported, so those links are ignored) and + to change some of the tuning parameter defaults. [false] + mtu-target (integer): Desired value of the inter-site MTU for this + peering. This value will be advertised to the peer (which ought + to affect incoming packets), and if the peer advertises an MTU its + value will be combined with this setting to compute the inter-site + MTU. (secnet will still accept packets which exceed the + (negotiated or assumed) inter-site MTU.) Setting a lower + inter-site MTU can be used to try to restrict the sizes of the + packets sent over the underlying public network (e.g. to work + around network braindamage). It is not normally useful to set a + larger value for mtu-target than the VPN's general MTU (which + should be reflected in the local private interface MTU, ie the mtu + parameter to netlink). If this parameter is not set, or is set + to 0, the default is to use the local private link mtu. + comm-info (dict): Information for the comm, used when this site + wants to transmit. If the comm does not support this, it is + ignored. + +Links involving mobile peers have some different tuning parameter +default values, which are generally more aggressive about retrying key +setup but more relaxed about using old keys. These are noted with +"mobile:", above, and apply whether the mobile peer is local or +remote. + +** transform-eax + +Defines: + eax-serpent (closure => transform closure) + +** transform-cbcmac + +Defines: + serpent256-cbc (closure => transform closure) + +** netlink + +Defines: + null-netlink (closure => closure or netlink closure) + +null-netlink: dict argument + name (string): name for netlink device, used in log messages + networks (string list): networks on the host side of the netlink device + remote-networks (string list): networks that may be claimed + by the remote site using this netlink device + local-address (string): IP address of host's tunnel interface + secnet-address (string): IP address of this netlink device + ptp-address (string): IP address of the other end of a point-to-point link + mtu (integer): MTU of host's tunnel interface + +Only one of secnet-address or ptp-address may be specified. If +point-to-point mode is in use then the "routes" option must also be +specified, and netlink returns a netlink closure that should be used +directly with the "link" option to the site closure. If +point-to-point mode is not in use then netlink returns a closure that +may be invoked using a dict argument with the following keys to yield +a netlink closure: + routes (string list): networks reachable down the tunnel attached to + this instance of netlink + options (string list): + allow-route: allow packets coming from this tunnel to be routed to + other tunnels as well as the host (used for mobile devices like laptops) + soft: remove these routes from the host's routing table when + the tunnel link quality is zero + mtu (integer): MTU of host's tunnel interface + +Netlink will dump its current routing table to the system/log on +receipt of SIGUSR1. + +** slip + +Defines: + userv-ipif (closure => netlink closure) + +userv-ipif: dict argument + userv-path (string): optional, where to find userv ["userv"] + service-user (string): optional, username for userv-ipif service ["root"] + service-name (string): optional, name of userv-ipif service ["ipif"] + buffer (buffer closure): buffer for assembly of host->secnet packets + plus generic netlink options, as for 'null-netlink' + +** tun + +Defines: + tun (closure => netlink closure) [only on linux-2.4] + tun-old (closure => netlink closure) + +tun: dict argument + flavour (string): optional, type of TUN interface to use + ("guess","linux","bsd","streams") + device (string): optional, path of TUN/TAP device file ["/dev/net/tun"] + interface (string): optional, name of tunnel network interface + ifconfig-path (string): optional, path to ifconfig command + route-path (string): optional, path to route command + ifconfig-type (string): optional, how to perform ifconfig + route-type (string): optional, how to add and remove routes + types are: "guess", "ioctl", "bsd", "linux", "solaris-2.5" + buffer (buffer closure): buffer for host->secnet packets + plus generic netlink options, as for 'null-netlink' + +I recommend you don't specify the 'interface' option unless you're +doing something that requires the interface name to be constant. + +** privcache + +Cache of dynamically loaded private keys. + +Defines: + priv-cache (closure => privcache closure) + +priv-cache: dict argument + privkeys (string): path prefix for private keys. Each key is + looked for at this path prefix followed by the 10-character + hex key id. + privcache-size (integer): optional, maximum number of private + keys to retain at once. [5] + privkey-max (integer): optional, maximum size of private key + file in bytes. [4095] -If want to set global variables, such as CC, that should only be done -once. You can put them in your top-level Dir.sd.mk, or a separate -file you `include' and declare using SUBDIRMK_MAKEFILES. +** pubkeys -If you need different settings of variables like CC for different -subdirectories, you should probably do that with target-specific -variable settings. See the info node `(make) Target-specific'. +Defines: + make-public (closure => sigpubkey closure) +make-public: ( + arg1: sigscheme name + arg2: base91s encoded public key data, according to algorithm -Directory templates `.sd.mk' vs plain autoconf templates `.mk.in' --------------------------------------------------------------------- +** rsa -There are two kinds of template files. +Defines: + sigscheme algorithm 00 "rsa1" + rsa-private (closure => sigprivkey closure) + rsa-public (closure => sigpubkey closure) + +rsa1 sigscheme algorithm: + private key: SSH private key file, version 1, no password + public key: SSH public key file, version 1 + (length, restrictions, email, etc., ignored) + +rsa-private: string[,bool] + arg1: filename of SSH private key file (version 1, no password) + arg2: whether to check that the key is usable [default True] + +rsa-public: string,string + arg1: encryption key (decimal) + arg2: modulus (decimal) - Filename .sd.mk .mk.in +The sigscheme is hardcoded to use sha1. Both rsa-private and +rsa-public look for the following config key in their context: + hash (hash closure): hash function [sha1] - Processed by &-substitution, autoconf only - then autoconf - Instantiated Usu. once per subdir Once only +** dh - Need to be mentioned No, but Dir.sd.mk All not in subdirmk/ - in configure.ac? via SUBDIRMK_SUBDIRS via SUBDIRMK_MAKEFILES +Defines: + diffie-hellman (closure => dh closure) - How to include `&:include foo.sd.mk' `include foo.mk' - in all relevant .sd.mk in only one - (but not needed for Dir.sd.mk - Prefix, Suffix, Final) +diffie-hellman: string,string[,bool] + arg1: modulus (hex) + arg2: generator (hex) + arg3: whether to check that the modulus is prime [default True] -If you `include subdirmk/regen.mk', dependency management and -automatic regeneration for all of this template substitution, and for -config.status etc. is done for you. +** md5 +Defines: + md5 (hash closure) -Tables of file reference syntaxes ---------------------------------- +** sha1 -In a nonrecursive makefile supporting out of tree builds there are -three separate important distinctions between different file -locations: +Defines: + sha1 (hash closure) - (i) In the build tree, or in the source tree ? +** conffile - (ii) In (or relative to) the subdirectory to which this Dir.sd.mk - relates, or relative to the project's top level ? +Defines: + makelist (dictionary => list of definitions) + readfile (string => string) + map (closure,list => list) - (iii) Absolute or relative pathname ? Usually relative pathnames - suffice. Where an absolute pathname is needed, it can be built - out of &/ and an appropriate make variable such as $(PWD). +makelist: dictionary + returns a list consisting of the definitions in the dictionary. The keys + are discarded. -Path construction &-expansions are built from the following: +readfile: string + reads the named file and returns its contents as a string - Relative paths in... - build source - - This directory & &^ - Top level . &~ - -In more detail, with all the various options laid out: - - Recommended Relative paths in... Absolute paths in... - for build source build source - - This lc &file &^file $(PWD)/&file $(abs_src)/&file - directory any &/file &^/file $(PWD)/&/file $(abs_src)/&/file - several & f g h &^ f g h $(addprefix...) - - Top lc file &~file - level any file &~/file $(PWD)/file $(abs_src)/file - .mk.in file $(src)/file $(PWD)/file $(abs_src)/file - several f g h &~ f g h $(addprefix...) - -(This assumes you have appropriate make variables src, PWD and -abs_src.) - - -Subdirectory and variable naming --------------------------------- - -The simple variable decoration scheme does not enforce a strict -namespace distinction between parts of variable names which come from -subdirectory names, and parts that mean something else. - -So it is a good idea to be a bit careful with your directory naming. -`TOP', names that contain `_', and names that are similar to parts of -make variables (whether conventional ones, or ones used in your -project) are best avoided. - -If you name your variables in ALL CAPS and your subdirectories in -lower case with `-' rather than `_', there will be no confusion. - - -Legal information -================= - -subdirmk is - Copyright 2019 Mark Wooding - Copyright 2019 Ian Jackson - - subdirmk and its example is free software; you can redistribute it - and/or modify it under the terms of the GNU Library General Public - License as published by the Free Software Foundation; either - version 2 of the License, or (at your option) any later version. - - This is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Library General Public License for more details. - - You should have received a copy of the GNU Library General Public - License along with this library as the file LGPL-2. - If not, see https://www.gnu.org/. - -Individual files generally contain the following tag in the copyright -notice, instead of the full licence grant text: - SPDX-License-Identifier: LGPL-2.0-or-later -As is conventional, this should be read as a licence grant. - -Contributions are accepted based on the git commit Signed-off-by -convention, by which the contributors' certify their contributions -according to the Developer Certificate of Origin version 1.1 - see -the file DEVELOPER-CERTIFICATE. - -Where subdirmk is used by and incorporated into another project (eg -via git subtree), the directory subdirmk/ is under GNU LGPL-2.0+, and -the rest of the project are under that other project's licence(s). -(The project's overall licence must be compatible with LGPL-2.0+.) +map: + applies the closure specified as arg1 to each of the elements in the list. + Returns a list made up of the outputs of the closure.