Static buffers: Provide new rotating static buffer macros

[secnet.git] / README

secnet - flexible VPN software

* Copying

secnet is Copyright (C) 1995--2003 Stephen Early <steve@greenend.org.uk>
It is distributed under the terms of the GNU General Public License,
version 2 or later.  See the file COPYING for more information.

The IP address handling library in ipaddr.py is Copyright (C)
1996--2000 Cendio Systems AB, and is distributed under the terms of
the GPL.

* Introduction

secnet allows large virtual private networks to be constructed
spanning multiple separate sites.  It is designed for the case where a
private network connecting many hosts is 'hidden' behind a single
globally-routable IP address, but can also be applied in other
circumstances.  It communicates entirely using UDP, and works well
with gateways that implement network address translation.

If you are installing secnet to join an existing VPN, you should read
the 'INSTALL' file and your particular VPN's documentation now.  You
may need to refer back to this file for information on the netlink and
comm sections of the configuration file.

If you are thinking about setting up a new VPN of any size (from one
providing complete links between multiple sites to a simple
laptop-to-host link), read the section in this file on 'Creating a
VPN'.

* Mailing lists and bug reporting

There are two mailing lists associated with secnet: an 'announce' list
and a 'discuss' list.  Their addresses are:
http://www.chiark.greenend.org.uk/mailman/listinfo/secnet-announce
http://www.chiark.greenend.org.uk/mailman/listinfo/secnet-discuss

The -announce list receives one message per secnet release.  The
-discuss list is for general discussion, including help with
configuration, bug reports, feature requests, etc.

Bug reports should be sent to <steve@greenend.org.uk>; they will be
forwarded to the -discuss list by me.

* Creating a VPN

XXX TODO

* secnet configuration file format

By default secnet on linux reads /etc/secnet/secnet.conf.  The default
may be different on other platforms.

This file defines a dictionary (a mapping from keys to values) full of
configuration information for secnet.  Two keys must be defined in
this file for secnet to start.  One is "system", a dictionary
containing systemwide control parameters.  The other is "sites", a
list of all the sites that you intend to communicate with.

The configuration file has a very simple syntax; keys are defined as
follows:

key definition;
or
key = definition;

(the "=" is optional)

Keys must match the following regular expression:
[[:alpha:]_][[:alnum:]\-_]*

i.e. the first character must be an alpha or an underscore, and the
remaining characters may be alphanumeric, '-' or '_'.

Keys can be defined to be a comma-separated list of any of the
following types:

  a boolean
  a string, in quotes
  a number, in decimal
  a dictionary of definitions, enclosed in { }
  a "closure", followed by arguments
  a path to a key that already exists, to reference that definition

Note that dictionaries can be nested: a key in one dictionary can
refer to another dictionary. When secnet looks for a key in a
particular directory and can't find it, it looks in the dictionary's
lexical 'parents' in turn until it finds it (or fails to find it at
all and stops with an error).

Definitions can refer to previous definitions by naming them with a
path.  Paths are key1/key2/key3... (starting from wherever we find
key1, i.e. in the current dictionary or any of its parents), or
alternatively /key1/key2/key3... (to start from the root).
Definitions cannot refer to future definitions.

Example:

a=1;
b=2;
c={ d=3; e=a; };
f={ a=4; g=c; };

The following paths are valid:
a is 1
b is 2
c is a dictionary:
 c/d is 3
 c/e is 1
f is a dictionary:
 f/a is 4
 f/g is a dictionary:
  f/g/d is 3
  f/g/e is 1

Note that f/g/e is NOT 4.

Elements that are lists are inserted into lists in definitions, not
referenced by them (i.e. you can't have lists of lists).

Some closures may be followed by an argument list in ( ), and may
return any number of whatever type they like (including other
closures).  Some types of closure (typically those returned from
invokations of other closures) cannot be invoked.

closure { definitions } is short for closure({definitions}).

The main body of secnet, and all the additional modules, predefine
some keys in the root dictionary.  The main ones are:

  yes, true, True, TRUE, on:   the boolean value True
  no, false, False, FALSE, off: the boolean value False
  makelist:   turns a dictionary (arg1) into a list of definitions
              (ignoring the keys)
  readfile:   reads a file (arg1) and returns it as a string
  map:        applies the closure specified as arg1 to each of the
              remaining elements in the list in turn.  Returns a list
              made up of the outputs of the closure.

Keys defined by modules are described below, in the module
documentation.

Other configuration files can be included inline by writing "include
filename" at the start of a line.

After the configuration file is read, secnet looks for particular keys
in configuration space to tell it what to do:

 system: a dictionary which can contain the following keys:
   log (log closure): a destination for system messages
   userid (string): the userid for secnet to run as once it drops privileges
   pidfile (string): where to store its PID
   
 sites: a list of closures of type 'site', which define other tunnel
        endpoints that secnet will attempt to communicate with

* secnet command line options

Usage: secnet [OPTION]...

  -f, --silent, --quiet   suppress error messages
  -w, --nowarnings        suppress warnings
  -v, --verbose           output extra diagnostics
  -c, --config=filename   specify a configuration file
  -j, --just-check-config stop after reading configfile
  -n, --nodetach          do not run in background
  -d, --debug=item,...    set debug options
      --help              display this help and exit
      --version           output version information and exit

* secnet builtin modules

** resolver

Defines:
  adns (closure => resolver closure)

adns: dict argument
  config (string): optional, a resolv.conf for ADNS to use

** random

Defines:
  randomsrc (closure => randomsrc closure)

randomsrc: string[,bool]
  arg1: filename of random source
  arg2: if True then source is blocking

** udp

Defines:
  udp (closure => comm closure)

udp: dict argument
  address (string list): IPv6 or IPv4 addresses to listen and send on;
   default is all local addresses
  port (integer): UDP port to listen and send on; optional if you
   don't need to have a stable address for your peers to talk to
   (in which case your site ought probably to have `local-mobile true').
  buffer (buffer closure): buffer for incoming packets
  authbind (string): optional, path to authbind-helper program

** polypath

Defines:
  polypath (closure => comm closure)

polypath: dict argument
  port (integer): UDP port to listen and send on
  buffer (buffer closure): buffer for incoming packets
  authbind (string): optional, path to authbind-helper program
  max-interfaces (number): optional, max number of different interfaces to
   use (also, maximum steady-state amount of packet multiplication)
  interfaces (string list): which interfaces to process; each entry is
   optionally `!' or `+' followed by a glob pattern (which is applied to a
   prospective interface using fnmatch with no flags).  If no list is
   specified, or the list ends with a `!' entry, a default list is
   used/appended: "!tun*","!tap*","!sl*","!userv*","!lo","*".  Patterns
   which do not start with `*' or an alphanumeric need to be preceded
   by `!' or `+'.
  monitor-command (string list): Program to use to monitor appearance
   and disappearance of addresses on local network interfaces.  Should
   produce lines of the form `+|-<ifname> 4|6 <addr>' where <addr> is
   an address literal.  Each - line should relate to a previously
   printed + line.  On startup, should produce a + line for each
   currently existing address.  secnet does filtering so there is no
   need to strip out tun interfaces, multicast addresses, and so on.
   The command is run as the user secnet is started as (not the one
   which secnet may drop privilege to due to the configured `userid').
   The default depends on the operating system.
  permit-loopback (boolean): Normally, loopback IPv6 and IPv4
   addresses on local interfaces are disregarded, because such
   interfaces are not interesting for communicating with distant
   hosts.  Setting this option will ignore that check, which can be
   useful for testing.  Setting this option also removes "!lo*" from
   the default interface pattern list.

When using this comm, packets are sent out of every active interface
on the host (where possible).  It is important that interfaces created
by secnet itself are not included!  secnet's default filter list tries
to do this.

This comm only makes sense for sites which are mobile.  That is, the
site closures used with this comm should all have the `local-mobile'
parameter set to `true'.  When the local site site is not marked
mobile the address selection machinery might fixate on an unsuitable
address.

For an interface to work with polypath, it must either have a suitable
default route, or be a point-to-point interface.  In the general case
this might mean that the host would have to have multiple default
routes.  However in practice the most useful configuration is two
interfaces being (1) wifi (2) mobile internet.

I have had success on Linux by using network-manager for wifi and
invoking ppp directly for mobile internet.  ppp sets up a
point-to-point link, and does not add a default route if there already
is one.  network-manager always sets up a default route.  The result
is that the wifi always has a default route (so is useable); ppp
(being a point-to-point link) does not need one.

The use of polypath requires that secnet be started with root
privilege, to make the setsockopt(,,SO_BINDTODEVICE,) calls.  If the
configuration specifies that secnet should drop privilege (see
`userid' above), secnet will keep a special process around for this
purpose; that process will handle local network interface changes but
does not deal with any packets, key exchange, etc.

polypath support is only available when secnet is built against an
IPv6-capable version of adns (because it wants features in the newer
adns).

** log

Defines:
  logfile (closure => log closure)
  syslog (closure => log closure)

logfile: dict argument
  filename (string): where to log to
  class (string list): what type of messages to log
    { "debug-config", M_DEBUG_CONFIG },
    { "debug-phase", M_DEBUG_PHASE },
    { "debug", M_DEBUG },
    { "all-debug", M_DEBUG|M_DEBUG_PHASE|M_DEBUG_CONFIG },
    { "info", M_INFO },
    { "notice", M_NOTICE },
    { "warning", M_WARNING },
    { "error", M_ERROR },
    { "security", M_SECURITY },
    { "fatal", M_FATAL },
    { "default", M_WARNING|M_ERROR|M_SECURITY|M_FATAL },
    { "verbose", M_INFO|M_NOTICE|M_WARNING|M_ERROR|M_SECURITY|M_FATAL },
    { "quiet", M_FATAL }

logfile will close and reopen its file upon receipt of SIGHUP.

syslog: dict argument
  ident (string): include this string in every log message
  facility (string): facility to log as
    { "authpriv", LOG_AUTHPRIV },
    { "cron", LOG_CRON },
    { "daemon", LOG_DAEMON },
    { "kern", LOG_KERN },
    { "local0", LOG_LOCAL0 },
    { "local1", LOG_LOCAL1 },
    { "local2", LOG_LOCAL2 },
    { "local3", LOG_LOCAL3 },
    { "local4", LOG_LOCAL4 },
    { "local5", LOG_LOCAL5 },
    { "local6", LOG_LOCAL6 },
    { "local7", LOG_LOCAL7 },
    { "lpr", LOG_LPR },
    { "mail", LOG_MAIL },
    { "news", LOG_NEWS },
    { "syslog", LOG_SYSLOG },
    { "user", LOG_USER },
    { "uucp", LOG_UUCP }

** util

Defines:
  sysbuffer (closure => buffer closure)

sysbuffer: integer[,dict]
  arg1: buffer length
  arg2: options:
    lockdown (boolean): if True, mlock() the buffer

** site

Defines:
  site (closure => site closure)

site: dict argument
  local-name (string): this site's name for itself
  name (string): the name of the site's peer
  link (netlink closure)
  comm (one or more comm closures): if there is more than one, the
   first one will be used for any key setups initiated by us using the
   configured address.  Others are only used if our peer talks to
   them.
  resolver (resolver closure)
  random (randomsrc closure)
  local-key (rsaprivkey closure)
  address (string list): optional, DNS name(s) used to find our peer;
    address literals are supported too if enclosed in `[' `]'.
  port (integer): mandatory if 'address' is specified: the port used
    to contact our peer
  key (rsapubkey closure): our peer's public key
  transform (transform closure): how to mangle packets sent between sites
  dh (dh closure)
  hash (hash closure)
  key-lifetime (integer): max lifetime of a session key, in ms
    [one hour; mobile: 2 days]
  setup-retries (integer): max number of times to transmit a key negotiation
    packet [5; mobile: 30]
  setup-timeout (integer): time between retransmissions of key negotiation
    packets, in ms [2000; mobile: 1000]
  wait-time (integer): after failed key setup, wait this long (in ms) before
    allowing another attempt [20000; mobile: 10000]
  renegotiate-time (integer): if we see traffic on the link after this time
    then renegotiate another session key immediately (in ms)
    [half key-lifetime, or key-lifetime minus 5 mins (mobile: 12 hours),
     whichever is longer].
  keepalive (bool): if True then attempt always to keep a valid session key.
    Not actually currently implemented. [false]
  log-events (string list): types of events to log for this site
    unexpected: unexpected key setup packets (may be late retransmissions)
    setup-init: start of attempt to setup a session key
    setup-timeout: failure of attempt to setup a session key, through timeout
    activate-key: activation of a new session key
    timeout-key: deletion of current session key through age
    security: anything potentially suspicious
    state-change: steps in the key setup protocol
    packet-drop: whenever we throw away an outgoing packet
    dump-packets: every key setup packet we see
    errors: failure of name resolution, internal errors
    peer-addrs: changes to sets of peer addresses (interesting for mobile peers)
    all: everything (too much!)
  mobile (bool): if True then peer is "mobile" ie we assume it may
    change its apparent IP address and port number without either it
    or us being aware of the change; so, we remember the last several
    port/addr pairs we've seen and send packets to all of them
    (subject to a timeout).  We maintain one set of addresses for key
    setup exchanges, and another for data traffic. Two communicating
    peers must not each regard the other as mobile, or all the traffic
    in each direction will be triplicated (strictly, transmitted
    mobile-peers-max times) and anyway two peers whose public contact
    address may suddenly change couldn't communicate reliably because
    their contact addresses might both change at once. [false]
  mobile-peers-max (integer): Maximum number of peer port/addr pairs we
    remember and send to.  Must be at least 1 and no more than 5.
    [4 if any address is configured, otherwise 3]
  static-peers-max (integer): Maximum number of peer port/addr pairs
    we can try for a static site.  Must be at least 1 and no more
    than 5.  [4 or 3, as above]
  mobile-peer-expiry (integer): For "mobile" peers only, the length
    of time (in seconds) for which we will keep sending to multiple
    address/ports from which we have not seen incoming traffic. [120]
  local-mobile (bool): if True then other peers have been told we are
    "mobile".  This should be True iff the peers' site configurations
    for us have "mobile True" (and if we find a site configuration for
    ourselves in the config, we insist on this).  The effect is to
    check that there are no links both ends of which are allegedly
    mobile (which is not supported, so those links are ignored) and
    to change some of the tuning parameter defaults. [false]
  mtu-target (integer): Desired value of the inter-site MTU for this
    peering.  This value will be advertised to the peer (which ought
    to affect incoming packets), and if the peer advertises an MTU its
    value will be combined with this setting to compute the inter-site
    MTU.  (secnet will still accept packets which exceed the
    (negotiated or assumed) inter-site MTU.)  Setting a lower
    inter-site MTU can be used to try to restrict the sizes of the
    packets sent over the underlying public network (e.g. to work
    around network braindamage).  It is not normally useful to set a
    larger value for mtu-target than the VPN's general MTU (which
    should be reflected in the local private interface MTU, ie the mtu
    parameter to netlink).  If this parameter is not set, or is set
    to 0, the default is to use the local private link mtu.

Links involving mobile peers have some different tuning parameter
default values, which are generally more aggressive about retrying key
setup but more relaxed about using old keys.  These are noted with
"mobile:", above, and apply whether the mobile peer is local or
remote.

** transform-eax

Defines:
   eax-serpent (closure => transform closure)

** transform-cbcmac

Defines:
  serpent256-cbc (closure => transform closure)

** netlink

Defines:
  null-netlink (closure => closure or netlink closure)

null-netlink: dict argument
  name (string): name for netlink device, used in log messages
  networks (string list): networks on the host side of the netlink device
  remote-networks (string list): networks that may be claimed
    by the remote site using this netlink device
  local-address (string): IP address of host's tunnel interface
  secnet-address (string): IP address of this netlink device
  ptp-address (string): IP address of the other end of a point-to-point link
  mtu (integer): MTU of host's tunnel interface

Only one of secnet-address or ptp-address may be specified.  If
point-to-point mode is in use then the "routes" option must also be
specified, and netlink returns a netlink closure that should be used
directly with the "link" option to the site closure.  If
point-to-point mode is not in use then netlink returns a closure that
may be invoked using a dict argument with the following keys to yield
a netlink closure:
  routes (string list): networks reachable down the tunnel attached to
    this instance of netlink
  options (string list):
    allow-route: allow packets coming from this tunnel to be routed to
      other tunnels as well as the host (used for mobile devices like laptops)
    soft: remove these routes from the host's routing table when
      the tunnel link quality is zero
  mtu (integer): MTU of host's tunnel interface

Netlink will dump its current routing table to the system/log on
receipt of SIGUSR1.

** slip

Defines:
  userv-ipif (closure => netlink closure)

userv-ipif: dict argument
  userv-path (string): optional, where to find userv ["userv"]
  service-user (string): optional, username for userv-ipif service ["root"]
  service-name (string): optional, name of userv-ipif service ["ipif"]
  buffer (buffer closure): buffer for assembly of host->secnet packets
 plus generic netlink options, as for 'null-netlink'

** tun

Defines:
  tun (closure => netlink closure) [only on linux-2.4]
  tun-old (closure => netlink closure)

tun: dict argument
  flavour (string): optional, type of TUN interface to use
    ("guess","linux","bsd","streams")
  device (string): optional, path of TUN/TAP device file ["/dev/net/tun"]
  interface (string): optional, name of tunnel network interface
  ifconfig-path (string): optional, path to ifconfig command
  route-path (string): optional, path to route command
  ifconfig-type (string): optional, how to perform ifconfig
  route-type (string): optional, how to add and remove routes
   types are: "guess", "ioctl", "bsd", "linux", "solaris-2.5"
  buffer (buffer closure): buffer for host->secnet packets
 plus generic netlink options, as for 'null-netlink'

I recommend you don't specify the 'interface' option unless you're
doing something that requires the interface name to be constant.

** rsa

Defines:
  rsa-private (closure => rsaprivkey closure)
  rsa-public (closure => rsapubkey closure)

rsa-private: string[,bool]
  arg1: filename of SSH private key file (version 1, no password)
  arg2: whether to check that the key is usable [default True]

rsa-public: string,string
  arg1: encryption key (decimal)
  arg2: modulus (decimal)

** dh

Defines:
  diffie-hellman (closure => dh closure)

diffie-hellman: string,string[,bool]
  arg1: modulus (hex)
  arg2: generator (hex)
  arg3: whether to check that the modulus is prime [default True]

** md5

Defines:
  md5 (hash closure)

** sha1

Defines:
  sha1 (hash closure)

** conffile

Defines:
  makelist (dictionary => list of definitions)
  readfile (string => string)
  map (closure,list => list)

makelist: dictionary
  returns a list consisting of the definitions in the dictionary. The keys
  are discarded.

readfile: string
  reads the named file and returns its contents as a string

map:
  applies the closure specified as arg1 to each of the elements in the list.
  Returns a list made up of the outputs of the closure.