X-Git-Url: http://www.chiark.greenend.org.uk/ucgi/~ian/git?p=cgi-auth-flexible.git;a=blobdiff_plain;f=caf.pod;h=fcb061521d4858218e432dd3cbd31f83d90bbb5d;hp=aa9566be42009f0e742c4a8b4b3d3917b04ea6ba;hb=8fdc8b3c4aafee033887c600384a7a78521b9a18;hpb=8a20940a918ec4b98c4a38c295dda6488b6f4ece

diff --git a/caf.pod b/caf.pod
index aa9566b..fcb0615 100644
--- a/caf.pod
+++ b/caf.pod
@@ -7,7 +7,7 @@ CGI::Auth::Flexible - web authentication optionally using cookies
 =head1 SYNOPSYS
 
  my $verifier = CGI::Auth::Flexible->new_verifier(setting => value,...);
- my $authreq = $verifier->new_request($cgi_query_object);
+ my $authreq = $verifier->new_request($cgi);
 
  # simple applications
  $authreq->check_ok() or return;
@@ -102,6 +102,11 @@ B<SECURITY>: If you do not override the source provision facility (see
 L</SOURCE CODE DOWNLOAD>), check that the assumptions it makes aren't
 going to leak security-critical data.
 
+=item *
+
+Set up HTTPS on your webserver, or set the C<encrypted_only> setting
+to a false value.
+
 =back
 
 These points will now be covered in more detail.
@@ -115,7 +120,7 @@ This call can be expensive and is best amortised.
 
 The resulting verifier object can be used to process individual
 requests, in each case with
-C<< $authreq = CGI::Auth::Flexible->new_request($cgi_query) >>.
+C<< $authreq = CGI::Auth::Flexible->new_request($cgi) >>.
 
 See L</SETTINGS>.
 
@@ -326,9 +331,9 @@ Initialises an instance and returns a verifier object.
 The arguments are setting pairs like a hash initialiser.
 See L</SETTINGS> below.
 
-=item C<< $verifier->new_request($cgi_query) >>
+=item C<< $verifier->new_request($cgi) >>
 
-Prepares to process a request.  I<$cgi_query> should normally
+Prepares to process a request.  I<$cgi> should normally
 be the query object from L<CGI(3perl)>.  Most of the default
 hook methods assume that it is; however if you replace enough of
 the hook methods then you can pass any value you like and it
@@ -362,7 +367,7 @@ Discards the resources (open files, etc.) in the verifier object.
 
 =back
 
-=head REQUEST-RELATED FUNCTIONS AND METHODS
+=head1 REQUEST-RELATED FUNCTIONS AND METHODS
 
 All of these are only valid after C<check_divert> or C<check_ok> has
 been called.  (In the case of C<check_ok> it won't normally be sensible
@@ -404,18 +409,19 @@ whenever it is handling anything except an HTML page loads, it must
 call this function.  See L</REQUEST TYPES>, and
 L<GENERATING URLS, FORMS AND AJAX QUERIES>.
 
-C<check_mutate> will either return successfully, indicating that all
+C<check_nonpage> will either return successfully, indicating that all
 is well and the request should proceed, or it will die, like
 C<check_mutate>.
 
-=head RESPONSE-RELATED FUNCTIONS AND METHODS
+=head1 RESPONSE-RELATED FUNCTIONS AND METHODS
 
 =item C<< $authreq->url_with_query_params($params, [$nonpagetype]) >>
 
 Convenience function which returns a url for a GET request to this
 application.
 
-I<$params> is a hashref specifying the parameters and the PATH_INFO.
+I<$params> is a hashref specifying the parameters and the PATH_INFO
+(not including any parameters related to CGI::Auth::Flexible).
 The keys are the parameter names, and the values are array refs with
 the parameter value(s) (as strings, as yet unquoted).  (They are array
 refs because it is possible to pass multiple values for the same
@@ -456,18 +462,16 @@ Returns the value of the secret cookie.  CGI::Auth::Flexible sets this
 cookie in the forms generated by C<check_ok>.  You may also set it
 yourself (and indeed you must do so if you use C<check_divert>).
 
-=item C<< $authreq->chain_params() >>
+item C<< $authreq->_chain_params() >>
 
 Returns a hash of the "relevant" parameters to this request, in a form
-used by XXX.  This is all of the query parameters which are not
-related to CGI::Auth::Flexible.  The PATH_INFO from the request is
-returned as the parameter C<< '' >>.
-
-xxx why use this function
+suitable for C<url_with_query_params>.  This is all of the query
+parameters which are not related to CGI::Auth::Flexible.  The
+PATH_INFO from the request is returned as the parameter C<< '' >>.
 
 =back
 
-=head OTHER FUNCTIONS AND METHODS
+=head1 OTHER FUNCTIONS AND METHODS
 
 =over
 
@@ -503,7 +507,22 @@ class or due to previous requests on the same verifier.)
 
 See L</REQUEST TYPES>.
 
-=item C<< $verifier_or_authreq->($data) | CGI::Auth::Flexible->>>
+=item C<< CGI::Auth::Flexible::srcdump_dir_cpio($cgi,$verifier,$dumpdir,$dir,$outfn,$how,$script) >>
+
+Helper function for implementing the C<srcdump_process_item> hook.
+Generates a tarball using cpio and includes it in the prepared source
+code distribution.
+
+The arguments are mostly the same as for that hook.  C<$dir> is the
+root directory at which to start the archive.  C<$how> is a short text
+string which will be mentioned in the log.
+
+C<$script> is a shell script fragment which must output a
+nul-separated list of filenames (e.g. the output of C<find -print0>).
+It is textually surrounded by C<( )> and will be executed with C<set -e>
+in force.  Its cwd will be C<$dir>.
+
+=item C<< $verifier_or_authreq->($data) | CGI::Auth::Flexible-> >>
 
 Hashes the supplied data using the hash function specified by the
 C<hash_algorithm> setting, and converts the result to a string of hex
@@ -511,6 +530,198 @@ digits.
 
 =back
 
+=head1 REQUEST TYPES
+
+The C<$reqtype> values understood by C<check_nonpage> are strings.
+They are:
+
+=over
+
+=item C<PAGE>
+
+A top-level HTML page load.  May contain confidential information for
+the benefit of the logged-in user.
+
+=item C<FRAME>
+
+An HTML frame.  May contain confidential information for
+the benefit of the logged-in user.
+
+=item C<IFRAME>
+
+An HTML iframe.  May contain confidential information for
+the benefit of the logged-in user.
+
+=item C<SRCDUMP>
+
+Source dump request, whether for the licence or actual source code
+tarball; returned value is not secret.
+
+=item C<STYLESHEET>
+
+CSS stylesheet.  B<MUST NOT> contain any confidential data.  If the
+stylesheet depends on the user, then attackers may be able to
+determine what stylesheet the user is using.  Hopefully this is not a
+problem.
+
+=item C<FAVICON>
+
+"Favicon" - icon for display in the browser's url bar etc.  We aren't
+currently aware of a way that attackers can get a copy of this.
+
+=item C<ROBOTS>
+
+C<robots.txt>.  Should not contain any confidential data (obviously).
+
+=item C<IMAGE>
+
+Inline image, for an C<< <img src=...> >> element.
+
+Unfortunately it is not possible to sensibly show top-level
+confidential images (that is, have the user's browser directly visit a
+url which resolves to an image rather than an HTML page with an inline
+image).  This is because images need to have a per-session hidden form
+parameter to avoid cross-site scripting, which breaks bookmarks etc.
+
+=item C<SCRIPT>
+
+JavaScript for a C<< <script> >> element.  (Possibly confidential for
+the user.)
+
+=item C<AJAX-XML>
+
+C<< XMLHttpRequest >> returning XML data.  (Possibly
+confidential for the user.)
+
+=item C<AJAX-JSON>
+
+C<< XMLHttpRequest >> returning JSON data.  (Possibly
+confidential for the user.)
+
+=item C<AJAX-OTHER>
+
+C<< XMLHttpRequest >> returning data of some other kind.  (Possibly
+confidential for the user.)
+
+=back.
+
+=head1 DIVERT SPEC
+
+The return value from C<check_divert> indicates how the request should
+be handled.  It is C<undef> if all is well and the user is logged in.
+
+Otherwise the return value is a hash ref with the following keys:
+
+=over
+
+=item C<Kind>
+
+Scalar string indicating the kind of diversion required.
+
+=item C<Message>
+
+Scalar string for display to the user in relation to the diversion.
+Has already been translated.  In HTML but normally does not contain
+any tags.
+
+=item C<CookieSecret>
+
+The login cookie which should be set along with whatever response is
+sent to the client.  The value in the hash is the actual value
+of the cookie as a string.  C<undef> means no cookie setting header
+should be sent; C<''> means the cookie should be cleared.
+
+=item C<Params>
+
+The extra hidden form parameters (and the C<PATH_INFO>) which should
+be set when the subsequent request bounces back from the client, in
+the form used by C<url_with_query_params>.
+
+The contents of this hashref does not include the CAF-specific
+parameters such as the secret cookie, those which follow from the kind
+of diversion requested, etc.
+
+It is correct to always include the contents of C<Params> as hidden
+parameters in the urls for all redirections, and as hidden input
+fields in all generated forms.  The specific cases where C<Params> is
+currently relevant are also mentioned in the text for each divert
+kind.
+
+=back
+
+The values of C<Kind> are:
+
+=over
+
+=item C<SRCDUMP->I<item>
+
+We should respond by sending our application source code.  I<item>
+(which will contain only word characters, and no lower case) is the
+specific item to send, normally C<SOURCE> or C<LICENCE>.
+
+=item C<REDIRECT-HTTPS>
+
+We should respond with an HTTP redirect to the HTTPS instance of our
+application.
+
+=item C<REDIRECT-LOGGEDOUT>
+
+We should redirect to a page showing that the user has been logged
+out.  (Ie, to a url with one of the the C<loggedout_param_names> set.)
+
+=item C<SMALLPAGE-LOGGEDOUT>
+
+We should generate a page showing that the user has been logged out.
+There can be a link on the page pointing to the login page so that the
+user can log back in.
+
+=item C<SMALLPAGE-NOCOOKIE>
+
+We should generate a page reporting that the user does not have
+cookies enabled.  It should probably contain a link pointing to the
+login page with additionally all the parameters in C<Params>.  When
+this divert spec is generated, C<Message> will explain the problem
+with cookies so there is no need to do that again in the page body if
+you include the contents of C<Message>.
+
+=item C<LOGIN-STALE>
+
+The user's session was stale (this is described in C<Message>).  We
+should generate a login form.
+
+=item C<LOGIN-BAD>
+
+The user supplied bad login credentials.  The details are in
+C<Message>.  We should generate a login form (with additionally the
+parameters from C<Params> as hidden fields).
+
+=item C<LOGIN-INCOMINGLINK>
+
+We should generate a login form (with the specified parameters); the
+user is entering the site via a cross-site link but is not yet logged
+in.
+
+=item C<LOGIN-FRESH>
+
+We should generate a login form.  The user is not yet logged in.
+
+=item C<REDIRECT-LOGGEDIN>
+
+We should redirect to our actual application, with the specified
+parameters.  (The user has just logged in.)
+
+=item C<MAINPAGEONLY>
+
+We should generate our main page but B<ignoring all form parameters>
+and B<ignoring the path_info>.  Most applications will find this
+difficult to implement.
+
+An alternative is to generate a small page with a form or link which
+submits our own main page without any parameters.
+
+(Applications which set C<promise_check_mutate> do not see this divert
+kind.)
+
 =head1 SETTINGS
 
 C<new_verifier> and C<new_request> each take a list of settings, as
@@ -520,6 +731,23 @@ The settings supplied to C<new_verifier> are stored in the verifier
 and will apply to all authreqs made from it unless overridden in the
 call to C<new_request>
 
+When a setting is described as a hook function, it should be a
+coderef.  The first argument will be the query object from
+L<CGI(3perl)> (strictly, it will be whatever value was passed to
+C<new_request>).  The second argument will be the authreq object (the
+return value from C<new_request>).
+Ie, C<< sub some_hook ($$...) { my ($cgi,$authreq,@stuff) = @_ ... >>
+
+In bullet point headings, the hook functions are shown in the form
+C<< some_hook($cgi,$authreq,@stuff) >> even though this would not be
+legal syntax.  This should be read to mean that the
+%implicit_settings_hash{'some_hook'}($cgi,$authreq,@stuff)
+would be a legal call.  (However, the settings hash is not exposed.)
+
+When a hook's default implementation is mentioned and named, that
+function won't also be described in the section on the module's
+functions.
+
 =over
 
 =head2 GENERAL SETTINGS
@@ -532,7 +760,7 @@ path settings are relative values.
 
 Must be an absolute filename.
 
-=item C<assocdb_dbh>
+=item C<db_dbh>
 
 CGI::Auth::Flexible needs a database for recording users' login
 session.  This database needs to be shared across all instances of the
@@ -546,26 +774,26 @@ particular, there is no need for transactional integrity across
 changes made by CAF and your own application.)
 
 By default, CAF uses a sqlite3 database stored on local disk in the
-file named by C<assocdb_path>.  This will be suitable for all
+file named by C<db_path>.  This will be suitable for all
 applications which run on a single host.
 
 This value, if supplied, should be a DBI handle for the database.
 
-=item C<assocdb_dsn>
+=item C<db_dsn>
 
 This is the DSN to pass to C<< DBI->connect >>.  Used only if
-C<assocdb_dbh> is not supplied.
+C<db_dbh> is not supplied.
 
-=item C<assocdb_path>
+=item C<db_path>
 
 Path to the sqlite3 database used for CAF's session storage.  The
-default is currently C<caf-assocs.db> but will change in the future.
+default is C<caf.db>.
 
-Used only if neither C<assocdb_dbh> or C<assocdb_dsn> are supplied.
+Used only if neither C<db_dbh> or C<db_dsn> are supplied.
 
 If this is a relative path, it is in C<dir>.
 
-=item C<assocdb_table>
+=item C<db_prefix>
 
 Prefix for the SQL tables and indices to use (and to create, if
 necessary).
@@ -619,11 +847,6 @@ this often (in seconds).  Default: 86400.
 
 Name of the hidden form parameter.  Default: C<caf_assochash>.
 
-=item C<dummy_param_name_prefix>
-
-Some of CAF's HTML-generating functions need to invent form parameter
-names.  They will all start with this string.  Default: C<caf__>.
-
 =item C<cookie_name>
 
 Name of the cookie used for login sessions.  Default:
@@ -636,16 +859,6 @@ Name of the password field in the login form.  Default: C<password>.
 Used by C<login_ok_password> (the default C<login_ok> hook),
 C<gen_plain_login_form> and the default C<is_login> hook.
 
-=item C<srcdump_param_name>
-
-Form parameter name used to indicate that this is a source download
-request.  If this parameter is supplied, C<check_ok> and
-C<check_divert> will arrange for the applicaton source code to be
-delivered as the response (in C<check_ok>'s case by doing it itself
-and in C<check_divert>'s case by asking your application to do so.
-
-Default is C<caf_srcdump>.
-
 =item C<username_param_names>
 
 Arrayref of name(s) of username form parameters.
@@ -672,12 +885,222 @@ C<< <input type="submit" name="caf_logout" ...>>
 
 The default is C<['caf_logout']>
 
+=item C<logged_param_names>
+
+Arrayref of name(s) of form parameters indicating that user has just
+logged out.  (During the logout process, the actual logout action is a
+POST request, whose response redirects to the "you have been logged
+out" page; these form parameters are for this second page.)
+
+Used by the default C<is_loggedout> hook.
+
+The first entry is used by C<check_ok> to generate the redirection.
+
+The default is C<['caf_loggedout']>
+
+=item C<promise_check_mutate>
+
+Boolean.  If true, is a declaration by the application that it is
+mutatin-aware.  See L</MUTATING OPERATIONS AND EXTERNAL LINKS>.
+
+The default is 0.
+
+=item C<encrypted_only>
+
+Boolean.  If true, CAF will insist that all transactions be done over
+an encrypted http connection.  It will redirect unencrypted requests
+to the https instance of the applicattion, and will set the encrypted
+only flag on its cookie.
+
+The default is 1.
+
+=back
+
+=item C<< get_url($cgi,$authreq) >>
+
+Hook which returns the URL of this web application.  By default, we
+call C<< $cgi->url() >> for each request, but you can fix this if you
+prefer.
+
+=item C<< is_login,is_logout,is_loggedout($cgi,$authreq) >>
+
+Hook which returns a boolean indicating whether the request was,
+respectively: a login form submission (ie, username and password); a
+logout request (submission resulting from the user pressing the
+"logout" button); "logged out" page (redirection from the logout
+POST).
+
+The default is to check whether any of the corresponding request
+parameters (C<< login_param_names >> etc.) was supplied, using the
+C<get_param> hook.
+
+=back
+
+=head2 SETTINGS (HOOKS) RELATED TO THE CGI REQUEST OBJECT
+
+=over
+
+=item C<< get_param($cgi,$authreq,$param) >>
+
+Returns the value of a single-valued form parameter.
+The default is to call C<< $cgi->param($param) >>.
+The semantics are the same as that of C<CGI::param>.
+
+=item C<< get_params($cgi,$authreq) >>
+
+Returns a hash of the parameters.  The return value is a hashref whose
+keys are the parameter names and whose values are arrayrefs, one entry
+in the arrayref for each value.
+
+The default is to call C<< $cgi->Vars() >>, expect the
+results to look like those from C<CGI::Vars>, and massage them into
+the required form with split.
+
+=item C<< get_path_info($cgi,$authreq) >>
+
+Returns the PATH_INFO of the request.  The default is to
+call C<< $cgi->path_info() >>.
+
+=item C<< get_cookie($cgi,$authreq) >>
+
+Returns the value of the CAF cookie sent with the request, or undef if
+none was supplied.  The default is to call C<<
+$cgi->cookie($cookie_name) >> (where C<$cookie_name> is from the
+setting of the same name).  The return value should be the scalar
+value of the cookie.
+
+=item C<< get_method($cgi,$authreq) >>
+
+Returns the HTTP method as a string.  The default is to call
+C<< $cgi->request_method() >>.
+
 =back
 
+=item C<< is_https($cgi,$authreq) >>
+
+Returns a boolean indicating whether the request was over an encrypted
+channel.  The default is C<< !!$cgi->https() >>.  See C<encrypted_only>.
+
+=back
+
+=head2 SETTINGS RELATED TO HTML GENERATION
+
+These are only used if you call C<check_ok> (or other functions
+mentioned in this section).
+
+Settings whose names are of the form C<gen_...> are hooks which each
+return an array of strings, normally HTML strings, for use by
+C<check_ok> (or, in turn, other hooks, or your application).  These
+are often documented simply by showing the output produced.  In many
+cases parts of the output are in turn obtained from other hooks.  In
+some cases the default implementations have been given names for
+convenient use by your application.  They will be called in array
+context.
+
+We'll write C<gettext(something)> even though actually there is a hook
+to control the translation function used.
+
+=over
+
+=item C<handle_divert>($cgi,$authreq,$divert))
+
+C<check_ok> calls this hook before producing output of its own.  If
+you want to handle some but not all diversions yourself, you may set
+this hook.  The hook should either do nothing and return false, or
+return true if it has handled the request (or arrange for the request
+to be handled).  If the hook returns true then C<check_ok> simply
+returns 0.
+
+=item C<gen_login_form>($cgi,$authreq,$divert))
+
+Default: a table (used mostly for layout) containing input fields for
+a login form.  Must be within a C<< <form> >> element, but doesn't
+generate it.  Has text fields for every entry in
+C<username_param_names> (in each case associated with a description
+C<< gettext(ucfirst $parameter_name) >>, a password field (with
+description C<gettext("Password")>, and a login submit button (with
+description C<gettext("Login")>.
+
+Default is available as the module function C<gen_plain_login_form>.
+
+=item C<gen_login_link>($cgi,$authreq))
+
+Default:
+
+ <a href="http:...">gettext(Log in again to continue.)</a>
+
+Default is available as the module function C<gen_plain_login_link>.
+
+=item C<gen_postmainpage_form>($cgi,$authreq,$params))
+
+Default: form contents (but not the C<< <form> >> element):
+
+C<$params> (in the form returned by the C<get_params> hook) as hidden
+fields, and also:
+
+ <input type="submit" ... value=getext('Continue')>
+
+Default is available as the module function C<gen_postmainpage_form>.
+
+=item C<gen_start_html>($cgi,$authreq,$title)
+
+Default: C<$cgi->start_html($title)>
+
+=item C<gen_end_html>($cgi,$authreq,$title)
+
+Default: C<$cgi->end_html($title)>
+
+=item C<gen_footer_html>($cgi,$authreq)>
+
+Default:
+
+ <hr><address>
+ Powered by Free / Libre / Open Source Software
+ according to the [gen_licence_link_html].
+ [gen_source_link_html].
+ </address>
+
+Default is available as the module function C<gen_plain_footer_html>.
+
+=item C<gen_licence_link_html>($cgi,$authreq)>
+
+Default: uses C<url_with_query_params> to generate a URL for
+downloading the licence, and returns:
+  <a href="...">GNU Affero GPL</a>
+
+Default is available as the module function C<gen_plain_licence_link_html>.
+
+=item C<gen_source_link_html>($cgi,$authreq)>
+
+Default: uses C<url_with_query_params> to generate a URL for
+downloading the source, and returns:
+  <a href="...">Source available</a>
+
+Default is available as the module function C<gen_plain_source_link_html>.
+
+=item C<form_entry_size>
+
+Size of generated text entry fields.  Default is 60.
+
+=item C<dummy_param_name_prefix>
+
+Some of CAF's HTML-generating functions need to invent form parameter
+names.  They will all start with this string.  Default: C<caf__>.
+
 =head2 SETTINGS FOR SOURCE CODE DOWNLOAD FACILITY
 
 =over
 
+=item C<srcdump_param_name>
+
+Form parameter name used to indicate that this is a source download
+request.  If this parameter is supplied, C<check_ok> and
+C<check_divert> will arrange for the applicaton source code to be
+delivered as the response (in C<check_ok>'s case by doing it itself
+and in C<check_divert>'s case by asking your application to do so.
+
+Default is C<caf_srcdump>.
+
 =item C<srcdump_path>
 
 Path to the directory used for storing pre-prepared source downloads.
@@ -685,19 +1108,209 @@ Defaults to C<caf-srcdump>.
 
 If this is a relative path, it is in C<dir>.
 
-=back
+=item C<srcdump_dump($cgi,$authreq,$srcobj)>
+
+Dump the source code (C<$srcobj='source'> or licence data
+(C<$srcobj='licence'>).  The default implementation checks that
+C<$srcobj> has reasonable syntax and uses the files C<$srcobj.data>
+and C<$srcobj.ctype> with the C<dump> hook.
+
+=item C<dump($cgi,$authreq,$contenttype,$datafilehandle)>
+
+Responds to the request by sending the contents of $datafilehandle
+(which should just have been opened) and specifying a content type of
+$contenttype.
+
+The default implmentation uses the C<print> hook, and also calls
+C<$cgi->header('-type' => $contenttype>, and is available as the
+module function C<dump_plain>.
+
+=item C<srcdump_prepare($cgi,$verifier)>
+
+Prepares the source code for download when requested.  Invoked by
+C<new_verifier>, always, immediately before it returns the
+just-created verifier object.
+
+The default implementation is the module function
+C<srcdump_dirscan_prepare>, which prepares a manifest, licence file
+and source code tarball of tarballs, as follows:
+
+It processes each entry in the return value from C<srcdump_listitems>.
+These are the software's include directories and any other directories
+containing source code.  It handles C<.> specially (see
+C<srcdump_filter_cwd>).
 
-=head2 SETTINGS FOR HTML GENERATION
+For each entry it looks, relative to that, for the licence as a file
+with a name mentioned in C<srcdump_licence_files>.  The first such
+file found is considered to be the licence.  It then calls the hook
+C<srcdump_process_item> for the entry.
 
-=over
+The licence, a manifest file, and all the outputs generated by the
+calls to C<srcdump_process_item>, are tarred up and compressed as a
+single source tarball.
 
-=item C<form_entry_size>
+It uses the directory named by C<srcdump_path> as its directory for
+working and output files.  It uses the filename patterns
+C<generate.*>, C<licence.*>, C<s.[a-z][a-z][a-z].*>, C<manifest.*>,
+C<source.*> in that directory.
 
-xxx
+=item C<srcdump_process_item>($cgi,$verifier,$dumpdir,$item,\&outfn,\$needlicence,\%dirsdone)>
 
-=back
+Processes a single include directory or software entry, so as to
+include the source code found there.  Called only by the default
+implementation of C<srcdump_prepare>.
+
+C<$dumpdir> is the directory for working and output files.  C<$item>
+is the real (no symlinks) absolute path to the item.
+
+C<\$needlicence> is a ref to a scalar: this scalar is undef if we have
+already found the licence file; otherwise it is the filename to which
+the licence should be copied.  If the referent is undef on entry,
+C<srcdump_process_item> needs to see if it finds the licence; if it
+does it should copy it to the named file and then set the scalar to
+undef.
+
+C<\%dirsdone> is a ref to the hash used by C<srcdump_prepare> to avoid
+including a single directory more than once.  If
+C<srcdump_process_item> decides to process a directory other than
+C<$item> it should check this hash with the real absolute path of the
+other directoy as a key: if the hash entry is true, it has already
+been done and should be skipped; otherwise the hash entry should be set.
+
+C<\&outfn> is a coderef which C<srcdump_process_item> should call each
+time it wants to generate a file which should be included as part of
+the source code.  It should be called using one of these patterns:
+   $outfn->("message for manifest");
+   $outfile = $outfn->("message for manifest", "extension");
+The former simply prints the message into the manifest in the form
+  none: message for manifest
+The latter generates and returns a filename which should then
+be created and filled with some appropriate data.  C<"extension">
+should be a string for the file extension, eg C<"txt">.  The output
+can be written directly to the named file: there is no need to
+write to a temporary file and rename.  C<$outfn> writes the filename
+and the message to the manifest, in the form
+  filename leaf: message
+In neither case is the actual name of C<$dir> on the system
+disclosed per se although of course some of the contents of some of
+the files in the source code dump may mention it.
+
+The default implementation is the module function
+C<srcdump_process_item>.
+
+It skips directories for which C<srcdump_system_dir> returns true.
+
+It then searches the item and its parent
+directories for a vcs metadata directory (one of the names in
+C<srcdump_vcs_dirs>); if found, it calls the C<srcdump_byvcs> hook
+(after checking and updaeing C<%dirsdone>).
+Otherwise it calls the C<srcdump_novcs> hook.
+
+=item C<srcdump_novcs($cgi,$verifier,$dumpdir,$item,$outfn)>
+
+Called by the default implementation of C<srcdump_process_item>, with
+the same arguments, if it doesn't find vcs metadata.
+
+The default implementation is the module function C<srcdump_novcs>.
+
+If C<$item> is a directory, it uses C<srcdump_dir_cpio> to prepare a
+tarball of all the files under C<$item> which have the world read bit
+set.  Directories are not included (and their permissions are
+disregarded).  The contents of C<srcdump_excludes> are excluded.
+
+If it's a plain file it uses C<srcdump_file> to include the file.
+
+=item C<srcdump_byvcs($cgi,$verifier,$dumpdir,$item,$outfn,$vcs)>
+
+Called by the default implementation of C<srcdump_process_item>, with
+the same arguments, if it finds vcs metadata.  The additional argument
+C<$vcs> is derived from the entry of C<srcump_vcs_dirs> which was
+used: it's the first sequence of word characters, lowercased.
+
+The default implementation is the module function C<srcdump_byvcs>.
+It simply calls C<srcdump_dir_cpio> with a script from the setting
+C<srcdump_vcsscript>.
+
+=item C<srcdump_vcs_dirs>
 
+Array ref of leaf names of vcs metadata directories.  Used by the
+default implementation of C<srcdump_process_item>.  The default value
+is C<['.git','.hg','.bzr','.svn']>.
 
+=item C<srcdump_vcs_script>
+
+Hash ref of scripts for generating vcs metadata.  Used by the default
+implementation of C<srcdump_byvcs>.  The keys are values of C<$vcs>
+(see C<srcdump_byvcs>); the values are scripts as for
+C<srcdump_dir_cpio>.
+
+The default has an entry only for C<git>:
+  git ls-files -z
+  git ls-files -z --others --exclude-from=.gitignore
+  find .git -print0
+
+=item C<srcdump_excludes>
+
+Array ref of exclude glob patterns, used by the default implementation
+of C<srcdump_novcs>.  The default value is C<['*~','*.bak','*.tmp','#*#']>.
+
+Entries must not contain C<'> or C<\>.
+
+=item C<srcdump_listitems($cgi,$verifier)>
+
+Returns an array of directories which might contain source code of the
+web application and which should be therefore be considered for
+including in the source code delivery.
+
+Used by the default implementation of C<srcdump_prepare>.
+
+Entries must be directories, plain files, or nonexistent; they may
+also be symlinks which resolve to one of those.
+
+If C<.> is included it may be treated specially - see
+C<srcdump_filter_cwd>.
+
+The default implementation returns 
+C<(@INC, $ENV{'SCRIPT_FILENAME'}, $0)>.
+
+=item C<srcdump_system_dir($cgi,$verifier,$dir)>
+
+Determines whether C<$dir> is a "system directory", in which any
+source code used by the application should nevertheless not be
+included in the source code dump.
+
+Used by the default implementation of C<srcdump_item>.
+
+The default implementation is as follows: Things in C</etc/> are
+system directories.  Things in C</usr/> are too, unless they are in
+C</usr/local/> or C</usr/lib/cgi*>.
+
+=item C<srcdump_filter_cwd>
+
+Boolean which controls the handling of C<.> if it appears in the
+return value from C<srcdump_listitems>.  Used only by the default
+implementation of C<srcdump_prepare>.
+
+If set to false, C<.> is treated normally and no special action is
+taken.
+
+However often the current directory may be C</>, or a data directory,
+or some other directory containing data which is confidential, or
+should not be included in the public source code distribution for
+other reasons.  And for historical reasons Perl has C<@INC> containing
+C<.> by default (which is arguably dangerous and wrong).
+
+So the default this setting is true, which has the following effects:
+
+C<.> is not searched for source code even if it appears in C<@INC>.
+C<.> is removed from C<@INC> and C<%INC> is checked to see if any
+modules appear to have already been loaded by virtue of C<.> appearing
+in C<@INC> and if they have it is treated as a fatal error.
+
+Only the literal string C<.> is affected.  If the cwd is included by
+any other name it is not treated specially regardless of this setting.
+
+=back
 
 =head1 DATABASE TABLES
 
@@ -706,19 +1319,41 @@ your application runs on multiple frontend hosts with a shared
 database, you may need to create for yourself the tables and indices
 used by CGI::Auth::Flexible.
 
+By default, every time CAF starts up, it attempts to execute certain
+fixed database statements to create the tables and indices it needs.
+These are run with C<$dbh->{PrintError}> set to 0.  The effect with
+sqlite (the default database) is that the tables and indices are
+created iff they do not already exist, and that no spurious errors are
+reported anywhere.
+
+If you use a different database, or just prefer to do things
+differently, you can set up the tables yourself and/or disable or
+modify the default setup statements, via the C<db_setup_stmts>
+setting.
+
+The tables needed are:
+
+
 xxx document _db_setup_do
 xxx make _db_setup_do explicitly overrideable
 
 
-xxx divert spec
-xxx reqtype
-xxx settings
-xxx html generators
-xxx document cookie
+xxx remaining settings
+ db_password
+ username_password_error
+ login_ok
+ get_cookie_domain
+ gettext
+ print
+ debug
+
+xxx document cookie usage
+xxx document construct_cookie fn
+
+xxx document @default_db_setup_statements
 
 xxx bugs wrong default random on Linux
 xxx bugs wrong default random on *BSD
 xxx bugs keys not shared should be in db
-xxx rename caf-assocs.db
 xxx rename caf_assocsecret default cookie name
 xxx mention relationship between login_timeout and cookies