.TH INNDUCT 8
.SH NAME
innduct \- quickly and reliably stream Usenet articles to remote site
.SH SYNOPSIS
.B innduct
.RI [ options ]
.I site
.RI [ fqdn ]
.SH DESCRIPTION
.B innduct
implements NNTP peer-to-peer news transmission including the streaming
extensions, for sending news articles to a remote site.

You need to run one instance of innduct for each peer site.  innduct
manages its interaction with innd, including flushing the feed as
appropriate, etc., so that articles are transmitted quickly, and
manages the retransmission of its own backlog.  innduct includes the
locking necessary to avoid multiple simutaneous invocations.

By default, innduct reads the default feedfile corresponding to
the site
.I site
(is
.IR pathoutgoing / site )
and feeds it via NNTP, streaming if possible, to the host
.IR fqdn .

If
.I fqdn
is not specified, it defaults to
.IR site .

innduct daemonises after argument parsing, and all logging (including
error messages) are sent to syslog (facility
.BR news ).

The best way to run innduct is probably to periodically invoke innduct
for each feed (e.g. from cron), passing innduct it the
.B \-q
option to arrange that it silently exits if an innduct is already
running for that site.
.SH INNDUCT VS INNFEED/NNTPSEND/INNXMIT
.TP
.B innfeed
does roughly the same thing as innduct.  However, the way it receives
information from innd can result in articles being lost (not offered
to peers) if innfeed crashes for any reason.  This is an inherent
defect in the innd channel feed protocol.  innduct uses a file feed,
constantly "tailing" the feed file, and where implemented uses
.BR inotify (2)
to reduce the latency which would come from having to constantly poll
the feed file.  innfeed is capable of feeding multiple peers from a
single innfeed instance, whereas each innduct process handles exactly
one peer.  innduct is much smaller and simpler, at 3kloc to innfeed's
25kloc.  innfeed needs a separate wrapper script or similar
infrastructure (of which there is an example in its manpage), whereas
innduct can be run directly and doesn't need help from shell scripts.
.TP
.B nntpsend
processes feed files in batch mode.  That is, you have to periodically
invoke nntpsend, and when you do, the feed is flushed and articles
which arrived before the flush are sent to the peer.  This introduces
a batching delay, and also means that the NNTP connection to the peer
needs to be remade at each batch.  nntpsend (which uses innxmit)
cannot make use of multiple connections to a single peer site.
However, nntpsend can be left to find automatically which sites need
feeding by looking in
.IR pathoutgoing .
.TP
.B innxmit
is the actual NNTP feeder program used by nntpsend.
.SH GENERAL OPTIONS
.TP
.BR \-f | \-\-feedfile= \fIfeedfile\fR
Specifies
.IR feedfile .
If the specified value ends in a
.B /
it is taken as a directory to use as if it were
.I pathoutgoing
and the actual feed file used is
.IR specified_feedfile / site .
.TP
.BR \-q | \-\-quiet-multiple
Makes innduct silently exit (with status 0) if another innduct holds
the lock for the site.  Without \fB-q\fR, this causes a fatal error to
be logged and a nonzero exit.
.TP
.BR \-\-no-daemon
Do not daemonise.  innduct runs in the foreground and all messages
(including all debug messages) are written to stderr.
.TP
.BI \-\-no-streaming
Do not try to use the streaming extensions to NNTP (for use eg if the
peer can't cope when we send MODE STREAM).
.TP
.BI \-\-inndconf= FILE
Read
.I FILE
instead of the default
.BR inn.conf .
This is currently only used if
.I feedfile
is not specified, to find the value
.I pathoutgoing
for constructing the feedfile name from the site name.
.TP
.BI \-\-port= PORT
Connect to port
.I PORT
at the remote site rather than to the NNTP port (119).
.TP
.BI \-\-help
Just print a brief usage message and list of the options to stdout.
.SH TUNING OPTIONS
You should not normally need to adjust these.  Time intervals may
specified in seconds, or as a number followed by one of the following
units:
.BR "s m h d" ,
.BR "sec min hour day" ,
.BR "das hs ks Ms" .
.TP
.BI \-\-max-connections= max
Restricts the maximum number of simultaneous NNTP connections used by
for each site to
.IR max .
The default is
.BR 10 .
There is no global limit on the number of connections.
.TP
.BI \-\-max-queue-per-conn= max
Restricts the maximum number of outstanding articles queued on any
particular connection
.IR max .
(Non-streaming connections can only handle one article at a time.)
The default is
.BR 200 .
.TP
.BI \-\-feedfile-flush-size= bytes
Specifies that innduct should flush the feed and start a new feedfile
when the existing feedfile size exceeds
.IR bytes ;
the effect is that the innduct will try to avoid the various
batchfiles growing much beyond this size while the link to the peer is
working.  The default is
.BR 100000 .
.TP
.BI \-\-period-interval= PERIOD-INTERVAL
Specifies wakup interval and period granularity.
innduct wakes up every PERIOD-INTERVAL to do various housekeeping
checks.  Also, many of the timeout and rescan intervals (those
specified in this manual as
.TP
.BI \-\-connection-timeout= TIME
How long to allow for a connection setup attempt before giving up.
The default is
.BR 200s .
.TP
.BI \-\-stuck-flush-timeout= TIME
How long to wait for innd to respond to a flush request before giving
up.  The default is
.BR 100s .
.TP
.BI \-\-no-check-proportion= PERCENT
If the moving average of the proportion of articles being accepted
(rather than declined) by the peer exceeds this value, innduct uses
"no check mode" - ie it just sends the peer the articles with TAKETHIS
rather than checking first with CHECK whether the article is wanted.
This only affects streaming connections.  The default is
.B 95
(ie, 95%).
.TP
.BI \-\-no-check-response-time= ARTICLES
The moving average mentioned above is an alpha-smoothed value with a
half-life of
.IR ARTICLES .
The default is
.BR 100 .
.TP
.BI \-\-reconnect-interval= PERIOD
Limits initiation of new connections to one each
.IR PERIOD .
This applies to reconnections if the peer has been down, and also to
ramping up the number of connections we are using after startup or in
response to an article flood.  The default is
.BR 1000s .
.TP
.BI \-\-flush-retry-interval= PERIOD
If our attempt to flush the feed failed (usually this will be because
innd is not running), try again after
.IR PERIOD .
The default is
.BR 1000s .
.TP
.BI \-\-earliest-deferred-retry= PERIOD
When the peer responds to our offer of an article with a 431 or 436
NNTP response code, indicating that the article has already been
offered to it by another of its peers, and that we should try again,
we wait at least
.IR PERIOD .
before offering the article again.  The default is
.BR 50s .
.TP
.BI \-\-backlog-rescan-interval= PERIOD
We scan the directory containing
.I feedfile
for backlog files at least every
.IR PERIOD ,
in case the administrator has manually dropped in a file there for
processing.
The default is
.TP
.BI \-\-max-flush-interval= PERIOD
We flush the feed at least every
.IR PERIOD
even if the current instance of the feedfile has not reached the size
threshold.
The default is
.BR 100000s .
.TP
.BI \-\-max-flush-interval= PERIOD
We flush the feed and start a new feedfile at least every
.IR PERIOD
even if the current instance of the feedfile has not reached the size
threshold.
The default is
.BR 100000s .
.TP
.BI \-\-idle-timeout= PERIOD
Connections which have had no activity for
.IR PERIOD
will be closed.  This includes connections where we have sent commands
or articles but have not yet had the responses, so this same value
doubles as the timeout after which we conclude that the peer is
unresponsive or the connection has become broken.
The default is
.BR 1000s .
.TP
.BI \-\-max-bad-input-data-ratio= PERCENT
We tolerate up to this proportion of badly-formatted lines in the
feedfile and other input files.  Every badly-formatted line is logged,
but if there are too many we conclude that the corruption to our
on-disk data is too severe, and crash; to successfully restart,
administrator intervention will be required.  This avoids flooding the
logs with warnings and also arranges to abort earlyish if an attempt
is made to process a file in the wrong format.
The default is
.BR 1
(ie, 1%).
.TP
.BI \-\-max-bad-input-data-init= LINES
Additionally, we tolerate this number of additional badly-formatted
lines, so that if the badly-formatted lines are a few but at the start
of the file, we don't crash immediately.
The default is
.BR 30
(which would suffice to ignore one whole corrupt 4096-byte disk block
filled with random data, or one corrupt 1024-byte disk block filled
with an inappropriate text file with a mean line length of at least
35).
.SH INTERACTING WITH INNDUCT
innduct dances a somewhat complicated dance with innd to make sure
that everything goes smoothly and that there are no races.  (See the
two ascii-art diagrams in innduct.c for details of the protocol.)  Do
not mess with the feedfile and other associated files, other than as
explained below in the section
.BR FILES .
.LP
If you tell innd to drop the feed, innduct will (when it notices,
which will normally be the next time it decides flushes) finish up the
articles it has in hand now, and then exit.  It is harmless to cause
innd to flush the feed (but innduct won't notice and flushing won't
start a new feedfile; you have to leave that to innduct).
.LP
There are no signals that can usefully be sent to innduct to give it
complicated instructions.  If you need to kill innduct, feel free to
send it a
.B SIGTERM
or
.B SIGKILL
and nothing will be broken or corrupted.
.SH EXIT STATUS
.TP
.B 0
An instance of innduct is already running for this
.I feedfile
and
.B -q
was specified.
.TP
.B 4
The feed has been dropped by innd, and we (or previous innducts) have
successfully offered all the old articles to the peer site.  Our work
is done.
.TP
.B 8
innduct was invoked with bad options or command line arguments.  The
error message will be printed to stderr, and also (if any options or
arguments were passed at all) to syslog with severity
.BR crit .
.TP
.B 12
Things are going wrong, hopefully shortage of memory, system file
table entries; disk IO problems; disk full; etc.  The specifics of the
error will be logged to syslog with severity
.B err
(if syslog is working!)
.TP
.B 16
Things are going badly wrong in an unexpected way: system calls which
are not expected to fail are doing so, or the protocol for
communicating with innd is being violated, or some such.  Details will
be logged with severity
.B crit
(if syslog is working!)
.TP
.BR 24 - 27
These exit statuses are used by children forked by innduct to
communicate to the parent.  You should not see them.  If you do, it is
a bug.
.SH FILES
.IX Header "FILES"
.IP \fIpathoutgoing\fR/\fIsite\fR
.IX Item "default feedfile"
Default
.IR feedfile .
.IP \fIfeedfile\fR
.IX Item feedfile
Main feed file as specified in
.IR newsfeeds (5).
This and other batchfiles used by innduct contains lines each of which
is of the form
\&    \fItoken\fR \fImessageid\fR
where \fItoken\fR is the inn storage API token.  Such lines can be
written by \fBTf,Wnm\fR in a \fInewsfeeds\fR(5) entry.  During
processing, innduct overwrites lines in the batch files which
correspond to articles it has processed: the line is replaced with
one containing only spaces.  Only innd should create this file, and
only innduct should remove it.
.IP \fIfeedfile\fR_lock
.IX Item "lock file"
Lockfile, preventing multiple innduct invocations for the same
feed.  A process holds this lock after it has opened the lockfile,
made an fcntl F_SETLK call, and then checked with stat and fstat that
the file it now has open and has locked still has the name
\fIfeedfile\fR_lock.  (Only) the lockholder may delete the lockfile.
.IP \fIfeedfile\fR_flushing
.IX Item "flushing file"
Batch file: the main feedfile is renamed to this filename by innduct
before it asks inn to flush the feed.  Only innduct should create or
remove this file.
.IP \fIfeedfile\fR_defer
.IX Item "flushing file"
Batch file containing details of articles whose transmission has
recently been deferred at the request of the recipient site.  Created,
written, read and removed by innduct.
.IP \fIfeedfile\fR_backlog.\fItime_t\fR.\fIinum\fR
.IX Item "backlog file"
Batch file containing details of articles whose transmission has less
recently been deferred at the request of the recipient site.  Created
by innduct, and will also be read and removed by innduct.  However you
(the administrator) may also safely remove backlog files.
.IP \fIfeedfile\fR_backlog\fIsomething\fR
.IX Item "manual backlog file"
Batch file manually provided by the administrator.  The file should be
complete and ready to process at the time it is renamed or hardlinked
to this name.  innduct will then automatically find and read and
process it and eventually remove it.  The administrator may also
safely remove backlog files.  \fIsomething\fR may not contain \fB#\fR
\fB~\fR or \fB/\fR.  Be sure to have finished writing the file before
you rename it to match the pattern \fIfeedfile\fR\fB_backlog\fR*, as
otherwise innduct may find and process the file and read it to EOF
before you have finished creating it.
.IP /etc/news/inn.conf
.IX Item inn.conf
Used to find
.IR pathoutgoing
if none is specified.
.SH HISTORY
Written by Ian Jackson <ijackson@chiark.greenend.org.uk>
.SH "SEE ALSO"
inn.conf(5),
newsfeeds(5)