wip split into multiple files and make compile

[inn-innduct.git] / README.notes

diff --git a/README.notes b/README.notes
new file mode 100644 (file)
index 0000000..1994bd2
--- /dev/null
+++ b/README.notes
@@ -0,0 +1,62 @@
+NEWSFEEDS
+
+Newsfeeds file entries should look like this:
+    host.name.of.site[/exclude,exclude,...]\
+            :pattern,pattern...[/distribution,distribution...]\
+            :Tf,Wnm
+            :
+or
+    sitename[/exclude,exclude,...]\
+            :pattern,pattern...[/distribution,distribution...]\
+            :Tf,Wnm
+            :host.name.of.site
+
+
+FILES
+
+Four files full of
+   token messageid
+or might be blanked out
+   <spc><spc><spc><spc>....
+
+F site.name                 main feed file
+                               opened/created, then written, by innd
+                               read by duct
+                               unlinked by duct
+                               tokens blanked out by duct when processed
+  site.name_lock            lock preventing multiple ducts
+                               to hold lock must open,F_SETLK[W]
+                                 and then stat to check that locked file
+                                 still has name site.name_lock
+                               holder of this lock is "duct"
+                               (only) lockholder may remove the lockfile
+D site.name_flushing        temporary feed file during flush (or crash)
+                               hardlink created by duct
+                               unlinked by duct
+  site.name_defer           431'd articles, still being written,
+                               created, written, used by duct
+
+  site.name_backlog.<date>.<inum>
+                            431'd articles, ready for innxmit or duct
+                               created (link/mv) by duct
+  site.name_backlog<anything-else>  (where <anything-else> does not
+                                     contain '#' or '~') eg
+  site.name_backlog.manual
+                            anything the sysadmin likes (eg, feed files
+                            from old feeds to be merged into this one)
+                               created (link/mv) by admin
+                               may be symlinks (in which case links
+                               may be written through, but only links
+                               will be removed.
+
+                            It is safe to remove backlog files manually,
+                            if it's desired to throw away the backlog.
+
+Backlog files are also processed by innduct.  We find the oldest
+backlog file which is at least a certain amount old, and feed it
+back into our processing.  When every article in it has been read
+and processed, we unlink it and look for another backlog file.
+
+If we don't have a backlog file that we're reading, we close the
+defer file that we're writing and make it into a backlog file at
+the first convenient opportunity.