README.states

   1
   2
   3 OVERALL STATES, defining interaction with innd:
   4
   5   F refers to the main feed file as written by innd
   6   D refers to the flushing file <feedfile>_flushing
   7
   8                                                                 START
   9                                                                   |
  10      ,-->--.                                                 check F, D
  11      |     |                                                      |
  12      |     |                                                      |
  13      |     |  <----------------<---------------------------------'|
  14      |     |                                       F exists       |
  15      |     |                                       D ENOENT       |
  16      |     |  duct opens F                                        |
  17      |     V                                                      |
  18      |  Normal                                                    |
  19      |   F: innd writing, duct reading                            |
  20      |   D: ENOENT                                                |
  21      |     |                                                      |
  22      |     |  duct decides time to flush                          |
  23      |     |  duct makes hardlink                                 |
  24      |     |                                                      |
  25      |     V                            <------------------------'|
  26      |  Hardlinked                                  F==D          |
  27      |   F == D: innd writing, duct reading         both exist    |
  28      ^     |                                                      |
  29      |     |  duct unlinks F                                      |
  30      |     |                        <-----------<-------------<--'|
  31      |     |                           open D         F ENOENT    |
  32      |     |                           if exists                  |
  33      |     |                                                      |
  34      |     V                        <---------------------.       |
  35      |  Moved                                             |       |
  36      |   F: ENOENT                                        |       |
  37      |   D: innd writing, duct reading; or ENOENT         |       |
  38      |     |                                              |       |
  39      |     |  duct requests flush of feed                 |       |
  40      |     |   (others can too, harmlessly)               |       |
  41      |     V                                              |       |
  42      |  Flushing                                          |       |
  43      |   F: ENOENT                                        |       |
  44      |   D: innd flushing, duct; or ENOENT                |       |
  45      |     |                                              |       |
  46      |     |   inndcomm flush fails                       |       |
  47      |     |`-------------------------->------------------'       |
  48      |     |                                                      |
  49      |     |   inndcomm reports no such site                      |
  50      |     |`---------------------------------------------------- | -.
  51      |     |                                                      |  |
  52      |     |  innd finishes writing D, creates F                  |  |
  53      |     |  inndcomm reports flush successful                   |  |
  54      |     |                                                      |  |
  55      |     V                                                      |  |
  56      |  Separated                                <----------------'  |
  57      |   F: innd writing                            F!=D             /
  58      |   D: duct reading; or ENOENT                  both exist     /
  59      |     |                                                       /
  60      |     |  duct gets to the end of D                           /
  61      |     |  duct opens F too                                   /
  62      |     V                                                    /
  63      |  Finishing                                              /
  64      |   F: innd writing, duct reading                        |
  65      |   D: duct finishing                                    V
  66      |     |                                            Dropping
  67      |     |  duct finishes processing D                 F: ENOENT
  68      |     V  duct unlinks D                             D: duct reading
  69      |     |                                                  |
  70      `--<--'                                                  | duct finishes
  71                                                               |  processing D
  72                                                               | duct unlinks D
  73                                                               | duct exits
  74                                                               V
  75                                                         Dropped
  76                                                          F: ENOENT
  77                                                          D: ENOENT
  78                                                          duct not running
  79
  80    "duct reading" means innduct is reading the file but also
  81    overwriting processed tokens.
  82
  83 \f
  84
  85 We implement this as follows:
  86
  87             .=======.
  88             ||START||
  89             `======='
  90                 |
  91                 | open F
  92                 |
  93                 |    F ENOENT
  94                 |`---------------------------------------------------.
  95       F OPEN OK |                                                    |
  96                 |`---------------- - - -                             |
  97        D ENOENT |       D EXISTS   see OVERALL STATES diagram        |
  98                 |                  for full startup logic            |
  99      ,--------->|                                                    |
 100      |          V                                                    |
 101      |     ============                                       try to |
 102      |      NORMAL                                            open D |
 103      |     [Normal]                                                  |
 104      |      main F tail                                              |
 105      |     ============                                              V
 106      |          |                                                    |
 107      |          | F IS SO BIG WE SHOULD FLUSH, OR TIMEOUT            |
 108      ^          | hardlink F to D                                    |
 109      |     [Hardlinked]                                              |
 110      |          | unlink F                                           |
 111      |          | our handle onto F is now onto D                    |
 112      |     [Moved]                                                   |
 113      |          |                                                    |
 114      |          |<-------------------<---------------------<---------+
 115      |          |                                                    |
 116      |          | spawn inndcomm flush                               |
 117      |          V                                                    |
 118      |     ==================                                        |
 119      |      FLUSHING[-ABSENT]                                        |
 120      |     [Flushing]                                                |
 121      |     main D tail/none                                          |
 122      |     ==================                                        |
 123      |          |                                                    |
 124      |          |   INNDCOMM FLUSH FAILS                             ^
 125      |          |`----------------------->----------.                |
 126      |          |                                   |                |
 127      |          |   NO SUCH SITE                    V                |
 128      ^          |`--------------->----.         ==================== |
 129      |          |                      \        FLUSHFAILED[-ABSENT] |
 130      |          |                       \         [Moved]            |
 131      |          | FLUSH OK               \       main D tail/none    |
 132      |          | open F                  \     ==================== |
 133      |          |                          \        |                |
 134      |          |                           \       | TIME TO RETRY  |
 135      |          |`------->----.     ,---<---'\      `----------------'
 136      |          |    D NONE   |     | D NONE  `----.
 137      |          V             |     |              V
 138      |     =============      V     V             ============
 139      |      SEPARATED-1       |     |              DROPPING-1
 140      |      flsh->rd!=0       |     |              flsh->rd!=0
 141      |     [Separated]        |     |             [Dropping]
 142      |      main F idle       |     |              main none
 143      |      flsh D tail       |     |              flsh D tail
 144      |     =============      |     |             ============
 145      |          |             |     | install       |
 146      ^          | EOF ON D    |     |  defer        | EOF ON D
 147      |          V             |     |               V
 148      |     ===============    |     |             ===============
 149      |      SEPARATED-2       |     |              DROPPING-2
 150      |      flsh->rd==0       |     V              flsh->rd==0
 151      |     [Finishing]        |     |             [Dropping]
 152      |      main F tail       |     `.             main none
 153      |      flsh D closed     |       `.           flsh D closed
 154      |     ===============    V         `.        ===============
 155      |          |                         `.          |
 156      |          | ALL D PROCESSED           `.        | ALL D PROCESSED
 157      |          V install defer as backlog    `.      | install defer
 158      ^          | close D                       `.    | close D
 159      |          | unlink D                        `.  | unlink D
 160      |          |                                  |  |
 161      |          |                                  V  V
 162      `----------'                               ==============
 163                                                  DROPPED
 164                                                 [Dropped]
 165                                                  main none
 166                                                  flsh none
 167                                                  some backlog
 168                                                 ==============
 169                                                       |
 170                                                       | ALL BACKLOG DONE
 171                                                       |
 172                                                       | unlink lock
 173                                                       | exit
 174                                                       V
 175                                                   ==========
 176                                                    (ESRCH)
 177                                                   [Droppped]
 178                                                   ==========
 179
 180
 181 rune for printing diagrams:
 182   a2ps -R -B -ops README.states
 183
 184
 185
 186 #  part of innduct
 187 #  tailing reliable realtime streaming feeder for inn
 188 #
 189 #  Copyright (C) 2010 Ian Jackson <ijackson@chiark.greenend.org.uk>
 190 #
 191 #  This program is free software: you can redistribute it and/or modify
 192 #  it under the terms of the GNU General Public License as published by
 193 #  the Free Software Foundation, either version 3 of the License, or
 194 #  (at your option) any later version.
 195 #
 196 #  This program is distributed in the hope that it will be useful,
 197 #  but WITHOUT ANY WARRANTY; without even the implied warranty of
 198 #  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 199 #  GNU General Public License for more details.
 200 #
 201 #  You should have received a copy of the GNU General Public License
 202 #  along with this program.  If not, see <http://www.gnu.org/licenses/>.
 203 #
 204 #  (I believe that when you compile and link this as part of the inn2
 205 #  build, with the Makefile runes I have provided, all the libraries
 206 #  and files which end up included in innduct are licence-compatible
 207 #  with GPLv3.  If not then please let me know.  -Ian Jackson.)
 208