remove old "calculate included patches in topgit branches" idea; we are going to...

[topbloke.git] / DESIGN

diff --git a/DESIGN b/DESIGN
index a7c142b3cefbeed0f09d5129d1caa4efaad527b5..44650578a3d5473243481e8022e6ce395453f25e 100644 (file)
--- a/DESIGN
+++ b/DESIGN
@@ -17,11 +17,12 @@ Basic update algorithm:
     ii. For each source in the best order, do the following merge:
         (Our base has sources:
             - the branch for each direct dep
     ii. For each source in the best order, do the following merge:
         (Our base has sources:
             - the branch for each direct dep

-            - the remote base)
+            - the remote base
+           - the topgit base, if this is a topgit import)

 
        Find the (latest) common ancestor.
 
 
        Find the (latest) common ancestor.
 

-       Check for unwanted dependency removals.  
+       Check for unwanted dependency removals.

        An unwanted dependency removal is
           A branch in the desired included deps
           Which exists in the common ancestor's actual included deps
        An unwanted dependency removal is
           A branch in the desired included deps
           Which exists in the common ancestor's actual included deps


@@ -32,13 +33,16 @@ Basic update algorithm:
        For each unwanted dependency removal (ie for each such
        branch), search as follows:
           * An "unwanted removal commit" is a non-merge commit in the
        For each unwanted dependency removal (ie for each such
        branch), search as follows:
           * An "unwanted removal commit" is a non-merge commit in the

-            history of the source, which unwantedly removes the dep
-            from the actual included deps.
+            history of the source, which removes the dep from the
+            actual included deps.

           * But the search stops at any point where we would have to
             traverse a commit where .topbloke/deps is empty (which
             stops us looking into the hitory of non-topbloke-controlled
             branches).  This can be done with git-rev-list
             --remove-empty.
           * But the search stops at any point where we would have to
             traverse a commit where .topbloke/deps is empty (which
             stops us looking into the hitory of non-topbloke-controlled
             branches).  This can be done with git-rev-list
             --remove-empty.

+         * It also stops at any point where we meet a commit which
+           does have the dep in the actual included deps.  We have
+           to do this by hand.

           * The the relevant unwanted removal commit for that dep is
             the most recent unwanted removal commit, as defined.
        Select the unwantedly removed dep whose relevant unwanted
           * The the relevant unwanted removal commit for that dep is
             the most recent unwanted removal commit, as defined.
        Select the unwantedly removed dep whose relevant unwanted


@@ -61,6 +65,10 @@ Basic update algorithm:
        from the source.
 
     iii.
        from the source.
 
     iii.

+       Check whether our list of dependencies has changed.  If so
+       we need to restart the whole base update.
+
+    iv.

        Check for missing or unwanted dependency inclusions.  Compare
        our base's desired included deps with our base's actual
        included deps.  In exceptional conditions, they will not
        Check for missing or unwanted dependency inclusions.  Compare
        our base's desired included deps with our base's actual
        included deps.  In exceptional conditions, they will not


@@ -86,12 +94,14 @@ Basic update algorithm:
          * Attempt to apply the appropriate diff to add (resp. remove)
            the contents of the relevant patch (adjusted appropriately
            for metadata, XXX??? particularly the actual inclusion list)
          * Attempt to apply the appropriate diff to add (resp. remove)
            the contents of the relevant patch (adjusted appropriately
            for metadata, XXX??? particularly the actual inclusion list)

+          XXX if we want to add a dep we need to update the dep first

          * Go round again looking for another discrepancy.
 
  3. Update our branch.
        Our branch has sources:
           - our base
           - the remote for our branch
          * Go round again looking for another discrepancy.
 
  3. Update our branch.
        Our branch has sources:
           - our base
           - the remote for our branch

+          - the topgit branch, if this is a topgit import

        For each source in the best order, do the merge.
 
        Double-check the actual dependency inclusions.  In
        For each source in the best order, do the merge.
 
        Double-check the actual dependency inclusions.  In


@@ -102,29 +112,8 @@ Basic update algorithm:
 The "best order" for merges is in order of recency of common
 ancestor, most recent first, and if that does not distinguish,
 merging from local branches first.
 The "best order" for merges is in order of recency of common
 ancestor, most recent first, and if that does not distinguish,
 merging from local branches first.

-     
-"Recency" refers to the order from git-rev-list --date-order.
-
-Actual included deps:
-
-  This is tracked explicitly in .topbloke/included, one branch per
-  line.  For compatibility with older versions, every time we think
-  about a base, branch or source above, we check whether
-  .topbloke/included is present.
-
-  If it isn't then we calculate a child commit which has a
-  .topbloke/included.  In the case of a remote branch or base, we
-  substitute this child commit for the relevant remote ref but do
-  not record it in the remote ref; in the case of a local branch or
-  base, we advance the local branch or base accordingly.

 
 

-  When .topbloke/included is calculated in this way, it always gets
-  the list of desired included deps.  (topgit,
-  which does not support dependency deletion, always has exactly the
-  desired deps actually included.)
-
-  Foreign branches cannot be removed from included and cannot
-  therefore be removed from dependency lists.
+"Recency" refers to the order from git-rev-list --date-order.

 
 
 Patch removal:
 
 
 Patch removal:


@@ -186,3 +175,148 @@ Unnecessary - instead, deal with leaf patches
 Operations like "go up the stack", goes towards leaf.  Hopefully unique.
 "Down" the stack, uses a "conventional" linearisation
 Stack reordering op ?  auto adjust deps
 Operations like "go up the stack", goes towards leaf.  Hopefully unique.
 "Down" the stack, uses a "conventional" linearisation
 Stack reordering op ?  auto adjust deps

+
+
+When merging, we need to DTRT with our metadata.
+Do this by running write-tree/read-tree etc. ourselves ?
+For a source we're merging from, we make a version where the
+metadata we shouldn't be merging is removed ?
+Or something.
+Have discovered that specifying a custom merge driver for a file does
+not have any effect if the three-way-merge looks trivial based
+on looking at the file contents - at least, if you use git-merge.
+
+Only thing which knows how to do all the gitattributes stuff and
+conflict markers and what have you is git-merge.  (git-merge-tree does
+too but the output format is unsuitable.)  "git-merge-index
+... git-merge-one-file" does not honour the merge drivers and is,
+contrary to what the git docs seem to suggest but don't actually
+state, not actualy used by git-merge.
+
+OK so here is a plan:
+  Use git-merge --no-commit
+  Perhaps on a HEAD, and/or against a tree, which have been massaged
+   to make the metadata suitable as input.
+  Filtering out the "merge was successful but we aren't committing"
+  message.  Use a single pipe for stdout/stderr to get interleaving
+  right; the message from git-merge is not i18n'd.
+  Afterwards we:
+  Check for merge success in the index and compare to exit status
+  from git-merge (which is 1 if the merge failed).
+  Adjust the metadata.
+  Print appropriate big fat warnings if we have merge conflicts in our
+  metadata.
+  Commit, adjusting the parents of the new commit to the original
+  parents if we made the merge with special massaged parents.
+  We may still need to have custom merge drivers for metadata.
+
+
+Strategies for each metadata file merge:
+
+       in base/tip     same patch & branch     dep -> base     base -> tip
+
+ msg            T      textual merge           rm from src     not in src
+ deps          B       list merge              rm from src     rm from src
+ deleted        T      std existence merge     rm from src     not in src
+ patch-                BT      must be same            rm from src     must be same
+ topgit-       B       std exist/text merge    rm from src     rm from src
+ [^+]*-                ??      textual merge           rm from src     rm from src
+ +included     BT      list merge        rm from non-tb src    list merge
+ +*-           ??      textual merge     rm from non-tb src    textual merge
+ *[^-]         ??      die, aborting all ops, if found in any tb src or branch
+
+
+
+
+
+Unwanted removal search subgraphs:
+
+  rm           relevant removal
+  inc
+
+   rm          impossible to undo removal, arrgh, terminates search
+inc  inc..
+/  ???   \
+
+    rm
+inc..  rm..    merge of a removal, search down the rm path
+/  [inc]  \
+
+  inc          ??? call it an inclusion, terminate the search
+  rm
+
+   inc         merge of an inclusion, terminates search
+inc..  rm..
+/  [rm]  \
+
+   inc         ??? call it an inclusion, terminate search
+rm..  rm..
+/  ???    \
+
+
+  rm   inc       inc          rm       irrelevant
+  rm   inc     inc inc..    rm rm..
+
+
+
+          OUR BASE              SOURCE                OUR IT TIP
+               |                  |                       |
+               |                  |                       |
+   ADD DEP WH/ inc                rm                      |
+    NEEDS IT   /|                 /\                      |
+             / |                rm  \                     |
+            /  | ______________'/    \                    |
+         inc   |'              /      \                   |     IT tip
+           /   inc     REMOVAL rm       \                         |    elsewhere
+          /     |             /          \                        |   |   |
+        /      |           inc           \                |   |   |
+     some IT    |           /              |                      |   |   |
+                |         inc              |                      |   |   |
+               /           |               |                      |   |   |
+              /          inc               |                      |   |   |
+             |           / |               |                      |   |   |
+           /           /  inc             |               |   |   |
+          /           |   | `------------ | ----<-------- | -.|   |
+                 /           inc  inc             |               |   *2  |
+         |            |`- | ------------- | ----<--------.|   |   |
+         |            `---inc             |              1*  /    |
+         |                |               |               | /     |
+         |     RE-INCLUDE inc             rm REMOVAL      |'      |
+        inc     \          |               |                      |       |
+         |      * REMOVAL rm              |               |       |
+                 |                |              /                |       |
+                 |`-------------- | ----------. /                 |       |
+                 |                |           inc ANC2            |       |
+        inc               inc         /                   |       |
+         |`.____________  |          /                    |       |
+         |              `inc ANC1   /                     |       |
+         |                |        /`--------<----------- | ---.  |
+       without            inc     /                       |     \/
+         |                |      /                        |     /
+          \    RE-INCLUDE inc   /                         |    /
+                   \   \          |    /                          |   /
+            \   * REMOVAL rm  /                           |  /
+              \            |  /                                   | /
+               \           inc FIRST ADD DEP                      |/
+               \          |  \                            *3
+                \         |   `------------<-------------.|
+                         \        |                               |
+                  \       without                         |
+                   \_____ |                               |
+                         `without                         |
+                          |                               |
+                                                         IT
+
+
+  Merge 1* and 2*, diff against some relevant base branch commit
+  or something, and apply to proposed. ???
+
+
+After we are done:
+  source tip is included in our base
+
+Each time:
+  * pick common ancestor
+  * compute whether merge from common anc would unwantedly remove
+  * if so we arrange that the common anc is a "rm" but our branch
+    actually contains IT