-From: Ian Jackson <ijackson@chiark.greenend.org.uk>
-To: Russ Allbery <rra@debian.org>
-Cc: debian-devel@lists.debian.org
-Subject: Re: Feedback on 3.0 source format problems
-Date: Wed, 4 Jan 2017 12:08:57 +0000
-Russ Allbery writes ("Re: Feedback on 3.0 source format problems"):
-> Even if we never used tarballs, and instead our unit of operation was the
-> upstream Git repository plus Debian branches, I would maintain a rebased
-> branch of Debian changes to upstream
-
-I think this is definitely the right thing for many packages.
-
-I don't think this depends on the interchange format representing the
-changes as patches rather than commits. At a technical level, they
-represent roughly the same information and it is possible to
-interconvert. However:
-
- * There are many changes that patches cannot represent.
-
- * git (working with a rebasing branch) is a much more powerful tool
- for editing a patch series than quilt or diff/patch or emacs
- (working with a series of patches as files). Ie, patches are a
- poor editing format. Consequently most people actually gateway to
- git at the first opportunity, and out again for export.
-
- * Few people outside Debian work with quilt stacks any more.
-
-All of these things mean that patches are a really poor interchange
-format.
-
-The only difficulty is this one:
-
-> This is actually, in a way, *harder* if we were using pure Git, since if I
-> have a rebased branch of Debian changes on top of upstream, and I need a
-> place to integrate that with Debian packaging, what does that
-> debian/master branch look like? I don't really want it to be a constantly
-> rebased branch; I want it to be a conventional branch. But I want to keep
-> merging the changes against upstream into it (but not maintain them on
-> that branch, only maintain the Debian packaging files on that branch).
My preferred answer is that it is a constantly rebasing branch topped
with a series of pseudomerges to make it fast-forwarding.
-git-dpm sort of does this. I have been experimenting with and
-blundering towards another approach, which is closer to raw git.
-
-Something like this:
-
- ------/--A-----/---B3---/--> interchange view
- / / /
- / / 3
- / / /
- 2 2 2
- / / /
- 1 1 1
- / / /
- ---p-----p--A----p---B---> `packaging-only' branch
- / / /
- --#-----#-------#-----> upstream
-
- Key: 1,2,3 commits touching upstream files
- A,B commits touching debian/ only
- B3 mixed commit (eg made by an NMUer)
- # upstream releases
-
- -p- special merge, takes contents of debian/
- / from the previous `packaging only'
- commit and rest from upstream
-
- -/- pseudomerge; contents are identical to
- / parent lower on diagram.
-
-Looking from the tip of the interchange view, it is I think always
-possible to classify these commits appropriately: pseudomerges are
-fairly obvious (if all three trees are identical, we descend to the
-parent with the most recent commit date), and the `p' special merge is
-the only non-pseudo merge and has a special form.
-
-So it would be possible to write a `git-debian-rebase' tool which
-would take (for example) B3, above, and be able to perform functions
-like:
-
- * Strip pseudomerges: Rewrite the current branch so it contains no
- pseudomerges, turning ...B3 into ...p-A-1-2-B3. (This should make
- a note, in your .git/ somewhere, of the latest pseudomerge to be
- deleted.)
-
- * Cleanup branch: Reorganise the current branch so that the debian/
- changes come first, turning -p-A-1-2-B3 into ...p-A-B-1-2.
-
- * New upstream rebase: Start rebasing onto a new upstream version,
- turning ...#...p-A-B-1-2 into ...#'...p'-1-2. This would be a
- wrapper around git-rebase, which prepares p' and then tries to
- rebase 1 2 onto p'. So if you ask for an interactive rebase p'
- doesn't appear in your commit list.
-
- Note that the rebasing of p into p' cannot fail because p' simply
- copies debian/ from B and and everything else from #'. (Rebasing A
- and B is undesirable. We want the debian/ files to be non-rebasing
- so we can `git log' and get the packaging history.)
-
- * Record pseudomerge. This is like "committing" your patch queue.
- The LH parent is taken from the previous strip pseudomerge. (We
- should probably check that this is consistent with what we see in
- debian/changelog, but that is not a sufficient check.)
-
-Maybe some of these operations should automatically edit
-debian/changelog.
-
-The only thing that this can't easily do is permit nonlinear (ie,
-merging) history on the `packaging-only' branch, because upstream
-might contain debian/ files too, so it is not possible to distinguish
-a merge of two `packaging-only' branches from the special merge `p'.
-(Indeed I since upstream might copy debian/ from us, I think it is not
-easy to reliably distinguish the two parents of a `p'. In the most
-exciting edge case, upstream might `git merge' a previous instance of
-our interchange view, but I think even then everything still works.)
-
-Ian.
-
---
-Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
-
-If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
-a private address which bypasses my fierce spamfilter.
-
-From: Ian Jackson <ijackson@chiark.greenend.org.uk>
-To: Sean Whitton <spwhitton@spwhitton.name>
-Cc: debian-devel@lists.debian.org
-Subject: Re: Feedback on 3.0 source format problems
-Date: Fri, 6 Jan 2017 15:29:38 +0000
-
-Sean Whitton writes ("Re: Feedback on 3.0 source format problems"):
-> Could you explain in general terms the difference between the
-> interchange and packaging-only branches
-
-See modified diagram below. Are the annotations I have added (and the
-name change) any help ?
-
-> Does the packaging-only branch contain debian/ alone?
-
-No, it also contains a complete unmodified copy of the upstream code.
-(Except that if upstream had a debian/ directory, it is completely
-replaced.) Perhaps this is the wrong name. Let's try
-`merging-baseline'
-
-For `3.0 (quilt)' the merging-baseline branch contains roughly what
-you would get if you untarred the origs and the debian.tar.gz, and
-then deleted all the patches without applying them.
-
-Not shown on the diagram is the commit `add patch queue to
-debian/patches', which would be needed for `3.0 (quilt)'. This is
-because the diagram is in terms of a sane source format, not `3.0
-(quilt)'. For use with quilty sources, there would be such a commit
-(probably dgit-generated) on top of the actual upstream change
-commits:
-
------/--A!----/--B3!--%--/--> interchange view
/ / / with debian/ directory
% % % all upstream changes applied
/ / / 3.0 (quilt) has debian/patches
- / 2* 2*
+ / / 3*
/ / /
- 2* 2 2
+ 2* 2* 2
/ / /
- 1 1 1 `merging-baseline' branch
+ 1 1 1 `breakwater' branch, merging baseline
/ / / unmodified upstream code
- ---p-----p--A----p--B--C--> plus debian/ (but no debian/patches)
- / / /
- --#-----#-------#-----> upstream
-
+ ---@-----@--A----@--B--C plus debian/ (but no debian/patches)
+ / / / no ref refers to this: we
+ --#-----#-------#-----> upstream reconstruct its identity by
+ inspecting interchange branch
Key:
1,2,3 commits touching upstream files only
B3 mixed commit (eg made by an NMUer)
# upstream releases
- -p- special merge, takes contents of debian/ from the
- / previous `merging-baseline' commit and rest from upstream
+ -@- anchor merge, takes contents of debian/ from the
+ / previous `breakwater' commit and rest from upstream
-/- pseudomerge; contents are identical to
/ parent lower on diagram.
Rebase branch launderer turns each ! into an
equivalent *.
-Ian.
---
-Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
+Looking from the tip of the interchange view, it is I think always
+possible to classify these commits appropriately: pseudomerges are
+fairly obvious (if all three trees are identical, we descend to the
+parent with the most recent commit date). The `@' special merge is
+the only non-pseudo merge and has a special form; also, it will be
+generated only by our tools so can have an annotation in the commit
+message.
-If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
-a private address which bypasses my fierce spamfilter.
+So it would be possible to write a `git-debrebase' tool which would
+take (for example) B4, above, and be able to perform functions like:
+
+ * Strip pseudomerges: Rewrite the current branch so it contains no
+ pseudomerges, turning ...B3 into ...@-A-1-2-B3. (This should
+ make a note, in your .git/ somewhere, of the original branch
+ tip so that it can be overwritten with a pseudomerge.)
+
+ * Cleanup branch: Reorganise the current branch so that the debian/
+ changes come first, turning -@-A-1-2-B3 into ...@-A-B-1-2-3.
+
+ * New upstream rebase: Start rebasing onto a new upstream version,
+ turning ...#..@-A-B-1-2-3 into (...#..@-A-B-|...#'-)@'-1-2. This
+ would be a wrapper around git-rebase, which prepares @' and then
+ tries to rebase 1 2 onto @'. So if you ask for an interactive
+ rebase @' doesn't appear in your commit list.
+
+ Note that the construction of @' cannot fail because @' simply
+ copies debian/ from B and and everything else from #'. (Rebasing A
+ and B is undesirable. We want the debian/ files to be non-rebasing
+ so we can `git log' and get the packaging history.)
+
+ * Record pseudomerge. This is like "committing" your patch queue.
+ The LH parent is taken from the previously recorded tip. (We could
+ perhaps check that this is consistent with what we see in
+ debian/changelog, but that is not a sufficient check so the
+ recorded tip check is primary.)
+
+Maybe some of these operations should automatically edit
+debian/changelog.
+
+Nonlinear (merging) history in the interchange branch is awkward
+because it (obviously) does not preserve the patch queue.
+
+Nonlinear (merging) history in the `packaging-only' branch is OK, if
+we could generate it. We will use the commit message annotation to
+distinguish a merge of two `packaging-only' branches from the special
+merge `@'. (Indeed I since upstream might copy debian/ from us,
+without the annotation and knowledge of the construction order it is
+not easy to reliably distinguish the two parents of a `@'. In the
+most exciting edge case, upstream might `git merge' a previous
+instance of our interchange view, but I think even then everything
+still works.)
+
+Sean Whitton writes ("Re: Feedback on 3.0 source format problems"):
+> Does the [breakwater] branch contain debian/ alone?
+
+No, it also contains a complete unmodified copy of the upstream code.
+(Except that if upstream had a debian/ directory, it is completely
+replaced.)
+
+For `3.0 (quilt)' the breakwater branch contains roughly what you
+would get if you untarred the origs and the debian.tar.gz, and then
+deleted all the patches without applying them.
+
+
+dgit import handling
+--------------------
+
+Consider a non-dgit NMU followed by a dgit NMU:
+
+
+
+ interchange --/--B3!--%--/----D*-->
+ / /
+ % 4
+ / 3
+ / 2
+ / 1
+ 2* &_
+ / /| \
+ 1 0 00 =XBC%
+ /
+ /
+ --@--A breakwater
+ /
+ --#--------> upstream
+
+
+ Key:
+
+ =XBC% dgit tarball import of .debian.tar.gz containing
+ Debian packaging including changes B C and patches
+
+ 0 dgit tarball import of upstream tarball
+ 00 dgit tarball import of supplementary upstream tarball
+ &_ dgit nearly-breakwater import
+ &' git-debrebase converted import (upstream files only)
+ D' git-debrebase converted debian/ changes import
+
+ * ** before and after HEAD
+
+Want to transform this into:
+
+ I. No new upstream version (0 + 00 eq #)
+
+ --/--B3!--%--/------D*-------------/-->
+ / / /
+ % 4 4**
+ / 3 3
+ / 2 2
+ / 1 1
+ 2* &_ /
+ / /| \ /
+ 1 0 00 =XBC% /
+ / /
+ / /
+ --@--A-----B-----------------------C--D
+ /
+ --#----------------------------------------->
+
+
+ II. New upstream (0 + 00 neq #)
+
+ --/--B3!--%--/------D*-------------/-->
+ / / /
+ % 4 4**
+ / 3 3
+ / 2 2
+ / 1 1
+ 2* &_ /
+ / /| \ /
+ 1 0 00 =XBC% /
+ / /
+ / /
+ --@--A-----B--------------------@--C--D
+ / /
+ --#----------------------- - - / - - ----->
+ /
+ &'
+ /|
+ 0 00