5 Sean Whitton and I are here to present a new git workflow tool for
8 Before I tell you all about it I need to show where it fits into the
9 ecosystem of Debian package management software. On this slide we
10 have you, the maintainer, just off of the left of the slide. On the
11 right we have the Debian repositories.
13 You may have heard me plugging dgit once or twice. You should all use
14 dgit! dgit push publishes your git history so Debian's users can use
15 it! But, that's not actually what I am here to talk about today.
16 git-debrebase does not need dgit, and dgit does not need
19 git-debrebase is an alternative to gbp pq and to git-dpm, and to
20 other tools in the same kind of area like gitkpkg. And, of course,
21 it's an alternative to using raw quilt.
23 git-debrebase is a tool to help manage your git branch containing the
24 Debian version of a package you maintain. git-debrebase helps you
25 maintain a useful git branch with the contents you need (for building
28 git-debrebase is mainly concerned with maintaining your Debian delta
29 queue. That is, that is, the changes you make, for Debian, to the
30 upstream parts of the package. Other tools tend to call this your
31 "quilt patch series". But with git-debrebase there is no quilt, and
32 you don't work with patches. Instead, there is a series of commits.
33 So, we prefer the term "delta queue".
35 It's primarily intended for Debian package maintainers, although you
36 could use it outside of or alongside Debian.
38 git-debrebase does not deal with building at all. You use whatever
39 your existing build tools are.
41 Nor does git-debrebase deal with source packages or orig tarballs. It
42 does not do uploads. Of course when you actually upload to Debian you
43 need to produce a source package. Getting a source package is, of
44 course, as easy or easier than with other workflows. Usually, it is
45 done automatically for you by dgit push-source, so you usually don't
46 need to concern yourself with .dsc source packages even when uploading
49 Of course you can share your git branch on a service like salsa,
50 without building or uploading.
54 git-debrebase offers a standard git-rebase workflow, where you edit
55 the whole of the source code for your package, including your changes
56 to upstream files and your changes to packaging, all together.
57 The experience is very like using plain git-rebase to edit a topic
58 branch. Delta queue editing can be done at any time, interleaved with
61 As far as I know there are no other tools that offer these features.
62 Both gbp pq and git-dpm require you to switch to a separate view in
63 order to edit the delta queue. Some tools have specific functions for
64 cherry-pick, but with none of them you can just use plain git cherry
65 pick or git-am onto your usual branch. With git-debrebase, you can
66 just edit the code and commit it, with git, in the completely usual
69 Specifically, at any point, you may make commits to upstream files,
70 and commits to packaging, in any order. So you can just
71 git-cherry-pick from upstream. You can make fixup commits, and use
72 the git-rebase autosquash syntax to have them automatically folded in
73 by the next rebase. If you wish, you may make "mixed" commits
74 containing both changes to upstream files and changes to packaging
77 Of course you can always directly edit the source if you use a plain
78 git merge workflow and non-quilt source package - for example, as
79 described in the dgit-maint-merge tutorial manpage. But of course
80 that does not maintain the Debian delta as a broken-down series;
81 and in the source package such a merge based workflow squashes all the
82 changes into a single debian-changes patch.
84 So supporting you maintaining a delta queue - that is, a linear series
85 of changes - is what git-rebase is for.
88 Also, unlike git-dpm and some other tools, git-debrebase has no
89 in-tree metadata, so it can't get out of date or be desynchronised,
90 or need any manual changing or fixup.
94 As I said, unlike with gbp pq and git-dpm, there is no need to ever
95 switch branches. git-debrebase only uses one branch to handle all for
98 Of course usually you will have an upstream remote-tracking branch as
99 well. And if you are working in multiple Debian releases, backports
100 for example, you will have branches for those. But it's only one
101 branch for each line of Debian development, and no temporary branches
102 or alternative views.
104 With git-debrebase, you can always immediately build binaries out of
105 your working tree, with dpkg-buildpackage (or whatever other build
106 tool you prefer). And your working tree is never made dirty by
107 git-debrebase or any of the other tooling: Because your working tree
108 always has the delta queue applied, it is never dirtied by patch
109 application. Because there is no metadata, you can never get a
112 Because git-debrebase treats the quilt patches in debian/patches/ as
113 an output, and handles them entirely automatically, your tree is never
114 dirtied by the generation of patches. And you never need to read any
120 And, the final part of my plug:
122 With git-debrebase, git-blame, and git-log on a file, work entirely
123 properly. For example, if you do git-log on a file from upstream
124 which was changed in the Debian delta queue, git-log will show the
125 Debian delta queue commits, preceded by the upstream history.
126 We'll show an example of this in the demo.
128 If you run git-blame you will see a correct indication of which
129 upstream and/or delta queue commits introduced each line. Or, for a
130 file in the debian directory, you will see a correct reporting of
131 which commits in the package's packaging history introduced each line.
133 With git-debrebase, you never need to use the quilt program. You can
134 mostly ignore the 3.0 quilt source format.
136 This is all really good for newbies. Particularly, for people from
137 other sofware development communities who don't know about Debian.
139 Unfortunately it is not possible to paper over the cracks completely:
140 you will still get trouble if you make changes in git which 3.0 quilt
143 On the other hand, when you use git-debrebase with 3.0 quilt, the
144 generated 3.0 quilt source package is perfect pretty, with your delta
145 queue commits converted nicely into pathes - just as other people
146 consuming .dscs have come to expect.
148 And finally: of course, git-debrebase is compatible with dgit. You do
149 not need to pass any quilt mode option to dgit. And, you always can
150 upload right away. All necessary bureaucracy is done automatically by
153 OK, that concludes the marketing spiel. Now we're going to have a
158 ===== data model slide "commit history structure" showing master=2
160 Now you've seen it in action, I'm going to quickly run through the
161 data model and history structure.
163 There are some important detail I'm going to be glossing over, so if
164 you actually want to know what's really going on, please read the
165 reference documentation in the section 5 manpage, where everything is
166 fully and formally defined.
168 So. This slide shows a likely situation, which you might find in the
169 middle of an editing session.
171 The horizontal part near the bottom is called the Breakwater. This
172 branch contains unpatched upstream source code, plus the Debian
173 packaging in the debian directory. It does not contain /any/
174 representation of the delta queue. So it contains neither any
175 of your Debian changes to upstream files, nor any debian/patches.
177 In the example, commits A and B are packaging work.
179 The Debian delta queue sits on top of that. In this example there are
180 two Debian delta queue commits, 1 and 2. These are commits touching
181 upstream files. In the diagram your current HEAD (ie, your local
182 master branch) is at 2. So your tree contains the patched source code
183 plus the packaging: ie, it is your actual, patched, source package.
185 You could build it with dpkg-buildpackage -uc -b, to produce binaries
186 for testing. You can git grep for things and be told where they are -
187 even if they are in the upstream source but in your delta queue. You
188 can git log -G for things, and be told where they came from and shown
189 the relevant commit (whether that's upstream, or one of yours).
191 OK then, suppose you make a change like the one Sean made in the demo:
193 ===== data model slide "commit history structure" showing master=C3
195 I'm calling this C3. The reason for this name will be clear in a
198 Sean's commit edited an upstream file and also debian/changelog. So
199 if you do that, your tree is, of course, still fine: you can build and
202 But suppose you want to tidy things up, and, in particular, that you
203 wanted the new upstream change to actually come before 2. Maybe it
204 just makes more sense there, or maybe you are going to change patch 3
207 ===== data model slide with git-debrebase -i overlay
209 So, you run git-debrebase -i. The usual git-rebase todo list comes
210 up, and you will see in it what looks like commit C3, and reorder
211 that. Assuming there are no conflicts, the result looks like this:
213 ===== data model slide "commit history structure after git-debrebase"
215 You can see that C3 has been split into two commits: C', which
216 contains the changelog change, and 3' which contains the upstream
217 change. The upstream change is now in the delta queue in the proper
220 C', the packaging part of your new commit, has been pushed down to the
221 bottom of the stack and become part of the breakwater.
223 This is the general scheme of things: we have a fast-forwarding
224 breakwater containing packaging and unchanged upstream files. It
225 doesn't have a ref to itself; instead, it is contained within your
226 master branch. Each time you git-debrebase, the rebase starts on the
229 What about a new upstream version ?
231 ===== data model slide "commit history structure after new-upstream"
233 To rebase onto a new upstream version, you run git-debrebase
234 new-upstream. git-debrebase expects the upstream code in the form of
235 a git commit, of course. (Actually, by default, it hopes to find a
236 tag named after the new upstream version number, but you can tell it
237 explicitly if that's not right.)
239 git-debrebase arranges to include the new upstream source into the
240 breakwater, and then rebases your delta queue series onto that.
242 There are new commits on the breakwater: firstly, a special
243 merge. which folds the new upstream source code, unchanged, into your
244 breakwater branch. This is called an anchor merge. The most recent
245 anchor merge is the backstop for rebase processing by git-debrebase.
247 The second commit is simply adding a new changelog entry for you.
249 Having provided the new base for your delta queue, it then uses
250 git-rebase --onto to rebase the delta queue commits. If you didn't
251 ask for an interactive rebase, and there are no merge conflicts,
252 that's it as far as the code in git is concerned.
254 Of course if you are going to upload to the Debian archive you'll also
255 have to make an orig tarball of the new upstream. If you're using the
256 workflow I've been describing so far, that's generally just a single
259 ===== data model slide "commit history structure after upload"
261 So, indeed, let's consider an upload to Debian. (And let's imagine
262 you made or obtained a suitable orig tarball.)
264 There's a certain amount of bureaucracy to be done. In the usual case
265 of an upload with dgit, this is all done for you automatically, so you
266 don't really need to worry about it.
268 But it's useful to understand what's going on, so I'm going to explain
271 Firstly, you're going to publish your history, so your history has to
272 be made fast forward from the previous version of the package. To
273 achieve this, git-debrebase will make a pseudomerge.
275 ===== data model slide - pseudomerge box overlay
277 A pseudomerge is a merge commit which takes its contents from only one
278 of its parents. You would make one by hand with git-merge -s ours,
279 if you wanted to make your HEAD fast forward, and know that all the
280 wanted changes from the other branch are included.
282 git-debrebase records the previous branch state, so that it can make
283 the right pseudomerge. Your new branch is derived from the previous
284 one, so it is right to declare that it is fast forward, too.
286 The branch with the pseudomerge is suitable for pushing to any git
287 server. It's what you would push to salsa, say.
289 ===== data model slide "commit history structure after upload"
291 Secondly, when you upload a 3.0 quilt package, the contents of
292 debian/patches need to be right. Again, that is taken care of
293 automatically: a commit is made adding a patch representation of the
294 delta queue to debian/patches.
296 You can ignore these autogenerated commits.
298 After uploading you'll want to push your branch to salsa, if you have
299 a team repository there. That makes sure all the views of your
300 package are up to date, so that other members of your team won't
301 accidentally base their work on an old version.
303 You can just push a git-debrebase branch which has had the pseudomerge
304 made, called a `stitched' branch, with git push. It's a normal fast
305 forwarding git branch.
307 If you want to push without uploading that's fine too: git-debrebase
308 stitch will just make the pseudomerge for you, giving you a
309 fast-forwarding branch suitable for pushing to salsa or whereever.
311 After upload, next time you come to the package, you can work directly
312 by adding commits on master. If you want to rebase, or just want to
313 tidy the branch up, you can run git-debrebase:
315 ===== data model slide "commit history structure rebasing again"
317 It strips off the bureaucracy commits. Those remain published, of
318 course, but they are removed from your own master branch.
320 If you made any commits on top on top of the pseudomerge (or you
321 pulled any such commits from salsa, or whereever), it folds those back
322 into the breakwater and the delta queue. So you once again a nice
325 git-debrebase makes a note of where you were previously, so that when
326 you next want to push or upload, it can stitch the history back in
327 with another pseudomerge. At the start of this walkthough, that ref
328 was indeed present - I glossed over that a bit.
330 ===== data model slide "commit history structure" again
332 More common is a different kind of merge problem.
334 What happens if you one of your delta queue commits doesn't apply
335 during the upstream rebase - that is, your patch does not apply to the
336 new upstream source code ?
338 ===== data model slide "commit history structure failed rebase"
340 git-rebase users will have seen this kind of situation before.
341 git-rebase stops at the first commit which can't be applied in
342 the new context, and asks the user for help.
344 This looks quite bad. Of course, it's not good. But, this is an
345 irreducible aspect of maintaining a delta queue on top of a moving
346 target. Sometimes, you'll need to fix up conflicts.
348 At least with git-rebase, you at least get good tools to help you fix
349 it up. Some of the other workflows can involve trying to resolve
350 merge conflicts during quilt apply. Much less fun.
352 Also, git-debrebase new-upstream is quite low commitment. Imagine,
353 like on the diagram here, git-rebase has applied commit 1, and stopped
354 because it can't apply commit 3'.
356 Now, if you decide that this is too difficult to deal with today, you
357 can just say git rebase --abort and everything just gets put back.
359 ===== data model slide "commit history structure aborted rebase"
361 The autogenerated special breakwater merge, and changelog entry, are
362 discarded, leaving you just where you were before. You've wasted no
363 effort because everything you're throwing away was machine-generated.
365 There is one caveat I should mention:
367 ===== data model slide "commit history structure after upload" AGAIN
369 Right now, if two git-debrebase branches diverge, it is not trivial to
370 merge them again. The data model I'm describing does not currently
371 allow general merge commits.
373 If git-debrebase encounters a normal `git merge' it will stop and
374 fail. In the general case, sorting out such a merge is not a trivial
377 gbp pq sometimes handles this kind of situation by expecting you to
378 merge the actual patches: ie, you can end up resolving merge conflicts
379 in diffs. Other tools don't always handle this well either.
381 I have ideas about how to do better at this, so watch this space.
382 But, for now, teams should coordinate to avoid creating diverging
383 git-debrebase branches. git-debrebase will help you with that by
384 often spotting when it is about to occur.
386 ===== status and references slide
388 git-debrebase is available in testing and stretch-backports and is in
389 good shape. Since early versions it has been battle-tested to help
390 with security updates to the Debian Xen packages. The documentation
393 No doubt the user interface and documentation will improve, and new
394 features will will be added. But you can start using it right away.
396 The best starting point is probably the tutorial manpage
397 dgit-maint-debrebase (7) (which is in the dgit package).
399 Sean and I are holding a workshop on Tuesday morning, where anyone is
400 invited to come and get help with git-debrebase - and also with dgit.
401 So if your questions dun't get answered before then, do drop in.
403 And of course I should refer to the reference documentation.
404 git-debrebase(5) has the data model and definition of terminology.
405 git-debrebase(1) is the command line reference.
407 But, really, start with the dgit-maint-rebase tutorial, or come to the