5 Sean Whitton and I are here to present a new git workflow tool for
8 Before I tell you all about it I need to show where it fits into the
9 ecosystem of Debian package management software. On this slide we
10 have you, the maintainer, just off of the left of the slide. On the
11 right we have the Debian repositories.
13 You may have heard me plugging dgit once or twice. You should all use
14 dgit! dgit push publishes your git history so Debian's users can use
15 it! But, that's not actually what I am here to talk about today.
16 git-debrebase does not need dgit, and dgit does not need
19 git-debrebase is an alternative to gbp pq and to git-dpm, and to
20 other tools in the same kind of area like gitkpkg. And, of course,
21 it's an alternative to using raw quilt.
23 git-debrebase is a tool to help manage your git branch containing the
24 Debian version of a package you maintain. git-debrebase helps you
25 maintain a useful git branch with the contents you need (for building
28 git-debrebase is mainly concerned with maintaining your Debian delta
29 queue. That is, that is, the changes you make, for Debian, to the
30 upstream parts of the package. Other tools tend to call this your
31 "quilt patch series". But with git-debrebase there is no quilt, and
32 you don't work with patches. Instead, there is a series of commits.
33 So, we prefer the term "delta queue".
35 git-debrebase is primarily intended for Debian package maintainers,
36 although you could use it outside of or alongside Debian.
38 It does not deal with building at all. You use whatever existing
41 Nor does git-debrebase deal with source packages or orig tarballs. It
42 does not do uploads. Of course when you actually upload to Debian you
43 need to produce a source package. Getting a source package is, of
44 course, as easy, or easier than, with other workflows. Usually, it is
45 done automatically for you by dgit push-source, so you usually don't
46 need to concern yourself with .dsc source packages even when uploading
49 Of course you can share your git branch on a service like salsa,
50 without building or uploading. Then you don't need to deal with
55 git-debrebase offers a standard git-rebase workflow, where you edit
56 the whole of the source code for your package, including your changes
57 to upstream files, and your changes to packaging, all together.
58 The experience is very like using plain git-rebase to edit a topic
59 branch. Delta queue editing can be done at any time, interleaved with
62 As far as I know there are no other tools that offer these features.
63 Both gbp pq and git-dpm require you to switch to a separate view in
64 order to edit the delta queue. Some tools have specific functions for
65 cherry-pick, but with none of them you can just use plain
66 git-cherry-pick or git-am onto your usual branch. With git-debrebase,
67 you can just edit the code and commit it, with git, in the completely
70 Specifically, at any point, you may make commits to upstream files,
71 and commits to packaging, in any order. So you can just cherry pick
72 from upstream. You may make fixup commits, and use the git-rebase
73 autosquash syntax to have them automatically folded in, by the next
74 rebase. If you wish, you may make "mixed" commits containing both
75 changes to upstream files and changes to packaging files.
77 Of course you can always directly edit the source if you use a plain
78 git merge workflow and non-quilt source package - for example, as
79 described in the dgit-maint-merge tutorial manpage. But that does not
80 maintain the Debian delta as a broken-down linear series of changes;
81 and, in the source package, such a merge based workflow squashes all
82 the changes into a single debian-changes patch.
84 So supporting you maintaining a delta queue - that is, a linear series
85 of changes - is what git-debrebase is for.
88 Also, unlike git-dpm and some other tools, git-debrebase has no
89 in-tree metadata, so it can't get out of date or be desynchronised,
90 or need any manual changing or fixup.
94 As I said, unlike with gbp pq and git-dpm, there is no need to ever
95 switch branches. git-debrebase only uses one branch to handle all
98 Of course usually you will have an upstream remote-tracking branch as
99 well. And if you are working in multiple Debian releases, backports
100 for example, you will have branches for those. But it's only one
101 branch for each line of Debian development, and no temporary branches
102 or alternative views.
104 With git-debrebase, you can always immediately build binaries out of
105 your working tree, with dpkg-buildpackage (or whatever other build
106 tool you prefer). And, your working tree is never made dirty by
107 git-debrebase or any of the other tooling: Because your working tree
108 always has the delta queue applied, it is never dirtied by patch
109 application. Because there is no metadata, you can never get a
112 Because git-debrebase treats the quilt patches in debian/patches/ as
113 an output, and handles them entirely automatically, your tree is never
114 dirtied by the generation of patches. And you never need to read any
120 And, the final parts of my plug:
122 With git-debrebase: git-blame, and git-log on a file, work entirely
123 properly. For example, if you do git-log on a file from upstream
124 which was changed in the Debian delta queue, git-log will show the
125 Debian delta queue commits, preceded by the upstream history.
126 We'll show you an example of this in the demo.
128 If you run git-blame you will see a correct indication of which
129 upstream and/or delta queue commits introduced each line. Or, for a
130 file in the debian directory, you will see a correct reporting of
131 which commits in the package's packaging history introduced each line.
133 With git-debrebase, you never need to use the quilt program. You can
134 mostly ignore the 3.0 quilt source format.
136 Not having to learn much about 3.0 quilt is really good for
137 newbies. Particularly, for people from other sofware development
138 communities who don't know about Debian, but usually do know git.
140 Unfortunately it is not possible to paper over the cracks completely:
141 you will still get trouble if you make changes in git which 3.0 quilt
144 On the other hand, when you use git-debrebase with 3.0 quilt, the
145 generated 3.0 quilt source package is perfect pretty, with your delta
146 queue commits converted nicely into patches - just as other people
147 consuming .dscs have come to expect.
149 And finally: of course, git-debrebase is compatible with dgit. You do
150 not need to pass any quilt mode option to dgit. And, you always can
151 upload right away. All necessary bureaucracy is done automatically
152 when you say dgit push-source.
154 OK, that concludes the marketing spiel. Now we're going to have a
159 ===== data model slide "commit history structure" showing master=2
161 Now you've seen it in action, I'm going to quickly run through the
162 data model and history structure.
164 There are some important details I'm going to be glossing over, so if
165 you actually want to know what's really going on, please read the
166 reference documentation in the section 5 manpage, where everything is
167 fully and formally defined.
169 So. This slide shows a likely situation, which you might find in the
170 middle of an editing session.
172 The horizontal part near the bottom is called the Breakwater. This
173 branch contains unpatched upstream source code, plus the Debian
174 packaging in the debian directory. It does not contain /any/
175 representation of the delta queue. So it contains neither any
176 of your Debian commits changing upstream files, nor any debian/patches.
178 In the example, commits A and B are packaging work.
180 The Debian delta queue sits on top of that. In this example there are
181 two Debian delta queue commits, 1 and 2. These are commits touching
182 upstream files. In the diagram your current HEAD (ie, your local
183 master branch) is at 2. So your tree contains the patched source code
184 plus the packaging: ie, it is your actual, patched, source package.
186 You could build it with dpkg-buildpackage -uc -b, to produce binaries
187 for testing. You can git grep for things and be told where they are -
188 even if they are in the upstream source files but introduced by your
189 delta queue. You can git log -G for things, and be told where they
190 came from and shown the relevant commit (whether that's upstream, or
193 OK then, suppose you make a change like the one Sean made in the demo:
195 ===== data model slide "commit history structure" showing master=C3
197 I'm calling this C3. The reason for this name will be clear in a
200 Sean's commit edited an upstream file and also debian/changelog. So
201 if you do that, your tree is, of course, still fine: you can build and
204 But suppose you want to tidy things up, and, in particular, that you
205 wanted the new upstream change to actually come before 2. Maybe it
206 just makes more sense there, or maybe you are going to change patch 3
209 ===== data model slide with git-debrebase -i overlay
211 So, you run git-debrebase -i. The usual git-rebase todo list comes
212 up, and you will see in it what looks like commit C3, and reorder that
213 to come before commit 2. Assuming there are no conflicts, the result
216 ===== data model slide "commit history structure after git-debrebase"
218 You can see that C3 has been split into two commits: C', which
219 contains the changelog change, and 3' which contains the upstream
220 change. The upstream change is now in the delta queue in the proper
223 C', the packaging part of your new commit, has been pushed down to the
224 bottom of the stack and become part of the breakwater.
226 This is the general scheme of things: we have a fast-forwarding
227 breakwater containing packaging and unchanged upstream files. It
228 doesn't have a ref to itself; instead, it is contained within your
229 master branch. Each time you git-debrebase, the rebase starts on the
232 What about a new upstream version ?
234 ===== data model slide "commit history structure before new-upstream"
236 To rebase onto a new upstream version, you run git-debrebase
237 new-upstream. git-debrebase expects the upstream code in the form of
238 a git commit, of course. (Actually, by default, it hopes to find a
239 tag named after the new upstream version number, but you can tell it
240 explicitly if that's not right.)
242 ===== data model slide "commit history structure after new-upstream"
244 git-debrebase arranges to include the new upstream source into the
245 breakwater, and then rebases your delta queue series onto that.
247 There are new commits on the breakwater: firstly, a special
248 merge. which folds the new upstream source code, unchanged, into your
249 breakwater branch. This is called an Anchor merge. The most recent
250 Anchor merge is the backstop for rebase processing by git-debrebase.
252 The second commit is simply adding a new changelog entry for you.
254 Having provided the new base for your delta queue, git-debrebase then
255 uses git-rebase --onto to rebase the delta queue onto the new
256 breakwater. If you didn't ask for an interactive rebase, and there
257 are no merge conflicts, that's it. You now have the new upstream code
258 with your rebased delta queue.
260 Of course if you are going to upload to the Debian archive you'll also
261 have to make an orig tarball of the new upstream. If you're using the
262 workflow I've been describing so far, that's generally just a single
265 So, indeed, let's consider an upload to Debian. (And let's imagine
266 you made or obtained a suitable orig tarball.)
268 ===== data model slide "commit history structure after upload"
270 There's a certain amount of bureaucracy to be done. In the usual case
271 of an upload with dgit, this is all done for you automatically, so you
272 don't really need to worry about it.
274 But it's useful to understand what's going on, so I'm going to explain
275 it. But, I'm going to describe it to help your understanding.
277 Firstly, you're going to publish your history, so your history has to
278 be made fast forward from the previous version of the package. To
279 achieve this, git-debrebase will make a pseudomerge.
281 ===== data model slide - pseudomerge box overlay
283 A pseudomerge is a merge commit which takes its contents from only one
284 of its parents. You would make one by hand with git-merge -s ours,
285 if you wanted to make your HEAD fast forward, and know that all the
286 wanted changes from the other branch are included.
288 In the example, git-debrebase had recorded the previous branch state,
289 so that it can make the right pseudomerge. Your new branch is derived
290 from the previous one, so it is right to declare that it is fast
293 The branch with the pseudomerge is suitable for pushing to any git
294 server. You could push to salsa, say.
296 ===== data model slide "commit history structure after upload"
298 Secondly, when you upload a 3.0 quilt package, the contents of
299 debian/patches need to be right. Again, that is taken care of
300 automatically: a commit is made adding a patch representation of the
301 delta queue to debian/patches.
303 You can ignore these autogenerated commits.
305 After uploading you'll want to push your branch to salsa, if you have
306 a team repository there. That makes sure all the views of your
307 package are up to date, so that other members of your team won't
308 accidentally base their work on an old version.
310 You can just push a git-debrebase branch which has had the pseudomerge
311 made, called a `stitched' branch, with git push. It's a normal fast
312 forwarding git branch.
314 If you want to push without uploading that's fine too: git-debrebase
315 stitch will just make the pseudomerge for you, giving you a
316 fast-forwarding branch suitable for pushing to salsa or whereever.
318 After upload, next time you come to the package, you can work directly
319 by adding commits on master. If you want to rebase, or just want to
320 tidy the branch up, you can run git-debrebase:
322 ===== data model slide "commit history structure rebasing again"
324 It strips off the bureaucracy commits. Those remain published, of
325 course, but they are removed from your own master branch.
327 If you made any commits on top on top of the pseudomerge (or you
328 pulled any such commits from salsa, or whereever), it folds those back
329 into the breakwater and the delta queue. So once again, you have a
330 nice delta queue to edit.
332 git-debrebase makes a note of where you were previously, so that the
333 next time you want to push or upload, it can stitch the history back
334 in with another pseudomerge. At the start of this walkthough, that
335 ref was indeed present - I glossed over that a bit.
337 ===== data model slide "commit history structure" again
339 What happens if one of your delta queue commits doesn't apply during
340 the upstream rebase - that is, your patch does not apply to the new
341 upstream source code ?
343 ===== data model slide "commit history structure failed rebase"
345 git-rebase users will have seen this kind of situation before.
346 git-rebase stops at the first commit which can't be applied in
347 the new context, and asks the user for help.
349 This looks quite bad. Of course, it's not good. But, this is an
350 irreducible aspect of maintaining a delta queue on top of a moving
351 target. Sometimes, you'll need to fix up conflicts.
353 At least with git-debrebase, you at get good tools to help you fix it
354 up. Some of the other workflows can involve trying to resolve merge
355 conflicts during quilt apply, or fix up conflicts in diffs. That's
358 Also, git-debrebase new-upstream is quite low commitment. Imagine,
359 like on the diagram here, git-debrebase has applied commit 1, and
360 stopped because it can't apply commit 3'.
362 Now, if you decide that this is too difficult to deal with today, you
363 can just say git rebase --abort and everything just gets put back.
365 ===== data model slide "commit history structure aborted rebase"
367 The autogenerated special breakwater merge, and changelog entry, are
368 discarded, leaving you just where you were before. You've wasted no
369 effort because everything you're throwing away was machine-generated.
371 There is one caveat I should mention:
373 ===== data model slide "commit history structure after upload" AGAIN
375 Right now, if two git-debrebase branches diverge, it is not trivial to
376 merge them again. The data model I'm describing does not currently
377 allow general merge commits.
379 If git-debrebase encounters a normal `git merge' it will stop and
380 fail. In the general case, sorting out such a merge is not a trivial
383 gbp pq sometimes handles this kind of situation by expecting you to
384 merge the actual patches: ie, you can end up resolving merge conflicts
385 in diffs. Other tools don't always handle this well either.
387 I have ideas about how to do better at this, so watch this space.
388 But, for now, teams should coordinate to avoid creating diverging
389 git-debrebase branches. git-debrebase will help you with that, by
390 often spotting when divergence is about to occur and warning you.
392 ===== status and references slide
396 git-debrebase is available in testing and stretch-backports and is in
397 good shape. Since early versions it has been battle-tested to help
398 with security updates to the Debian Xen packages. The documentation
401 No doubt the user interface and documentation will improve, and new
402 features will will be added. Indeed you'll see we're referring you to
403 the version of the tutorial manpage from sid: that's because we've
404 just done some documentation updates. But you can start using
405 git-debrebase from stretch-backports, testing, or unstable, right
408 The best starting point is probably the tutorial manpage
409 dgit-maint-debrebase (which is in the dgit package).
411 Sean and I are holding a workshop on Tuesday morning, where anyone is
412 invited to come and get help with git-debrebase - and also with dgit.
413 So if your questions don't get answered before then, do drop in.
415 And of course I should refer to the reference documentation.
416 git-debrebase(5) has the data model and definition of terminology.
417 git-debrebase(1) is the command line reference.
419 But, really, start with the dgit-maint-rebase tutorial, or come to the