5 Sean Whitton and I are here to present a new git workflow tool for
8 Before I tell you all about it I need to show where it fits into the
9 ecosystem of Debian package management software. On this slide we
10 have you, the maintainer, just off of the left of the slide. On the
11 right we have the Debian repositories.
13 You may have heard me plugging dgit once or twice. You should all use
14 dgit! dgit push publishes your git history so Debian's users can use
15 it! But, that's not actually what I am here to talk about today.
16 git-debrebase does not need dgit, and dgit does not need
19 git-debrebase is an alternative to gbp pq and to git-dpm, and to
20 other tools in the same kind of area like gitkpkg. And, of course,
21 it's an alternative to using raw quilt.
23 git-debrebase is a tool to help manage your git branch containing the
24 Debian version of a package you maintain. git-debrebase helps you
25 maintain a useful git branch with the contents you need (for building
28 git-debrebase is mainly concerned with maintaining your Debian delta
29 queue. That is, that is, the changes you make, for Debian, to the
30 upstream parts of the package. Other tools tend to call this your
31 "quilt patch series". But with git-debrebase there is no quilt, and
32 you don't work with patches. Instead, there is a series of commits.
33 So, we prefer the term "delta queue".
35 git-debrebase is primarily intended for Debian package maintainers,
36 although you could use it outside of or alongside Debian.
38 It does not deal with building at all. You use whatever existing
41 Nor does git-debrebase deal with source packages or orig tarballs. It
42 does not do uploads. Of course when you actually upload to Debian you
43 need to produce a source package. Getting a source package is, of
44 course, as easy, or easier than, with other workflows. Usually, it is
45 done automatically for you by dgit push-source, so you usually don't
46 need to concern yourself with .dsc source packages even when uploading
49 Of course you can share your git branch on a service like salsa,
50 without building or uploading. Then you don't need to deal with
55 git-debrebase offers a standard git-rebase workflow, where you edit
56 the whole of the source code for your package, including your changes
57 to upstream files, and your changes to packaging, all together.
58 The experience is very like using plain git-rebase to edit a topic
59 branch. Delta queue editing can be done at any time, interleaved with
62 As far as I know there are no other tools that offer these features.
63 Both gbp pq and git-dpm require you to switch to a separate view in
64 order to edit the delta queue. Some tools have specific functions for
65 cherry-pick, but with none of them you can just use plain
66 git-cherry-pick or git-am onto your usual branch. With git-debrebase,
67 you can just edit the code and commit it, with git, in the completely
70 Specifically, at any point, you may make commits to upstream files,
71 and commits to packaging, in any order. So you can just cherry pick
72 from upstream. You may make fixup commits, and use the git-rebase
73 autosquash syntax to have them automatically folded in, by the next
74 rebase. If you wish, you may make "mixed" commits containing both
75 changes to upstream files and changes to packaging files.
77 Of course you can always directly edit the source if you use a plain
78 git merge workflow and non-quilt source package - for example, as
79 described in the dgit-maint-merge tutorial manpage. But that does not
80 maintain the Debian delta as a broken-down linear series of changes;
81 and, in the source package, such a merge based workflow squashes all
82 the changes into a single debian-changes patch.
84 So supporting you maintaining a delta queue - that is, a linear series
85 of changes - is what git-debrebase is for.
88 Also, unlike git-dpm and some other tools, git-debrebase has no
89 in-tree metadata, so it can't get out of date or be desynchronised,
90 or need any manual changing or fixup.
94 As I said, unlike with gbp pq and git-dpm, there is no need to ever
95 switch branches. git-debrebase only uses one branch to handle all
98 Of course usually you will have an upstream remote-tracking branch as
99 well. And if you are working in multiple Debian releases, backports
100 for example, you will have branches for those. But it's only one
101 branch for each line of Debian development, and no temporary branches
102 or alternative views.
104 With git-debrebase, you can always immediately build binaries out of
105 your working tree, with dpkg-buildpackage (or whatever other build
106 tool you prefer). And, your working tree is never made dirty by
107 git-debrebase or any of the other tooling: Because your working tree
108 always has the delta queue applied, it is never dirtied by patch
109 application. Because there is no metadata, you can never get a
112 Because git-debrebase treats the quilt patches in debian/patches/ as
113 an output, and handles them entirely automatically, your tree is never
114 dirtied by the generation of patches. And you never need to read any
120 And, the final parts of my plug:
122 With git-debrebase: git-blame, and git-log on a file, work entirely
123 properly. For example, if you do git-log on a file from upstream
124 which was changed in the Debian delta queue, git-log will show the
125 Debian delta queue commits, preceded by the upstream history.
126 We'll show you an example of this in the demo.
128 If you run git-blame you will see a correct indication of which
129 upstream and/or delta queue commits introduced each line. Or, for a
130 file in the debian directory, you will see a correct reporting of
131 which commits in the package's packaging history introduced each line.
133 With git-debrebase, you never need to use the quilt program. You can
134 mostly ignore the 3.0 quilt source format.
136 Not having to learn much about 3.0 quilt is really good for
137 newbies. Particularly, for people from other sofware development
138 communities who don't know about Debian, but usually do know git.
140 Unfortunately it is not possible to paper over the cracks completely:
141 you will still get trouble if you make changes in git which 3.0 quilt
144 On the other hand, when you use git-debrebase with 3.0 quilt, the
145 generated 3.0 quilt source package is perfect pretty, with your delta
146 queue commits converted nicely into patches - just as other people
147 consuming .dscs have come to expect.
149 And finally: of course, git-debrebase is compatible with dgit. You do
150 not need to pass any quilt mode option to dgit. And, you always can
151 upload right away. All necessary bureaucracy is done automatically
152 when you say dgit push-source.
154 OK, that concludes the marketing spiel.
155 I'm going to give you a bit more detail now about how it works.
156 Sean'll be doing a demo later.
158 ===== data model slide "commit history structure" showing master=2
160 There are some important details I'm going to be glossing over, so if
161 you actually want to know what's really going on, please read the
162 reference documentation in the section 5 manpage, where everything is
163 fully and formally defined.
165 So. This slide shows a likely situation, which you might find in the
166 middle of an editing session.
168 The horizontal part near the bottom is called the Breakwater. This
169 branch contains unpatched upstream source code, plus the Debian
170 packaging in the debian directory. It does not contain /any/
171 representation of the delta queue. So it contains neither any
172 of your Debian commits changing upstream files, nor any debian/patches.
174 In the example, commits A and B are packaging work.
176 The Debian delta queue sits on top of that. In this example there are
177 two Debian delta queue commits, 1 and 2. These are commits touching
178 upstream files. In the diagram your current HEAD (ie, your local
179 master branch) is at 2. So your tree contains the patched source code
180 plus the packaging: ie, it is your actual, patched, source package.
182 You could build it with dpkg-buildpackage -uc -b, to produce binaries
183 for testing. You can git grep for things and be told where they are -
184 even if they are in the upstream source files but introduced by your
185 delta queue. You can git log -G for things, and be told where they
186 came from and shown the relevant commit (whether that's upstream, or
189 OK then, suppose you make a change which both edits upstream files,
190 and files in the debian directory. A common example might be
191 a change which edits some upstream file but also adds a changelog
194 ===== data model slide "commit history structure" showing master=C3
196 I'm calling this C3. The reason for this name will be clear in a
199 Now, with this commit your tree is, of course, still fine: you can
200 build and test it right away.
202 But suppose you want to tidy things up, and, in particular, that you
203 wanted the new upstream change to actually come before 2. Maybe it
204 just makes more sense there. Or, maybe, you are going to change patch
205 3 to make use of it, in which case commit 3 really ought to come
208 ===== data model slide with git-debrebase -i overlay
210 So, you run git-debrebase -i. The usual git-rebase todo list comes
211 up, and you will see in it what looks like commit C3, and reorder that
212 to come before commit 2. Assuming there are no conflicts, the result
215 ===== data model slide "commit history structure after git-debrebase"
217 You can see that C3 has been split into two commits: C', which
218 contains the changelog change, and 3' which contains the upstream
219 change. The upstream change is now in the delta queue in the proper
222 C', the packaging part of your new commit, has been pushed down to the
223 bottom of the stack and become part of the breakwater.
225 This is the general scheme of things: we have a fast-forwarding
226 breakwater containing packaging and unchanged upstream files. It
227 doesn't have a ref to itself; instead, it is contained within your
228 master branch. Each time you git-debrebase, the rebase starts on the
231 What about a new upstream version ?
233 ===== data model slide "commit history structure before new-upstream"
235 To rebase onto a new upstream version, you run git-debrebase
236 new-upstream. git-debrebase expects the upstream code in the form of
237 a git commit, of course. (Actually, by default, it hopes to find a
238 tag named after the new upstream version number, but you can tell it
239 explicitly if that's not right.)
241 ===== data model slide "commit history structure after new-upstream"
243 git-debrebase arranges to include the new upstream source into the
244 breakwater, and then rebases your delta queue series onto that.
246 There are new commits on the breakwater: firstly, a special
247 merge. which folds the new upstream source code, unchanged, into your
248 breakwater branch. This is called an Anchor merge. The most recent
249 Anchor merge is the backstop for rebase processing by git-debrebase.
251 The second commit is simply adding a new changelog entry for you.
253 Having provided the new base for your delta queue, git-debrebase then
254 uses git-rebase --onto to rebase the delta queue onto the new
255 breakwater. If you didn't ask for an interactive rebase, and there
256 are no merge conflicts, that's it. You now have the new upstream code
257 with your rebased delta queue.
259 Of course if you are going to upload to the Debian archive you'll also
260 have to make an orig tarball of the new upstream. If you're using the
261 workflow I've been describing so far, that's generally just a single
264 So, indeed, let's consider an upload to Debian. (And let's imagine
265 you made or obtained a suitable orig tarball.)
267 ===== data model slide "commit history structure after upload"
269 There's a certain amount of bureaucracy to be done. In the usual case
270 of an upload with dgit, this is all done for you automatically, so you
271 don't really need to worry about it.
273 But it's useful to understand what's going on, so I'm going to explain
274 it. But, I'm going to describe it to help your understanding.
276 Firstly, you're going to publish your history, so your history has to
277 be made fast forward from the previous version of the package. To
278 achieve this, git-debrebase will make a pseudomerge.
280 ===== data model slide - pseudomerge box overlay
282 A pseudomerge is a merge commit which takes its contents from only one
283 of its parents. You would make one by hand with git-merge -s ours,
284 if you wanted to make your HEAD fast forward, and know that all the
285 wanted changes from the other branch are included.
287 In the example, git-debrebase had recorded the previous branch state,
288 so that it can make the right pseudomerge. Your new branch is derived
289 from the previous one, so it is right to declare that it is fast
292 The branch with the pseudomerge is suitable for pushing to any git
293 server. You could push to salsa, say.
295 ===== data model slide "commit history structure after upload"
297 Secondly, when you upload a 3.0 quilt package, the contents of
298 debian/patches need to be right. Again, that is taken care of
299 automatically: a commit is made adding a patch representation of the
300 delta queue to debian/patches.
302 You can ignore these autogenerated commits.
304 After uploading you'll want to push your branch to salsa, if you have
305 a team repository there. That makes sure all the views of your
306 package are up to date, so that other members of your team won't
307 accidentally base their work on an old version.
309 You can just push a git-debrebase branch which has had the pseudomerge
310 made, called a `stitched' branch, with git push. It's a normal fast
311 forwarding git branch.
313 If you want to push without uploading that's fine too: git-debrebase
314 stitch will just make the pseudomerge for you, giving you a
315 fast-forwarding branch suitable for pushing to salsa or whereever.
317 After upload, next time you come to the package, you can work directly
318 by adding commits on master. If you want to rebase, or just want to
319 tidy the branch up, you can run git-debrebase:
321 ===== data model slide "commit history structure rebasing again"
323 It strips off the bureaucracy commits. Those remain published, of
324 course, but they are removed from your own master branch.
326 If you made any commits on top on top of the pseudomerge (or you
327 pulled any such commits from salsa, or whereever), it folds those back
328 into the breakwater and the delta queue. So once again, you have a
329 nice delta queue to edit.
331 git-debrebase makes a note of where you were previously, so that the
332 next time you want to push or upload, it can stitch the history back
333 in with another pseudomerge. At the start of this walkthough, that
334 ref was indeed present - I glossed over that a bit.
336 ===== data model slide "commit history structure" again
338 What happens if one of your delta queue commits doesn't apply during
339 the upstream rebase - that is, your patch does not apply to the new
340 upstream source code ?
342 ===== data model slide "commit history structure failed rebase"
344 git-rebase users will have seen this kind of situation before.
345 git-rebase stops at the first commit which can't be applied in
346 the new context, and asks the user for help.
348 This looks quite bad. Of course, it's not good. But, this is an
349 irreducible aspect of maintaining a delta queue on top of a moving
350 target. Sometimes, you'll need to fix up conflicts.
352 At least with git-debrebase, you at get good tools to help you fix it
353 up. Some of the other workflows can involve trying to resolve merge
354 conflicts during quilt apply, or fix up conflicts in diffs. That's
357 Also, git-debrebase new-upstream is quite low commitment. Imagine,
358 like on the diagram here, git-debrebase has applied commit 1, and
359 stopped because it can't apply commit 3'.
361 Now, if you decide that this is too difficult to deal with today, you
362 can just say git rebase --abort and everything just gets put back.
364 ===== data model slide "commit history structure aborted rebase"
366 The autogenerated special breakwater merge, and changelog entry, are
367 discarded, leaving you just where you were before. You've wasted no
368 effort because everything you're throwing away was machine-generated.
370 There is one caveat I should mention:
372 ===== data model slide "commit history structure after upload" AGAIN
374 Right now, if two git-debrebase branches diverge, it is not trivial to
375 merge them again. The data model I'm describing does not currently
376 allow general merge commits.
378 If git-debrebase encounters a normal `git merge' it will stop and
379 fail. In the general case, sorting out such a merge is not a trivial
382 gbp pq sometimes handles this kind of situation by expecting you to
383 merge the actual patches: ie, you can end up resolving merge conflicts
384 in diffs. Other tools don't always handle this well either.
386 I have ideas about how to do better at this, so watch this space.
387 But, for now, teams should coordinate to avoid creating diverging
388 git-debrebase branches. git-debrebase will help you with that, by
389 often spotting when divergence is about to occur and warning you.
391 OK, that's enough explanation. Time for the demo. Sean ?
397 ===== status and references slide
401 git-debrebase is available in testing and stretch-backports and is in
402 good shape. Since early versions it has been battle-tested to help
403 with security updates to the Debian Xen packages. The documentation
406 No doubt the user interface and documentation will improve, and new
407 features will will be added. Indeed you'll see we're referring you to
408 the version of the tutorial manpage from sid: that's because we've
409 just done some documentation updates. But you can start using
410 git-debrebase from stretch-backports, testing, or unstable, right
413 The best starting point is probably the tutorial manpage
414 dgit-maint-debrebase (which is in the dgit package).
416 Sean and I are holding a workshop on Tuesday morning, where anyone is
417 invited to come and get help with git-debrebase - and also with dgit.
418 So if your questions don't get answered before then, do drop in.
420 And of course I should refer to the reference documentation.
421 git-debrebase(5) has the data model and definition of terminology.
422 git-debrebase(1) is the command line reference.
424 But, really, start with the dgit-maint-rebase tutorial, or come to the