Hi. I'm here to plug dgit, which is a system for treating the Debian
archive like a git remote.
-I'm going to talk for about 35 minutess and then I'll take questions.
+I'm going to talk for about 25 minutess and then I'll take questions.
Firstly, a bit of framing.
notoriously confusing.
+So, in summary, if you are doing cross-archive work, or bug squashing,
+you can start using dgit right away and I hope it will make your life
+a lot easier.
+
+
==== NMU linear history on top of basic dgit history
Sadly, unless the maintainer uses dgit, the history you see in dgit
directly from dgit's objectives.
The most important requirement is that your git tree is identical to
-the unpacked source package. (Technically, in the case of a `3.0
-(quilt)' package, it is what is sometimes called a `patches-applied
-packaging branch without .pc directory'. Patches-applied means that the upstream
-source files in the main package tree correspond to the actual source
-code that will be used when the package is built, rather than to the
-upstream versions.)
-
-For all native packages, and for users of git-dpm and raw git, this is
+the unpacked source package.
+
+For all native and source format 1.0 packages (which will include
+people using raw git to manage their delta from upstream), this is
already the interchange format. These maintainers can start using
dgit right away for all their maintainer uploads. Please do!
-For those using git-buildpackage with `3.0 (quilt)', things are a bit
-more complicated. I'm told that gbp pq can be used to generate a
-patches-applied branch, and that some users prefer to use that as the
-interchange git branch, but I know this is far from universal. I'm
-talking to the git-buildpackage maintainers about gbp integration, so
-watch this space.
-
The other requirement of dgit is simply that the dgit branches are
fast-forwarding. So if your tools have made a rebasing branch, you
may need to make a fake merge (with git merge -s ours) before pushing.
haven't decided the exact shape yet.
+==== 3.0 quilt applied slide
+
+For `3.0 (quilt)' packages, things are more complicated. The purpose
+of dgit is to provide a git view which is identical to the source
+package, and can be worked on without knowledge of the maintainer's
+chosen source format, patch system or git workflow.
+
+This means that the dgit git tree for a `3.0 (quilt)' package is what
+is sometimes called a `patches-applied packaging branch without .pc
+directory'.
+
+This means that all the changes made in Debian are represented in the
+main source tree, as well as being contained within a patch in
+debian/patches.
+
+But: I think the most popular tool for working with `3.0 (quilt)' is
+git-buildpackage, and git-buildpackage likes to work with
+patches-unapplied tress. I've spoken to Guido Guenther (the
+git-buildpackage maintainer) about this. We had a really good
+conversation. We have a plan for interoperating, which involves dgit
+synthesising an extra git commit, to apply the patches, and then
+pushing that to the dgit git server. (The extra git commit wouldn't
+appear on your own branch; if it did, it would cause trouble.)
+
+=== xxxx status table
+
+There is another problem which affects both git-buildpackage and
+git-dpm: namely, .gitignore files. Neither git-buildpackage nor
+git-dpm represent the .gitignore, which typically occurs in the git
+tree, as a patch in debian/patches. Rather, these tools tell
+dpkg-source to ignore it when buildinng the source package. So the
+source package does not contain .gitignore, although the git tree
+does.
+
+I have spoken to Bernhard Link (the git-dpm maintainer) about this and
+he wasn't keen on having git-dpm treat .gitignore normally, and record
+.gitignores in a patch in the git-dpm branch. So unless we can
+persuade Bernhard to offer this, at least as an option, dgit is
+probably going to have to synthesise a commit to add them, in much the
+same way as it will add a commit to convert a git-buildpackage
+patches-unapplied branch to a directly-editable patches-applied one.
+
+
+
+xxx empty directories
+
+
+ I'm told that gbp pq can be used to generate a
+patches-applied branch, and that some users prefer to use that as the
+interchange git branch, but I know this is far from universal. I'm
+talking to the git-buildpackage maintainers about gbp integration, so
+watch this space.
+
+
+, and for users of git-dpm and
+
+(more or less)
+ xxx
+
+
+
==== data flow slide
There are a few other things I ought to cover, since they often come
The second thing that's less than ideal is that the dgit git history
does not generally include the package upload history.
git-import-dscs can produce a git branch representing the upload
+
+xxx patches-applied
+
history, but dgit doesn't run that itself. It would be difficult for
dgit to do so because deciding which set of versions to include is
nontrivial and of course it would involve an awful lot of downloading.
resolved by rerunning autotools.
-And a second way dgit's rule can bite is that it is normally best to
-use one of dgit's build operations to build for upload. This is
-mainly because most other tools remove .gitignore by default. dgit
-requires that the source package and git tree are the same, so if your
-git tree has .gitignore in it, your source package should too.
+And a second way dgit's rule can bite is .gitignore. Most existing
+packaging tools remove .gitignore from source packages by default.
+But, dgit dgit requires that the source package and git tree are the
+same, so if your git tree has .gitignore in it, your source package
+should too.
+==== manpage slide
+
+So it is normally best to use one of dgit's build operations to build
+for upload; in generally, all that those build operations do
+differently to the underlying tool, is pass some appropriate -I
+options, to exclude exactly and only the .git directory.
+
+
+==== manpage clean slide
Finally, there is one compelling advantage of dgit's git-based
approach.
If you're not in the habit of forgetting to say git-add, you can set a
configuration option to have dgit always use git-clean. Then you will
-never have to fight a buggy clean target, in a strange package, ever again.
+never have to fight a buggy clean target, in a strange package, ever
+again.
==== Future plans slide
-I have a number of plans for the future, some of which I need help
-with. But I don't have time, I'm afraid, to go through them.
+I have a number of plans for the future, some of which I may even get
+around to:
+
+
+dgit can't currently upload to DELAYED queues, which makes it a bit
+awkward for a fire-and-forget NMU. One way to implement this would be
+to push to the suite branch but send the package upload to DELAYED.
+But such an upload which was deleted by the maintainer would still
+present be in the suite branch, and the changes might be resurrected
+by a future dgit user. At the very least there should be a way for
+the maintainer to revert a push which was accompanied by a DELAYED
+upload.
+
+The other implementation would have the pushed branch stashed away
+somewhere and "revealed" only when it was ready, if it was still a
+fast forward. That makes it tricker for someone else to find it to
+work on top of it, and is more work to implement, but it's probably
+better.
+
+
+It would be easy for dgit, when pushing an NMU, to construct the
+NMUdiff and email it to the BTS. Of course a maintainer can fetch
+your NMU as git commits easily with dgit fetch, but maybe the
+maintainer isn't using dgit (or, even, isn't using git at all), so you
+for the benefit of those maintainers we should still be sending these
+diffs.
+
+
+dgit push is slightly more work than it needs to be when doing a
+source-only upload. In this case there is not really any need for a
+separate `dgit build-source' step. I think there should be a way to
+have dgit push make the source package as well, since it easily can.
+
+
+The dgit git server has most of the moving parts to become a
+fairly-general-purpose git server with access control equivalent to
+the Debian archive. In this role it could replace some uses of
+Alioth, which is good because Alioth is doing too many other things.
+
+In particular, with the use of signed pushes, we could have
+traceability of pushes, which makes a gitish packaging workflow more
+practical - in particular, maintainers would not need to as thoroughly
+audit work found on for-upload branches on the git server, before
+pushing the code to the archive. (I suspect many maintainers already
+don't do this as thoroughly as they should.)
+
+
+dgit requires the suite branches to be fast forwarding. If you have a
+workflow involving git rebase, you need to make a fake merge (ie, a
+merge made with git merge -s ours), at the tip of your branch, to make
+it be a fast forward from the previous state of the suite branch. And
+then when you want to work on the package again, you need to strip
+that same fake merge.
+
+This can be done with raw git operations (git merge and git reset)
+quite easily, but it's ugly. Also doing it by hand like this doesn't
+provide many safety catches.
+
+I intend to provide some tooling which will help with this, but I
+haven't exactly decided what the UI should be and how it should work.
+
+
+The final thing I can do myself is arrange for non-dgit uploads to be
+automatically converted, and appended to the dgit suite branches on
+the dgit git server. That would make the dgit history, as shown on
+browse.dgit.debian.org for example, for any package be more useful,
+starting with the first dgit-based upload. In particular it would
+avoid the browse view ever being out of date.
+
+
+There are also some things that I need help with.
+
+The documentation for how dgit works together with existing git
+management tools is rather sparse.
+
+Ideally each git workflow tool would explain how (or whether) it works
+with dgit. Tasks that should be covered include how to incorporate
+changes made in an NMU into the tool's favoured history format, and
+how to upgrade a package to a new upstream version. I have spoken to
+the git-buildpackage and git-dpm maintainers about this. I'm
+optimistic particularly about git-buildpackage.
+
+
+There is a difficulty with empty directories in source packages. git
+refuses to handle empty directories - they tend to just vanish from
+commits and checkouts. But I'm told there are some source packages
+whose orig tarballs contain empty directories, and build systems
+depend on those empty directories.
+
+Currently if you dgit clone such a package, it won't build. I don't
+have a particularly good answer to this problem right now. It's no
+good to have dgit create these directories in the working tree, when
+it notices, because the git branches might be transported by plain git
+and used elsewhere.
+
+This can be worked around in the source package by either adding
+something to debian/rules to create these directories, or by adding
+some files to the directories to make them nonempty. But at the
+moment a maintainer is not required to do either of these things.
+
+
+And then there are a couple of exciting possibilities for new uses:
+
+Firstly, it would be very nice to have a way for a sponsee to provide
+a dgitish git branch, which the sponsor could work with directly and
+ultimately dgit push. I'm really need to talk about this with some
+people more familiar than me with the sponsorship queue
+infrastructure.
+
+
+
+Secondly, dgit is not just for Debian. Other distros currently using
+only source packages can use it too. Read-only support doesn't
+necessarily depend on specific support at the distro end. So, you can
+already dgit fetch from ubuntu; I'd welcome patches for read-only
+support for Mint, for example.
+
+dgit push support requires a suitably set up git server, and maybe
+some thought, depending on the distro's views on push access control.
+I hear rumours that Ubuntu might grow a dgit git server and support
+dgit push. Patches very welcome!
+
+
+==== Thanks and more info slide
+
+Finally, there are some people I need to thank.
+
+At least half of the design of dgit is Joey Hess's, at the now-famous
+session in Vaumarcus.
+
+Debian System Administration, and particularly Peter Palfrader, have
+been absolutely brilliant. The level of service and competence has
+been outstanding. Ever time I noticed some aspect I had forgotten,
+and made a blatherous and ill-thought-out request, DSA would propose a
+much better alternative and implement their end of it it almost
+immediately.
+
+Also, I'd like to thank the ftpmasters for setting up the new
+ftpmaster data service. This has enabled me to extend dgit's
+useability to non-DDs.
-Instead, I'm going to open the talk up to questions now.
+So that's most of what I have prepared. There's, of course, a lot
+more detailed information in the manpages.
-12 mins
+I'm going to take questions now.
~ijackson / talk-2015-debconf-dgit.git / blobdiff