dgit: Provide "dgit setup-mergechangelogs" operation

[dgit.git] / dgit.1

.TH dgit 1 "" "Debian Project" "dgit"
.SH NAME
dgit \- git integration with the Debian archive
.
.SH SYNOPSIS
.B dgit
[\fIdgit\-opts\fP] \fBclone\fP [\fIdgit\-opts\fP]
\fIpackage\fP [\fIsuite\fP] [\fB./\fP\fIdir|\fB/\fP\fIdir\fR]
.br
.B dgit
[\fIdgit\-opts\fP] \fBfetch\fP|\fBpull\fP [\fIdgit\-opts\fP]
[\fIsuite\fP]
.br
.B dgit
[\fIdgit\-opts\fP] \fBbuild\fP|\fBsbuild\fP|\fBbuild-source\fP
[\fIbuild\-opts\fp]
.br
.B dgit
[\fIdgit\-opts\fP] \fBpush\fP [\fIdgit\-opts\fP]
[\fIsuite\fP]
.br
.B dgit
[\fIdgit\-opts\fP] \fBrpush\fR \fIbuild-host\fR\fB:\fR\fIbuild-dir\fR
[\fIpush args...\fR]
.br
.B dgit
[\fIdgit\-opts\fP] \fIaction\fR ...
.SH DESCRIPTION
.B dgit
allows you to treats the Debian archive as if it were a git
repository.  See \fBdgit\fP(7) for detailed information about the data
model, common problems likely to arise with certain kinds of package,
etc.

The usual workflow is:
.br
1.      \fBdgit clone\fR or \fBfetch\fR;
.br
2.      make, do dev tests, and commit changes in git as desired;
.br
3.      build packages for upload, using e.g. \fBdgit sbuild\fR
.br
4.      do pre-upload tests of the proposed upload;
.br
5.      \fBdgit push\fR.
.SH OPERATIONS
.TP
\fBdgit clone\fR \fIpackage\fP [\fIsuite\fP] [\fB./\fP\fIdir|\fB/\fP\fIdir\fR]
Consults the archive and dgit-repos to construct the git view of
history for
.I package
in
.I suite
.RB ( sid
by default)
in a new directory (named
.BI ./ package
by default);
also, downloads any necessary orig tarballs.

The suite's git tip is
left on the local branch
.BI dgit/ suite
ready for work, and on the corresponding dgit remote tracking branch.
The
.B origin
remote will be set up to point to the package's dgit-repos tree
for the distro to which
.I suite
belongs.

For your convenience, the
.B vcs-git
remote will be set up from the package's Vcs-Git field, if there is
one - but note that in the general case the history found there may be
different to or even disjoint from dgit's view.
.TP
\fBdgit fetch\fR [\fIsuite\fP]
Consults the archive and git-repos to update the git view of
history for a specific suite (and downloads any necessary orig
tarballs), and updates the remote tracking branch
.BR remotes/dgit/dgit/ \fIsuite\fR.
If the current branch is
.BI dgit/ suite
then dgit fetch defaults to
.IR suite ;
otherwise it parses debian/changelog and uses the suite specified
there.
.TP
\fBdgit pull\fR [\fIsuite\fP]
Does dgit fetch, and then merges the new head of the remote tracking
branch
.BI remotes/dgit/dgit/ suite
into the current branch.
.TP
\fBdgit build\fR ...
Runs
.B dpkg-buildpackage
with some suitable options.  Options and argumments after build
will be passed on to dpkg-buildpackage.  It is not necessary to use
dgit build when using dgit; it is OK to use any approach which ensures
that the generated source package corresponds to the relevant git
commit.

Tagging, signing and actually uploading should be left to dgit push.
.TP
\fBdgit build-source\fR ...
Builds the source package, and a changes file for a prospective
source-only upload, using
.BR dpkg-source .
The output is left in
.IR package \fB_\fR version \fB.dsc\fR
and
.IR package \fB_\fR version \fB_source.changes\fR.

Tagging, signing and actually uploading should be left to dgit push.
.TP
.B dgit clean
Cleans the current working tree (according to the --clean= option in
force).
.TP
.B dgit help
Print a usage summary.
.TP
\fBdgit sbuild\fR ...
Constructs the source package, uses
.B  sbuild
to do a binary build, and uses mergechanges to merge the source and
binary changes files.  Options and argumments after sbuild will be
passed on to sbuild.  Changes files matching
.IB package _ version _*.changes
in the parent directory will be removed; the output is left in
.IR package \fB_\fR version \fB_multi.changes\fR.

Tagging, signing and actually uploading should be left to dgit push.
.TP
\fBdgit git-build\fR ...
Runs
.B git-buildpackage
with some suitable options.  Options and argumments after git-build
will be passed on to git-buildpackage.

Tagging, signing and actually uploading should be left to dgit push.
.TP
\fBdgit push\fR [\fIsuite\fP]
Does an `upload', pushing the current HEAD to the archive (as a source
package) and to dgit-repos (as git commits).  The package must already
have been built ready for upload, with the .dsc and .changes
left in the parent directory.  It is normally best to do the build
with dgit too (eg with dgit sbuild): some existing build tools pass
unhelpful options to dpkg-source et al by default, which can result in
the built source package not being identical to the git tree.

In more detail: dgit push checks that the current HEAD corresponds to
the .dsc.  It then pushes the HEAD to the suite's dgit-repos branch,
makes a signed git tag, edits the .dsc to contain the dgit metadata
field, runs debsign to sign the upload (.dsc and .changes), pushes the
signed tag, and finally uses dput to upload the .changes to the
archive.

dgit push always uses the package, suite and version specified in the
debian/changelog and the .dsc, which must agree.  If the command line
specifies a suite then that must match too.

If dgit push fails while uploading, it is fine to simply retry the
dput on the .changes file at your leisure.
.TP
\fBdgit rpush\fR \fIbuild-host\fR\fB:\fR\fIbuild-dir\fR [\fIpush args...\fR]
Pushes the contents of the specified directory on a remote machine.
This is like running dgit push on build-host with build-dir as the
current directory; however, signing operations are done on the
invoking host.  This allows you to do a push when the system which has
the source code and the build outputs has no access to the key:

1.      Clone on build host (dgit clone)
.br
2.      Edit code on build host (edit, git commit)
.br
3.      Build package on build host (dgit build)
.br
4.      Test package on build host or elsewhere (dpkg -i, test)
.br
5.      Upload by invoking dgit rpush on host with your GPG key.

However, the build-host must be able to ssh to the dgit repos.  If
this is not already the case, you must organise it separately, for
example by the use of ssh agent forwarding.

The remaining arguments are treated just as dgit push would handle
them.

build-host and build\-dir can be passed as separate
arguments; this is assumed to be the case if the first argument
contains no : (except perhaps one in [ ], to support IPv6 address
literals).

You will need similar enough versions of dgit on the build-host and
the invocation host.  The build-host needs gnupg installed, with your
public key in its keyring (but not your private key, obviously).
.TP
.B dgit setup-mergechangelogs
Configures a git merge helper for the file
.B debian/changelog
which uses
.BR dpkg-mergechangelogs .
You can use this in any git repository, not just ones used with
the other dgit operations.
.TP
.B dgit quilt-fixup
`3.0 (quilt)' format source packages need changes representing not
only in-tree but also as patches in debian/patches.  dgit quilt-fixup
checks whether this has been done; if not, dgit will make appropriate
patches in debian/patches and also commit the resulting changes to
git.

This is normally done automatically by dgit build and dgit push.

dgit will try to turn each relevant commit in your git history into a
new quilt patch.  dgit cannot convert nontrivial merges, or certain
other kinds of more exotic history.  If dgit can't find a suitable
linearisation of your history, by default it will fail, but you can
ask it to generate a single squashed patch instead.
.TP
.B dgit version
Prints version information and exits.
.TP
.BI "dgit clone-dgit-repos-server" " destdir"
Tries to fetch a copy of the source code for the dgit-repos-server,
as actually being used on the dgit git server, as a git tree.
.SH OPTIONS
.TP
.BR --dry-run " | " -n
Go through the motions, fetching all information needed, but do not
actually update the output(s).  For push, dgit does
the required checks and leaves the new .dsc in a temporary file,
but does not sign, tag, push or upload.
.TP
.BR --damp-run " | " -L
Go through many more of the motions: do everything that doesn't
involve either signing things, or making changes on the public
servers.
.TP
.BI -k keyid
Use
.I keyid
for signing the tag and the upload.
.TP
.BR --no-sign
does not sign tags or uploads (meaningful only with push).
.TP
.TP
.BI -p package
Specifies that we should process source package
.I package
rather than looking in debian/control or debian/changelog.
Valid with dgit fetch and dgit pull, only.
.TP
.BR --clean=git " | " -wg
The source tree should be cleaned, before building a source package
with one of the build options, using
.BR "git clean -xdf" .
This will delete all files which are not tracked by git.  Also, -wg
causes dgit to pass
.B -nc
to dpkg-buildpackage, which prevents the package's own clean target
from being run.

--clean=git is useful when the package's clean target is troublesome;
the downside is simply that git clean may delete files you forgot to
git add.
.TP
.BR --clean=none " | " -wn
Do not clean the tree before building a source package.  If there are
files which are not in git, or if the build creates such files, a
subsequent dgit push will fail.
.TP
.BR --clean=dpkg-source " | " -wd
Use dpkg-buildpackage to do the clean, so that the source package
is cleaned by dpkg-source running the package's clean target.
This is the default.  It requires the package's build dependencies.
.TP
.BR -N " | " --new
The package may be new in this suite.  Without this, dgit will
refuse to push.
.TP
.BR --ignore-dirty
Do not complain if the working tree does not match your git HEAD.
This can be useful with build, if you plan to commit later.  (dgit
push will still ensure that the .dsc you upload and the git tree
you push are identical, so this option won't make broken pushes.)
.TP
.BI --deliberately- something
Declare that you are deliberately doing
.IR something .
This can be used to override safety catches, including safety catches
which relate to distro-specific policies.  The meanings of
.IR something s
understood in the context of Debian are discussed below:
.TP
.BR --deliberately-not-fast-forward
Declare that you are deliberately rewinding history.  When pushing to
Debian, use this when you are making a renewed upload of an entirely
new source package whose previous version was not accepted for release
from NEW because of problems with copyright or redistributibility.
.TP
.BR --deliberately-include-questionable-history
Declare that you are deliberately including, in the git history of
your current push, history which contains a previously-submitted
version of this package which was not approved (or has not yet been
approved) by the ftpmasters.  When pushing to Debian, only use this
option after verifying that: none of the rejected-from-NEW (or
never-accepted) versions in the git history of your current push, were
rejected by ftpmaster for copyright or redistributability reasons.
.TP
.BR --deliberately-fresh-repo
Declare that you are deliberately rewinding history and want to
throw away the existing repo.  Not relevant when pushing to Debian,
as the Debian server will do this automatically when necessary.
.TP
.BR --quilt=linear
When fixing up source format `3.0 (quilt)' metadata, insist on
generating a linear patch stack.  If such a stack cannot be generated,
fail.
.TP
.BR --quilt=auto
When fixing up source format `3.0 (quilt)' metadata, prefer to
generate a linear patch stack, but if that doesn't seem possible,
generate a single squashed patch for all the changes made in git.
This is not a good idea for an NMU in Debian.
.TP
.BR --quilt=smash
When fixing up source format `3.0 (quilt)' metadata,
generate a single squashed patch for all the changes made in git.
This is not a good idea for an NMU in Debian.
.TP
.BR --quilt=nofix
Check whether source format `3.0 (quilt)' metadata would need fixing
up, but, if it does, fail.  You must then fix the metadata yourself
somehow before pushing.  (NB that dpkg-source --commit will not work
because the dgit git tree does not have a
.B .pc
directory.)
.TP
.BR --quilt=nocheck " | " --no-quilt-fixup
Do not check whether up source format `3.0 (quilt)' metadata needs
fixing up.  If you use this option and the metadata did in fact need
fixing up, dgit push will fail.
.TP
.BI -D
Prints debugging information to stderr.  Repeating the option produces
more output (currently, up to -DD is meaningfully different).
.TP
.BI -c name = value
Specifies a git configuration option.  dgit itself is also controlled
by git configuration options.
.TP
.RI \fB-v\fR version "|\fB_\fR | " \fB--since-version=\fR version |\fB_\fR
Specifies the
.BI -v version
option to pass to dpkg-genchanges, during builds.  Changes (from
debian/changelog) since this version will be included in the built
changes file, and hence in the upload.  If this option is not
specified, dgit will query the archive and use the latest version
uploaded to the intended suite.

Specifying
.B _
inhibits this, so that no -v option will be passed to dpkg-genchanges
(and as a result, only the last stanza from debian/changelog will
be used for the build and upload).
.TP
.RI \fB-m\fR maintaineraddress
Passed to dpkg-genchanges (eventually).
.TP
.RI \fB--ch:\fR option
Specifies a single additional option to pass, eventually, to
dpkg-genchanges.
.TP
.RI \fB--curl=\fR program " | \fB--dput=\fR" program  " |..."
Specifies alternative programs to use instead of
.BR curl ,
.BR dput ,
.BR debsign ,
.BR dpkg-source ,
.BR dpkg-buildpackage ,
.BR dpkg-genchanges ,
.BR sbuild ,
.BR gpg ,
.BR ssh ,
.BR dgit ,
or
.BR mergechanges .

For dpkg-buildpackage, dpkg-genchanges, mergechanges and sbuild,
this applies only when the program is invoked directly by dgit.

For dgit, specifies the command to run on the remote host when dgit
rpush needs to invoke a remote copy of itself.  (dgit also reinvokes
itself as the EDITOR for dpkg-source --commit; this is done using
argv[0], and is not affected by --dgit=).

For ssh, the default value is taken from the
.B DGIT_SSH
or
.B GIT_SSH
environment variables, if set (see below).  And, for ssh, when accessing the
archive and dgit-repos, this command line setting is overridden by the
git config variables
.BI dgit-distro. distro .ssh
and
.B .dgit.default.ssh
(which can in turn be overridden with -c).  Also, when dgit is using
git to access dgit-repos, only git's idea of what ssh to use (eg,
.BR GIT_SSH )
is relevant.
.TP
.RI \fB--curl:\fR option " | \fB--dput:\fR" option " |..."
Specifies a single additional option to pass to
.BR curl ,
.BR dput ,
.BR debsign ,
.BR dpkg-source ,
.BR dpkg-buildpackage ,
.BR dpkg-genchanges ,
.BR sbuild ,
.BR ssh ,
.BR dgit ,
or
.BR mergechanges .
Can be repeated as necessary.

For dpkg-buildpackage, dpkg-genchanges, mergechanges and sbuild,
this applies only when the program is invoked directly by dgit.
Usually, for passing options to dpkg-genchanges, you should use
.BR --ch: \fIoption\fR.

See notes above regarding ssh and dgit.

NB that --gpg:option is not supported (because debsign does not
have that facility).  But see -k.
.TP
.BR -d "\fIdistro\fR | " --distro= \fIdistro\fR
Specifies that the suite to be operated on is part of distro
.IR distro .
This overrides the default value found from the git config option
.BR dgit-suite. \fIsuite\fR .distro .
The only effect is that other configuration variables (used
for accessing the archive and dgit-repos) used are
.BR dgit-distro. \fIdistro\fR .* .

If your suite is part of a distro that dgit already knows about, you
can use this option to make dgit work even if your dgit doesn't know
about the suite.  For example, specifying
.B -ddebian
will work when the suite is an unknown suite in the Debian archive.

To define a new distro it is necessary to define methods and URLs
for fetching (and, for dgit push, altering) a variety of information both
in the archive and in dgit-repos.  How to do this is not yet
documented, and currently the arrangements are unpleasant.  See
BUGS.
.TP
.BI -C changesfile
Specifies the .changes file which is to be uploaded.  By default
dgit push looks for single .changes file in the parent directory whose
filename suggests it is for the right package and version - or,
if there is a _multi.changes file, dgit uses that.

If the specified
.I changesfile
pathname contains slashes, the directory part is also used as
the value for
.BR --build-products-dir ;
otherwise, the changes file is expected in that directory (by
default, in
.BR .. ).
.TP
.BI --build-products-dir= directory
Specifies where to find the built files to be uploaded.
By default, dgit looks in the parent directory
.BR .. ).
.TP
.BI --existing-package= package
dgit push needs to canonicalise the suite name.  Sometimes, dgit
lacks a way to ask the archive to do this without knowing the
name of an existing package.  Without --new we can just use the
package we are trying to push.  But with --new that will not work, so
we guess
.B dpkg
or use the value of this option.  This option is not needed with the
default mechanisms for accessing the archive.
.TP
.BR -h | --help
Print a usage summary.
.TP
.BI --initiator-tempdir= directory
dgit rpush uses a temporary directory on the invoking (signing) host.
This option causes dgit to use
.I directory
instead.  Furthermore, the specified directory will be emptied,
removed and recreated before dgit starts, rather than removed
after dgit finishes.  The directory specified must be an absolute
pathname.
.TP
.BI --no-rm-on-error
Do not delete the destination directory if clone fails.
.SH WORKFLOW - SIMPLE
It is always possible with dgit to clone or fetch a package, make
changes in git (using git-commit) on the suite branch
.RB ( "git checkout dgit/" \fIsuite\fR)
and then dgit push.  You can use whatever gitish techniques you like
to construct the commit to push; the only requirement is that it is a
descendant of the state of the archive, as provided by dgit in the
remote tracking branch
.BR remotes/dgit/dgit/ \fIsuite\fR.

If you are using dgit to do an NMU, and don't know about the
maintainers' preferred packaging workflows, you should make your
changes as a linear series of (logicially separated) commits on top of
what's already in the archive.

If you are lucky the other uploaders have also used dgit and
integrated the other relevant git history; if not you can fetch it
into your tree and cherry-pick etc. as you wish.
.SH WORKFLOW - INTEGRATING BETWEEN DGIT AND OTHER GIT HISTORY
If you are the maintainer of a package dealing with uploads made
without dgit, you will probably want to merge the synthetic commits
(made by dgit to represent the uploads) into your git history.
Normally you can just merge the dgit branch into your own master, or
indeed if you do your work on the dgit local suite branch
.BI dgit/ suite
you can just use dgit pull.

However the first time dgit is used it will generate a new origin
commit from the archive which won't be linked into the rest of your
git history.  You will need to merge this.

If last upload was in fact made with git, you should usually proceed
as follows: identify the commit which was actually used to build the
package.  (Hopefully you have a tag for this.)  Check out the dgit
branch
.RB ( "git checkout dgit/" \fIsuite\fR)
and merge that other commit
.RB ( "git merge debian/" \fIversion\fR).
Hopefully this merge will be trivial because the two trees should
be the same.  The resulting branch head can be merged into your
working branches
.RB ( "git checkout master && git merge dgit/" \fIsuite\fR).

If last upload was not made with git, a different approach is required
to start using dgit.  First, do
.B dgit fetch
(or clone) to obtain a git history representation of what's in the
archive and record it in the
.BI remotes/dgit/dgit/ suite
tracking branch.  Then somehow, using your other git history
plus appropriate diffs and cherry picks from the dgit remote tracking
branch, construct a git commit whose tree corresponds to the tree to use for the
next upload.  If that commit-to-be-uploaded is not a descendant of the
dig remote tracking branch, check it out and say
.BR "git merge -s ours remotes/dgit/dgit/" \fIsuite\fR;
that tells git that we are deliberately throwing away any differences
between what's in the archive and what you intend to upload.
Then run
.BR "dgit push"
to actually upload the result.
.SH CONFIGURATION
dgit looks at the following git config keys to control its behaviour.
You may set them with git-config (either in system-global or per-tree
configuration), or provide
.BI -c key = value
on the dgit command line.
.TP
.BI dgit-suite. suite .distro
.TP
.BI dgit.default.distro
.TP
.BI dgit-distro. distro .username
.TP
.BI dgit-distro. distro .git-url
.TP
.BI dgit-distro. distro .git-user
.TP
.BI dgit-distro. distro .git-host
.TP
.BI dgit-distro. distro .git-proto
.TP
.BI dgit-distro. distro .git-path
.TP
.BI dgit-distro. distro .git-check
.TP
.BI dgit-distro. distro .git-create
.TP
.BI dgit-distro. distro .upload-host
.TP
.BI dgit-distro. distro .mirror
.TP
.BI dgit-distro. distro .archive-query
.TP
.BI dgit-distro. distro .archive-query-default-component
.TP
.BI dgit-distro. distro .sshpsql-user
.TP
.BI dgit-distro. distro .sshpsql-host
.TP
.BI dgit-distro. distro .sshpsql-dbname
.TP
.BI dgit-distro. distro .ssh
.TP
.BI dgit-distro. distro .keyid
.TP
.BR dgit.default. *
for each
.BR dgit-distro. \fIdistro\fR . *
.SH ENVIRONMENT VARIABLES
.TP
.BR DGIT_SSH ", " GIT_SSH
specify an alternative default program (and perhaps arguments) to use
instead of ssh.  DGIT_SSH is consulted first and may contain arguments;
if it contains any whitespace will be passed to the shell.  GIT_SSH
specifies just the program; no arguments can be specified, so dgit
interprets it the same way as git does.
See
also the --ssh= and --ssh: options.
.TP
.BR gpg ", " dpkg- "..., " debsign ", " git ", " curl ", " dput ", " LWP::UserAgent
and other subprograms and modules used by dgit are affected by various
environment variables.  Consult the documentaton for those programs
for details.
.SH BUGS
We should be using some kind of vhost/vpath setup for the git repos on
alioth, so that they can be moved later if and when this turns out to
be a good idea.

dgit push should perhaps do `git push origin', or something similar,
by default.

Debian does not have a working rmadison server, so to find out what
version of a package is in the archive, or to canonicalise suite
names, we ssh directly into the ftpmaster server and run psql there to
access the database.

The mechanism for checking for and creating per-package repos on
alioth is a hideous bodge.  One consequence is that dgit currently
only works for people with push access.

Debian Maintainers are currently not able to push, as there is not
currently any mechanism for determining and honouring the archive's
ideas about access control.  Currently only DDs can push.

dgit's git representation of format `3.0 (quilt)' source packages does
not represent the patch stack as git commits.  Currently the patch
series representation cannot round trip between git and the archive.
Ideally dgit would represent a quilty package with an origin commit of
some kind followed by the patch stack as a series of commits followed
by a pseudo-merge (to make the branch fast-forwarding).  This would
also mean a new `dgit rebase-prep' command or some such to turn such a
fast-forwarding branch back into a rebasing patch stack, and a `force'
option to dgit push (perhaps enabled automatically by a note left by
rebase-prep) which will make the required pseudo-merge.

If the dgit push fails halfway through, it should be restartable and
idempotent.  However this is not true for the git tag operation.
Also, it would be good to check that the proposed signing key is
available before starting work.

dgit's handling of .orig.tar.gz is not very sophisticated.  Ideally
the .orig.tar.gz could be transported via the git repo as git tags.
Doing this is made more complicated by the possibility of a `3.0
(quilt)' package with multiple .orig tarballs.

dgit's build functions, and dgit push, should not make any changes to
your current HEAD.  Sadly this is necessary for packages in the `3.0
(quilt)' source format.  This is ultimately due to what I consider
design problems in quilt and dpkg-source.

There should be an option which arranges for the `3.0 (quilt)'
autocommit(s) to not appear on your HEAD, but instead only in the
remote tracking suite branch.

The option parser requires values to be cuddled to the option name.

dgit assumes knowledge of the archive database.  (The information dgit
needs is not currently available via any public online service with a
well-defined interface, let alone a secure one.)

--dry-run does not always work properly, as not doing some of the git
fetches may result in subsequent actions being different.  Doing a
non-dry-run dgit fetch first will help.
.SH SEE ALSO
\fBdgit\fP(7),
\fBcurl\fP(1),
\fBdput\fP(1),
\fBdebsign\fP(1),
\fBgit-config\fP(1),
\fBgit-buildpackage\fP(1),
\fBdpkg-buildpackage\fP(1),
.br
https://wiki.debian.org/Alioth