WIP

WIP

WIP

WIP classification core done?

WIP before reorgs?

Dgit: "confess" when we die due to a warning, rather than symply dieing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Stub script to prove config handling works

Tested with
  PATH=$PWD:$PATH git -c wombat.foo.bar=23 -c wombat.foo.bar=37 debrebase

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Break out git_get_symref

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: git_cat_file: Introduce $etype parameter

No functional change for existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move $playground global to dgit.

It makes the interface to the playground functions confusing.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: archive-api-query: Avoid crashing due to lack of $isuite

Otherwise:

  Use of uninitialized value $isuite in concatenation (.) or string at dgit line 705.

This breaks the infrastructure.  Closes:#886592.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 4.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 4.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Set distribution to unstable

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: Document using dpkg-source to build source packages

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Entry for push-source subcommand

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: push-source: new tests

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add library function t-push-was-source-only

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Update some copyright info

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Entry for build-source recommends using push-source

If the user invokes build-source because they are going to upload,
they should use push-source rather than push to benefit from the
additional safety check that the upload is actually source-only.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Note that push-source doesn't need --rm-old-changes

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: New command: push-source

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Implement test_source_only_changes

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: build_source bypasses dpkg-buildpackage

This is to avoid the inclusion of .buildinfo in the .changes.  Quoting
<https://lists.debian.org/debian-dpkg/2017/06/msg00005.html>:

    A .buildinfo file is not useful for a source-only upload which is
    veried to be identical to the intended source as present in the
    uploader's version control (eg, by the use of dgit).

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

quilt-gbp test: Set DGIT_TEST_TOLERATE_UNPATCHED_CLEAN

In quilt-gbp we want to test whether dgit detects that quilt fixups
cannot be linear because the user has failed to pass an appropriate
quilt option in a patches-unapplied tree.

build_source() cleans before applying patches, and the package's clean
target might error out because patches are not applied.  This prevents
dgit from attempting quilt linearisation and thus determining that the
tree might be patches-unapplied.  So set an environment variable
telling the package's clean target not to do that.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

example_1.0 worktree: Accept DGIT_TEST_TOLERATE_UNPATCHED_CLEAN

The clean target in the quilt-tip-2 branch in the example_1.0 worktree
errors out if patches are not applied.  With this change, if
DGIT_TEST_TOLERATE_UNPATCHED_CLEAN is set to "true", it will not error
out in this case.

No users, so no functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Fix pod syntax

This was broken in
  b405f072572fd2e4e2acaa94562b697bc639b86d
  dgit-maint-merge(7): Include instructions to clone existing repo

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib: t-git-fsck: put dgit-test-fsck.errs in git dir

This needs to not be in the working tree; it causes trouble there.

This was broken in 090071ae8e31c8f81283af477c1114835ed8fd02
"test suite: Tolerate worktrees when trying to git-fsck"

Reported-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-git-fsck: Introduce fsckerrs variable

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Do not fail if we have no tty when we fail

Tolerate failure of the error log reporting arrangement.  That way the
retry loop is effective even if we don't have a tty.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Fix sleep length in apt-get gpg --import case

This was supposed to sleep for 5 seconds _instead_.  This is a
kind-of-expected error and it's best to carry on a bit quicker.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document gnupg workarounds

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: In ad-hoc runs, honour DGIT_TEST_RETRY_COUNT

This lets us retry each failed test.

This is beneficial because the gnupg race is sort-of independent.
When we want to test every commit, the success probability before
is
  ( 1 - P(gnupg causes test fail))        ) ^ (number of tests)
for each commit.  With this change it's
  ( 1 - P(gnupg causes test fail)^retries ) ^ (number of tests)
which is much much better.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Do not count as "trouble" a 0-byte input file failure

apt-key seems to feed gpg --import an empty pipe.  This makes gpg exit
with status 2.  We think this is trouble, but it isn't.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Save stdin on apt-key's import invocations

Recognise these with an ad-hoc pattern on the command line arguments.
When they occur, save stdin to a temporary file.

We can't do this generally, because stdin might be some thing that
gpg's caller doesn't expect gpg to eat.

When we do this saving, print the size in bytes of the input file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Run with a lock held

This is also an attempt to reduce the impact of the gnupg races.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Export troot as an environment variable

This will be useful in a moment.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg: Have three goes, rather than two

The retry strategy is, empirically, fairly successful.  But the
failure probability is still high enough to be a real nuisance.  Let's
try running it a 3rd time if the second fails.  Maybe this will reduce
the overall impact.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gnupg setup: give each individual test its own AGENT.log

This was always the intent.  However, the structure here arranged to
put the actual value of GNUPGHOME into the import script.

Instead, move the GNUGPHOME setting into the setup fragment, which is
evaluated by the importer.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge branch stable into master

Also regenerate debian/tests/control

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: When source discrepancy involves mode changes, report them specially.

Closes:#886442.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix suggested diff rune in split brain mode

In split brain mode, with unexpected diffs, print dgit view commitid
in suggested diff rune.

HEAD is wrong in this case.

Closes:#886443.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix message about missing quilt cache entry

Should refer to HEAD rather than tree, since dgit needs a commit.

Closes:#884646.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Remove dependency alternative on realpath

That package last existed in Debian wheezy.  Closes:#877552.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Add a bit more rationale (polemic, even).  Closes:#874221.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Recommend mk-build-deps rather than apt-get build-dep.

Closes:#863361.

Suggested-by: Nikolaus Rath <Nikolaus@rath.org>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: mention dgit-*(7) change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-*(7): --deliberately-not-fast-forward for first dgit push

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: mention dgit-maint-merge changes (more)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Include instructions to clone existing repo

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: mention dgit-maint-merge changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Mandate pushing upstream branch to alioth

In the case where we are using gbp-import-orig(1) to maintain a
virtual upstream branch, we must push that branch somewhere.  It is
needed whenever we import a new upstream release.

Thanks to Johannes Schauer for pointing out that the manpage
previously said that pushing to alioth is always optional.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Restructure "NEW UPSTREAM RELEASES"

Previously, some instructions applicable and needed whether or not
upstream tags releases in git were given only for the case where
upstream tags releases in git.

Thanks Johannes Schauer for noticing this problem.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Configure --merge-mode for gbp-import-orig(1)

For 3.0 (quilt) source packages, gbp-import-orig(1) defaults to
--merge-mode=replace, which does the wrong thing for patches-applied
repositories (see gbp-import-orig(1)).

Note that we do override this when importing the first upstream
version.  This is needed because git-merge(1) will refuse to merge
unrelated histories.

Thanks to Johannes Schauer for reporting the problem.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Create an empty branch for upstream source

Thanks to Johannes Schauer for finding the problem.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: dpkgsourceignores-docs: Correct restriction

Restrictions need to be invoked in test suites with
  t-restrict NAME-OF-RESTRICTION
not by calling t-restrction-something.  Only the former gets
translated to the format used in debian/tests/control.

Reported-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.13

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add missing `chiark-utils-bin' to Test-Depends.

Not normally a visible bug because dgit-infrastructure depends on it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Add missing `use' for Dpkg::Compresson et al.

Closes:#879526.

Reported-by: Didier 'OdyX' Raboud
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 3.13~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: document Sean's changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix trailing whitespace

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: Fix typo

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: start 4.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 4.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document test suite changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitworktree: Test quiltification

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitworktree: Add some more operations

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: dpkgsourceignores-correct: Test that the rune DTRT

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: dpkg_source_ignores: Exclude the right set of things

Change the dpkg-source -i argument to exclude exactly the right set of
things.  (Sadly this is not a simple rune.)

Previously we might exclude `foo.git', for example !

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Properly shellquote --git-builder argument to gbp

This is about to contain more exciting shell metacharacters.
(Even now, it is wrong without quoting as we end up telling gbp to
pass -i.git/ rather than -i\.git/ to dpkg-source.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: dpkgsourceignores-docs: Use print-dpkg-source-ignores

Rather than fishing the value out of the in-tree dgit source code.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: print-dpkg-source-ignores action

New print-dpkg-source-ignores option to print the big rune you need to
pass to dpkg-source to make it work exactly the right.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: dpkgsourceignores-docs: Check dgit-maint-merge(7) -i -I rune

Check that the rune we document is the same as dgit actually uses.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Add ' ' around -i\.git/

This manpage should present a shell rune, not a half-quoted thing
whose interpretation is left as a tricky exercise to othe reader.

The ' ' protect the \ so that it would actually get passed to
dpkg-buildpackage.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Reformat slightly

Introduce semantic newlines around the dpg-buildpackage -i -I
rune.  This will make it easier to test.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out -i -I rune into a variable

There is also an instance in dgit-maint-merge.7.pod but that's
trickier to unify.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Break out t-dgit-manpage

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: stunt gnupg: Much better logging of race errors

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: stunt gnupg runs gpg again if it exits status 2

Work around gnupg agent connection races by having our stunt gpg
wrapper simply try running gpg again, once, if it exits 2.
This does not fully suppress the bug but it does significantly reduce
the probability.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Make t-dgit print pwd to stderr, not stdout

This means that tests can capture the output from t-dgit without going
wrong.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document worktree support

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Make commit_getclogp not leave a lot of detritus in .git/dgit

Have it use a fixed filename instead.  It's only needed right away so
that's fine.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

playground refactor: Dgit.pm: fresh_playground: Improve error message

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitworktree: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Tolerate worktrees when trying to git-fsck

A `git worktree' has a file for .git.  We still want to fsck them.  We
can simply run the fsck in the directory containing .git, instead.
That works for ordinary trees with a .git directory, and for
worktrees.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory refactor: open_main_gitattrs confesses if $maindir not set

This improves the error message considerably for "didn't call
record_maindir" bugs.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

worktree support: dgit: Fix for .git/logs/refs/

Experimentally, both the common dir and the worktree's git dir have a
.git/logs.  But, as might be expected, the worktree's one has the
reflog for its HEAD and not any of the refs/ reflogs.

Implicitly, we are deciding here that the dgit quilt cache is shared
between all worktrees.  That seems fine.  I think we don't ever rely
on its actual current value, so even concurrent dgit runs ought to be
fine.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

worktree support: dgit: Fix for .git/info/attributes

Experimentally, this file is in the common dir, not the per-worktree
dir.  Honour $maindir_gitcommon everywhere.

Rename `open_main_gitattrs'.  Previously it could in theory be used in
the playtree (but, it isn't).  Now it always operates on the main
tree.

Indeed, as we require, the `setup' stuff is only done in the main
tree.  The gitattributes fixes for playtrees are done,
unconditionally, with the much simpler hammer in playtree_setup.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

worktree support: Fix playtree_setup

In a worktree, .git is not a directory.  We need to use
$maindir_common.  Add a comment, too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory handling: dgit: Introduce dgit_privdir and use it

This removes another pile of open-coded references to .git/

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: sbuild-gitish: Insist on using "build" chroot

In "test suite: sbuild-gitish: Find sbuild rune in the manpage" aka
f5c0d33ede8a we accidentally switched from "build" to "jessie".
Randomly messing about with the user's "jessie" chroot is not very
nice.  (And it would break in a hypothetical formal test providing the
x-dgit-schroot-build capability.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

worktree support: Fix ensure_a_playground (gives absolute path now)

Make it honour $maindir_gitdir so it will DTRT in a `git worktree'.
The .git/dgit play area ends up in the .git/worktrees/<worktreename>
for this worktree.

It now returns an absolute path.  The only call site that needs
adjusting is the one in fresh_playground.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory handling: dgit: Use $maindir rather than ../../..

Replace all the open-coded ../.. constructs with references to
$maindir.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory refactoring: dgit clone: call record_maintree in multisuite too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory refactoring: dgit clone: call record_maintree

We are making up our own tree, here.  We will need to call
record_maintree in case we need to use information which it obtains.
Currently, we don't.

But git worktrees are complicated: we are going to have to depend on
the new $maindir_git* variables, in setup_new_tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory refactoring: most invocations: call record_maintree

We need to call record_maintree so that $maindir and its friends are
set, since we're going to rely on them heavily.  Most operations will
need them.

Any operation that is invoked in a git tree should definitely have
this called.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

directory refactoring: Rename no_local_git_cfg

This is actually called when we don't know that we are starting in a
git tree.  We want to use this knowledge to control an automatic call
to record_maindir, too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

worktree support: Dgit.pm: Introduce $maindir_gitdir and _commondir

A "git worktree" separates out some of the things which used to be
found in .git, into "common" things and "gitdir" things.

In this patch we simply collect the relevant informaation.  No-one
uses it yet so there is no significant functional change.

However, while we are here, we do improve an error message slightly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

playground refactoring: Dgit.pm: Provide ensure_a_playground

dgit wants to make a lot of temporary things in .git/dgit.
That's like a playground, but dgit doesn't want it wiped.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>