test suite: lib-reprepro: Add a couple of comments

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Refactor lib-reprepro

* Break out t-reprepro-cfg.
* Introduce the variables `mir' and `etcapt' to many functions.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add t-stunt-parsechangelog to a few tests which were missing it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Parse Dgit: field properly (look for initial \w+ only)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: import-dsc: Parse `Dgit:' field properly, using only \w+

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gbp-orig: Add a missing -m

Without this, it would run an editor if stdout was a tty (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Do not investigate symrefs

Closes:#850547.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
dgit-badcommit-fixup:
* Do not investigate symrefs.  Closes:#850547.

test suite: lib-orig-include-exclude Strip block count out of find -ls

This is unstable!  I have just seen it vary spontaneously in the
middle of a test run, resulting in this:

 ++ diff -u ../before ../after
 --- ../before   2017-01-08 21:39:48.176033763 +0000
 +++ ../after    2017-01-08 21:39:48.208033833 +0000
 @@ -1,2 +1,2 @@
 -   208380      4 -rw-rw-r--   9 ian      ian           236 Nov  6 20:49 /home/ian/things/Dgit/2dgit/tests/tmp/orig-include-exclude-chkquery/mirror/pool/main/example_1.0.orig-docs.tar.gz
 -   208381      4 -rw-rw-r--   8 ian      ian           373 Nov  6 20:49 /home/ian/things/Dgit/2dgit/tests/tmp/orig-include-exclude-chkquery/mirror/pool/main/example_1.0.orig.tar.gz
 +   208380      4 -rw-rw-r--  10 ian      ian           236 Nov  6 20:49 /home/ian/things/Dgit/2dgit/tests/tmp/orig-include-exclude-chkquery/mirror/pool/main/example_1.0.orig-docs.tar.gz
 +   208381      4 -rw-rw-r--   9 ian      ian           373 Nov  6 20:49 /home/ian/things/Dgit/2dgit/tests/tmp/orig-include-exclude-chkquery/mirror/pool/main/example_1.0.orig.tar.gz
 + test 1 = 0
 + t-report-failure

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib-orig-include-exclude: Introduce origs_findls

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Arrange to pass --debug-quick-random to gpg-agent.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-rewrite: Fix operation using installed version of fixup.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Generalise DGIT_TEST_REAL_<FOO> handling

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tests/run-all: Use a bit more cpu

Make allowance for tests that do a bit of downloading or whatever.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.17~ (again)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.16.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup; In --check mode, exit status 2 if things are not fine

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Fix crash when running for 2nd time in bare repo.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.17~ (again)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.16.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: New mode --check which is readonly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.17~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.16

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Put our version in our rewrite map commit message

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Skip our own output refs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Instructions comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Shell bug: avoid local=$(....) as it defeats set -e

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: badcommit-rewrite: Check HEAD manipulation is right

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Do not detach HEAD - this is not needed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: New test for history-rewriting

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Now seems to work

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Debugging (commented out)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib-core: Provide DGIT_BADCOMMIT_FIXUP

So that tests can test this program.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-make-badcommit: Generate a commit we can fix up

This is very like a commit generated by #849041.  The wrongness is the
same.  The difference is that it's not in a dgit-generated merge, but
the fixup script doesn't care about that.

The existing test case which uses this (drs-push-rejects) still sees
badness and is therefore still happy.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Bugfixes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Change name of rewrite map ref (needs an extra / for git)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Break out t-make-badcommit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Honour dgit-rewrite-map

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Provide git_cat_file

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Implement dgit-rewrite-map

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Break out filter_updates

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Remove obsolete variable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Do update sameness filtering a bit later

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Remove failing approach

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Ship

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-badcommit-fixup: Rename

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-fixup: Merge from a filtered view of my personal playground

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.16~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-fixup: remove some debug

badcommit-fixup: dummy add seems workingish

badcommit-fixup: break out $bare

badcommit-fixup: wip dummy commits

badcommit-fixup: wip dummy commits

changelog: finalise 2.15 (will not be uploaded)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-fixup: break out $bare

badcommit-fixup: reorganise arg parsing

test suite: Do not tolerate any stderr output from git-fsck

We must tolerate
  notice: HEAD points to an unborn branch (master)
  notice: No default references
which are generated by some of our existing test cases and are pretty
much harmless.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Introduce t-expect-fsck-fail

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Make expect_fsck_fail contain objids

This will make it possible to be stricter.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge tag dgit/2.14 into `defence in dgit-repos-server' branch

Fix up semantic conflict: the new git-fsck test (in 2.14) fails on the
new broken commits (in the `defence in dgit-repos-server' branch).

We need to disable this.  Ideally we would make a tighter test, but
that's too much to do in a merge.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test bad commit defence in drs-push-rejects

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: Defend against bad commits (from eg #849041).

Right now, this causes the test `debpolicy-quilt-gbp' to fail,
because in this branch #849041 is unfixed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-fixup: only detach bare

badcommit-fixup: fix a git-

badcommit-fixup: seems to work

badcommit-fixup: seems to do main thing

badcommit-fixup: wip

badcommit-fixup: wip g-f-b

badcommit-fixup: wip g-f-b

badcommit-fixup: before no g-f-b

badcommit-fixup: more wip

changelog: finalise 2.14

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

badcommit-fixup wip

changelog: Document changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Do not generate bogus commits

There were two places in the code where ill-formed commits were
generated: both psedumerges.

One is in dgit import-dsc, a fairly minor problem.

The other is in --overwrite and is very bad because that's an
important option.

Nothing in git seems to notice, unless you run git-fsck.  Even a git
server does not, by default.  However, some other popular git servers
do reject these broken commits.

I have tested this changes with git-fsck (by using the test suite
patches which follow in this series) and now everything seems fine.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Clean out erroneous reflog in a worktree

git-reflog expire --expire-unreachable=now --all

Without this, this worktree causes git-fsck to complain!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Run git-fsck a lot

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add some missing calls to t-ok.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Introduce t-ok.

No functional change.

 perl -i~ -pe 's/^echo ok\.$/t-ok/' tests/tests/*[^~]
 perl -i~ -pe 's/^echo ok\.$/t-ok/' tests/setup/*[^~]
 perl -i~ -pe 's/^echo done\.$/t-ok/' tests/tests/*[^~]
 perl -i~ -pe 's/^echo done\.$/t-ok/' tests/setup/*[^~]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.14

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.14

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.13

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: document dgit-sponsorship fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-sponsorship(7): Update in light of fixed #844129

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: document dgit-maint-gbp fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-gbp(7): Remove reference to closed bug.

Thanks Felipe Sateler for the bug report.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

Import fix: Switch back to unpa branch on patch import iterations.

In particular, do not fail utterly if dpkg-source and gbp
disagree. Closes:#848843.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Improve "cannot represent change" message

Print the git old and new modes too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

quilt fixup: Permit creation of patches which delete files

By psssing --include-removal to dpkg-source, and tolerating it when we
do our quilt fixup analysis.

dpkg-source has supported this since at least stretch.

Closes:#848901.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.13

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.12

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Do not fail to import a .dsc containing patches to .git/ (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git Do not fail to import certain weird .dscs

A .dsc containing patches which patch files multiple times would be
rejected by dpkg-source due to #848611.

Work around this by arranging for the patch of interest to never be
the first patch, by introducing (and then evaporating) a dummy patch.

Yuk.

Closes:#848391.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Produce better error reporting when absurd git wrapper fails on a patch

during .dsc import.  Apropos of #848391.

gbp swallows the error from the first run (without --whitespace=fix).
Previously only that first run would do anything useful; the second
would complain about --whitespace=fix.  So failures of dpkg-source
would always produce a useless error message.

Instead, tolerate (and ignore) the --whitespace=fix option which gbp
passes during the second run.  This means that the second run tries to
do the actual work.

This only does extra work if the first run failed, and since the first
and second runs are now equivalent, that happens when the second run
fails too.  But in return for that extra work, we get a report which
actually mentions something that unexpectedly went wrong.

(The alternative would be to have absurd/git try to stash and then
reproduce its previous error, or to make more fragile assumptions
about gbp's behaviour.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Better debugging

Honour DGIT_ABSURD_DEBUG, which is set by dgit from $debuglevel.
Also, in dgit, open the output file for append.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Introduce log()

Right now this simply replaces all the echo >&2, producing no
significant functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Do not fail when cloning a package containing dangling symlinks.

Closes:#848512.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

By default, generate a DEP-14 tag as well as a dgit archive/* tag

Even in non-split-view quilt modes.  Closes:#844129.

This breaks one of the test suite test cases, so disable it there.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out access_cfg_tagformats_can_splitbrain

While we're at it, use a hash %y rather than repeatedly grepping.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Suppress some leftover debugging output from import-dsc. Closes:#847658.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

If we cannot hardlink origs into our extraction area, use symlinks instead.

Closes:#844570.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>