dgit: Do not generate bogus commits

There were two places in the code where ill-formed commits were
generated: both psedumerges.

One is in dgit import-dsc, a fairly minor problem.

The other is in --overwrite and is very bad because that's an
important option.

Nothing in git seems to notice, unless you run git-fsck.  Even a git
server does not, by default.  However, some other popular git servers
do reject these broken commits.

I have tested this changes with git-fsck (by using the test suite
patches which follow in this series) and now everything seems fine.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Clean out erroneous reflog in a worktree

git-reflog expire --expire-unreachable=now --all

Without this, this worktree causes git-fsck to complain!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Run git-fsck a lot

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add some missing calls to t-ok.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Introduce t-ok.

No functional change.

 perl -i~ -pe 's/^echo ok\.$/t-ok/' tests/tests/*[^~]
 perl -i~ -pe 's/^echo ok\.$/t-ok/' tests/setup/*[^~]
 perl -i~ -pe 's/^echo done\.$/t-ok/' tests/tests/*[^~]
 perl -i~ -pe 's/^echo done\.$/t-ok/' tests/setup/*[^~]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.14

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.13

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: document dgit-sponsorship fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-sponsorship(7): Update in light of fixed #844129

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: document dgit-maint-gbp fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-gbp(7): Remove reference to closed bug.

Thanks Felipe Sateler for the bug report.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

Import fix: Switch back to unpa branch on patch import iterations.

In particular, do not fail utterly if dpkg-source and gbp
disagree. Closes:#848843.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Improve "cannot represent change" message

Print the git old and new modes too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

quilt fixup: Permit creation of patches which delete files

By psssing --include-removal to dpkg-source, and tolerating it when we
do our quilt fixup analysis.

dpkg-source has supported this since at least stretch.

Closes:#848901.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.13

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.12

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Do not fail to import a .dsc containing patches to .git/ (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git Do not fail to import certain weird .dscs

A .dsc containing patches which patch files multiple times would be
rejected by dpkg-source due to #848611.

Work around this by arranging for the patch of interest to never be
the first patch, by introducing (and then evaporating) a dummy patch.

Yuk.

Closes:#848391.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Produce better error reporting when absurd git wrapper fails on a patch

during .dsc import.  Apropos of #848391.

gbp swallows the error from the first run (without --whitespace=fix).
Previously only that first run would do anything useful; the second
would complain about --whitespace=fix.  So failures of dpkg-source
would always produce a useless error message.

Instead, tolerate (and ignore) the --whitespace=fix option which gbp
passes during the second run.  This means that the second run tries to
do the actual work.

This only does extra work if the first run failed, and since the first
and second runs are now equivalent, that happens when the second run
fails too.  But in return for that extra work, we get a report which
actually mentions something that unexpectedly went wrong.

(The alternative would be to have absurd/git try to stash and then
reproduce its previous error, or to make more fragile assumptions
about gbp's behaviour.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Better debugging

Honour DGIT_ABSURD_DEBUG, which is set by dgit from $debuglevel.
Also, in dgit, open the output file for append.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd/git: Introduce log()

Right now this simply replaces all the echo >&2, producing no
significant functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Do not fail when cloning a package containing dangling symlinks.

Closes:#848512.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

By default, generate a DEP-14 tag as well as a dgit archive/* tag

Even in non-split-view quilt modes.  Closes:#844129.

This breaks one of the test suite test cases, so disable it there.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out access_cfg_tagformats_can_splitbrain

While we're at it, use a hash %y rather than repeatedly grepping.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Suppress some leftover debugging output from import-dsc. Closes:#847658.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

If we cannot hardlink origs into our extraction area, use symlinks instead.

Closes:#844570.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Give better advice if .dsc/.changes signing fails

If no changes are needed to the package, user may indeed just debsign
and dput.  Closes:#844131.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Mention #844128

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document closure of #847987

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-native: clarify which sort of native

Slightly better message when .dsc not found.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Version tags mangling: Protect dots, as per proposed update to DEP-14.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Introduce dep14_version_quote

Removes three identical copies of this y/// rune.  No functional
change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Mention dgit-maint-merge changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge remote-tracking branch 'spwhitton/maint-merge-updates' into fix

Provide and use stunt lintian and debuild

This avoids lintian complaining about our stupid test packages.

We need the stunt debuild because debuild likes to launder the
environment, removing our stunt lintian from PATH !

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-policy-debian; Unset GIT_ALTERNATE_OBJECT_DIRECTORIES

This broke due to recent git setting GIT_ALTERNATE_OBJECT_DIRECTORIES
in the pre-receive-hook.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Add upstream remote after dgit clone

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Missing 'first'

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Add upstream remote when converting

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): After converting, need --overwrite

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Converting existing packages

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Missing -o

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: start 2.12~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.11

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Improve formatting of rpush section.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1), dgit(7): Better reference docs for combined suites.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document dgit-user(7) updates

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Explain comma notation better.

Wording suggested by Sean Whitton, and amended.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Suggest avoiding ,-security for sid

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Replace make in Test-Depends with build-essential.

Most of the tests do in fact run dpkg-buildpackage which bombs out if
build-essential is missing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.11~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.10

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): More info about multisuite branch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Gardening

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: import-tarbomb: Split out from import-nonnative

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: lib-import-chk: Break out from import-nonnative

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Tests suite: import-nonnative: Put a .pc in the craziest .orig too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: remove_stray_gits: Print a slightly better message

Change all call sites, and callers of mktree_in_ud_from_only_subdir.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: git_add_write_tree: Replace two open-coded git-add sites

No overall functional change.  (We end up passing `.' to git add,
which these two sites didn't previously.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: import-nonnative: Test origs with .git directories

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: import-nonnative: Test tarbombs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Cope when an orig tarball is a tarbomb.

Ie, if it contains other than one single directory toplevel.
Closes:#843422.

Specifically:

 * We change the unpack directory to _unpack-tar in unpack, rather
   than ../unpack-tar.  This means that mktree_in_ud DTRT in that
   directory, rather than in its supposed only subdiredctory
 * We do write-tree on the whole tarball.
 * Then if the tarball had only one subdir in it, we fish the
   subtree's tree object out of the git tree object.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: import-nonnative: Check that .orig contents match imported trees

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: import-nonnative: Prepare for more tests

* Break chk into chk1 and chk2, so we can do stuff in the middle
* Remove $p between runs, so that we can do another test with the same $p

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Introduce git_add_write_tree

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Fix an unconditional print that was supposed to be a printdebug:

origs <blah>.orig.tar.gz f.same=1 #f._differ=-1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: aptget: un-disable cache clean

Have fudged it to see what it did, and it seems to work.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Add caveat about multisuite-generated branches

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: clone-reprepro: Use stunt parsechangelog

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Better error message for inappropriate multi-suite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Document multisuite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: reprepro: Test multisuite with package absent in suite

It is convenient to do this in this test case, particularly as that
lets us check that our apt-get method handles missing packages
properly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: multisuite: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Implement multi-suites

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: lib-reprepro: cope with non-codenamed suites

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-archive: DTRT if $suite is not sid

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Actually honour the branch name for $isuite, if we are on dgit branch

As documented in the manpage.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Print a debug message if clone removes due to error

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Move canonicalise_suite into fetch()

fetchpullargs has to run before we have dealt with multisuites, which
cannot be straightforwardly canonicalised.

Move the canonicalisation out of fetchpullargs.  Previously,
fetchpullargs would always call canonicalise_suite if it wasn't going
to fail.  In one of the cases it would print a message, but we have
just changed canonicalise_suite to print a message too, so the one
from fetchpullargs was redundant.

fetchpullargs is called right before fetch, in cmd_fetch; and, in
cmd_pull.  In cmd_pull the split brain check intervenes between
fetchpullargs and pull which calls fetch.  So overall there is now no
functional change other than the loss of the now-redundant message.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

clone: Postpone canonicalise_suite

We are going to want to do some other things beforehand, but after the
dryrun safety catch.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Make clone_set_head not depend on $giturl

When cloning a distro which has no git server, correctly leave the
user on the local dgit branch, not on `master'.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clone: Break out clone_set_head and clone_finish

Again, we are going to want to reuse these bits.

No functional change right now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: fetch_from_archive: Break out _record_1 and _2.

We are going to want to reuse these bits of the fetch finishing-up.

No functional change right now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Always print canonical suite name

Not only when a noncanonical one was specified.  This makes some
output less confusing - especially when combined with the forthcoming
multisuite feature.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: printdone: Use $us.

No significant functional change right now.
When $us changes, it will be more right.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Pass --force-distribution to dch

If it's wrong, we know what we're doing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Provide reprepro test (for aptget method)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: orig-include-exclude: Remove some commented-out obsolete bit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Introduce $t_archive_method

* Indirect t-archive-none and t-archive-query via this variable

* Rename t-archive-updated; it is for callers that have updated
  aq/package.$suite.$p.  There is one out-of-lib caller.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: canonicalise_suite_aptget: Add a bit of debug

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Support the Debian *-security suites.

(Provide the configuration to use the aptget method.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: aptget archive access method

Uses Sources.  This is not very efficient and should be avoided for
large apt repositories.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: config: Allow dgit-suite.PATTERN.distro

This is going to be useful for *-security.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Introduce access_nomdistro()

This distro name is used for tag names, protocol elements, etc., and
also as a search element for config lookups.

Each distro can specify a nominal-distro.

No functional change with current config.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Introduce archive_query_prepend_mirror

Now the archive_query functions return urls, not subpaths.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: parsecontrol: Support $allowsigned (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add fakeroot and make to Test-Depends.

These aren't necessarily pulled in by anything else.  (dpkg-dev
Recommends build-essential.  But we don't actually need
build-essential.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.10~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

finalise 2.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>