build changes handling: Introduce postbuild_mergechanges

Affects only the `sbuild' subcommand.

Mostly code motion.

Other changes include:
 * Provide in_parent, and use it.
 * "sbuild" changed to "build" in the msg_if_onlyone failure message
 * Unapply patches before merging changelogs
 * Reorganise so that it is possible to use this code when there
   is only one output changes file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: bm-*: turn bm-report-source into bm-report-test

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd git-apply fallback: defend against debian/patches/series being an unusual object, in case dpkg-source doesn't.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.8~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Provide absurd-gitapply test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Allow --force to enable/disable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Ignore unknown force options (with a warning)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-dsc-changes-mismatch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-unsupported-source-format

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Docuemnt absurdity in changelog

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Use absurd `git apply' emulation if gbp pq import fails

gbp import can fail due to git apply not understanding patches.
This is #841867 (against dgit).

The underlying problem is #841866 (in gbp pq) which exposes things
like #841865 and #829067 (in git).  I imagine there are other lurking
incompatibilities between git-apply and dpkg-source.

We could in principle reimplement the gbp patch metadata extraction.
But that would be quite tiresome and have its own compatibility
problems.

The real problem is just `git apply'.  (Indeed gbp already tries git
apply without, and then with, a whitespace fix option.)  We work
around the trouble by providing our own implementation of `git apply'.

Specifically:

We try to do things the sane way (just running gbp pq import) first.
If that works, great.  If it doesn't, we put /usr/share/dgit/absurd on
the PATH.  That contains only a sh script called `git'.  This sh
script figures out whether the caller is trying to invoke `git apply'.
If not, it runs the real git.

If the caller wanted git-apply, the absurd git script emulates it
using dpkg-source --before-build.  Conveniently, we know that the
series file will not be touched by patches.  So we can write just the
patch we care about into the series file, and run --before-build,
which applies just that one patch.

The results are committed (minus the .pc), and for the next patch,
dpkg-source sees again a tree with simply a single patch to apply.

We try ordinary gbp pq first because our absurd approach is very slow
on a big tree.  Also we would like to maximise our chances of the
import working.  If git and/or gbp ever work better by themselves, all
of this craziness will simply not happen.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Build system substitutions, for absurdities

* Introduce absurddir, which is where the absurdities will go
* Substitute the absurddir into the dgit script during make all
* Install the substituted scripts rather than the originals in make install
* Provide a default in dgit, based on $0
* Clean the substitutions up in make clean

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Break out failedcmd_waitstatus

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-unrepresentable.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: --force-*: Provide framework

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Some more info about --deliberately.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Reorder the options, moving more important ones earlier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.7~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: ftpmaster api fetch: Check http error code (!)

We are going to want to handle 404 specially so do not just use -f.

Handle file:/// specially since curl just always invents 000 for the
http error code in that case.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: failedcmd_report_cmd: Break out of failedcmd

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: archive-api-query subcommand: pass -f to curl

This makes it fail properly when it should.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: Always honour @curl

Use --curl= and --curl: in ftpmaster api queries and in http git
checks.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: Remove -f from @curl, and add it at the call site

No significant functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.6~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.5

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --gbp and --dpm as aliases for --quilt=gbp and --quilt=dpm.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Tidy up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

solit brain: Do not crash in when the two brains are actually identical.

The dgit-view ref is created by quiltify_splitbrain_needed, which is
not always called.  But, quiltify_splitbrain_needed's other function
is to switch branches.  If it was called then HEAD will refer to
dgit-view.  If it wasn't then the original HEAD is fine.

Closes:#841770.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

default config: Switch to new archive/ tag format

Even in non-split-brain mode.
Also change the test suite to expect this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: overwrite junk: Use $tagpfx where applicable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Introduce setup/gnupg

To work around #841143 and improve performance by amortising gnupg
migration cost.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Propagate tmpbase into nested (setup) runs

This stops t-setup-import wrongly deciding we are in ADT non-shared
mode.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: setup import: Cope with nested and multiple imports

Name the IMPORT file after the setup name.

This is important when running under autopkgtest, where all the setups
share the tmp directory with each other and with the main test.  The
setups need distinct IMPORTS.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Add paragraph split

Suggested-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Move remarks on upstream orig

See discussion in #840153.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Tweak wording

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

debian/changelog: Document dgit-maint-merge(7)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Fiercer instructions about legally dangerous git history

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Fix a typo "upsteram"

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Mention what to do if upstream make good tarballs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpages: Add note about other contributors to dgit-maint-merge(7)

I am going to edit this manpage myself in a moment.  In general we
don't want to have to update the AUTHOR section manually.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpage: add dgit-maint-merge(7) to SEE ALSO

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

debian/copyright: add myself

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpage: add dgit-maint-merge.7.pod

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Makefile: build and clean prospective *.7.pod

Build the pod section 7 manpages in "make all"; and install them,

There aren't any yet, so this does nothing.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.5

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Document which --ch: options are a good idea.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Add a dummy ref update when necessary

git-update-ref does not make a new reflog entry if the ref is not
actually being changed.

This can mean that `dgit build' might fail to find a good cache entry,
but also fail to generate one that push will accept.

In this case, generate a dummy reflog update.  We make a spurious
child of the $dgitview, as that's straightforward.  Giving it the same
tree and a parent means that human use of reflog commands will produce
empty results rather than lots of confusing output.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Use git log -g

git-reflog is

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Fix a wrong implicit reference to $_. Closes:#841383.

Otherwise we can get:

  Use of uninitialized value $_ in pattern match (m//) at /usr/bin/dgit
  line 4364.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-cache reflog: Make note in comment about --create-reflog

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit gbp-build: No longer invent a --git-debian-branch option.

Usually the user is a maintainer using split brain, and we should rely
on their own gbp configuration to specify the right check.
Closes:#841100.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document fixes to #841101

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: overwrite-splitbrains: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: Suppress checks in split brain mode with --overwrite

If the user has specified an appropriate --overwrite (=VERSION with
the archive's version, or without a version and the archive's version
being in debian/changelog), do not do all the automatic checks.

Instead, in this case, simply make the dgit view be a descendant of
the archive, as instructed.

Closes:#841101.

When generating split brain pseudomerge with --overwrite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: When generating pseudomerge in quilt split brain mode due to --overwrite, actually include the version number in the commit message.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-gbp-example-prep-no-ff: break out from t-gbp-example-prep

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: Abolish confusing $dgitview variable

This seems mostly to have existed to avoid writing `my $cachekey'
separately.  It has no separate function from $dgithead AFAICT.
Abolish it.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: ovewrite-junk: New test

Check that we can overwrite things even if the tags are broken.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: ovewrite-version: Add missing "echo ok."

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: In non-splitbrain, do not look at previous tags

We only want to know: is the current archive version what was
specified in --overwrite=VERSION, or mentioned in the changelog with
--overwrite (without VERSION).  In that case we want to overwrite
whatever it is, even if the tags are missing or broken.

Prompted by #841101.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Detect SIGPIPE (and SIGCHLD) being blocked or ignored. Closes:#841085.

In fact, it seems in my test that Perl resets SIGCHLD itself, printing
something to stderr, so that trip does not actually fire.  But it
makes sense to keep it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian config; actually make split brain mode work.  Closes:#841085.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add test that #840989 is fixed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Import handling Do not crash due in clone

Previously we failed to handle dpkg-parsechangelog SIGPIPE.
Closes:#840989.

Fixes:
  dgit: failed command: dpkg-parsechangelog --format rfc822 --all
  dgit: subprocess died due to fatal signal PIPE

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Filter out .../git-core from PATH so that we catch git-foo

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document git- prefix changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Test suite: fix tests/tests/quilt

Broken since 20ee7135bcbb "Test suite: quilt: Test that our patch
makes it through dgit and gbp pq".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Test suite: Fix t-gbp-example-prep

Broken ever since it was introduced.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Fix dgit-repos-policy-debian

Use `git for-each-ref' instead of `git-for-each-ref'.

Broken in 8a178bf100d4 "Tag change: Update dgit-repos-policy-debian".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Change `git-foo' to `git foo' in messages

No change other that to error messages, debug output, etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Change `git-foo' to `git foo' in docs and comments

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Improve two error messages relating to .changes handling and sbuild

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Prepare for 2.0 (~~ right now)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Do not fail when git requires --allow-unrelated-histories.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: stunt parsechangelog: Some commented-out debugging

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: tstunt-parsechangelog: Produce Timestamp field

Like official one does, nowadays.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: unset empty redacted paths

Some programs might treat that as a path of implicitly ".".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Fix entirely-broken tstunt PERLLIB arrangements

Specifically:
 * Properly dereference the variable when looking for the stunt
   dir to be there already
 * Do not add "." to the path if the path starts out empty.
 * Crash _with the right message_ in dpkg-parsechangelog if PERLLIB is
   unset.  (PERLLIB mustn't be unset because we're trying to strip
   our own value from it.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: tstunt/dpkg-parsechangelog: Do not complain if PERLLIB is empty.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Minor tidying-up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Add missing dependency on Text/Iconv.pm

This was broken by 66a602dbe193 "dgit: When generating patch filenames
from titles, first transliterate them (lossily) to ascii."

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Add missing dependencies to dgit-infrastructure

Now it doesn't Depend: dgit, it needs its own copies of these two.
I have done  git grep '^use'  and it seems that these are the only
ones that are needed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Add missing dependency on List::MoreUtils

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Infra: dgit-infrastructure has, and uses, its own copies of the perl modules

This avoids introducing a versioned dependency between dgit and
dgit-infrastructure (and also makes it easier to test cross-version
compatibility).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Infra: Introduce Debian::Dgit::Infra

This is used by all the Perl in dgit-infrastructure.deb.
So far it does nothing, so no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add debhelper (>=8) to test suite's global Depends.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gbp: Check that we do not update user's HEAD

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-splitbrains: Check that we do not update user's HEAD

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Tidy up a bit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>