dgit-user(7): work in progress

Used one of the existing tutorials as a template, so currently
contains many lies (including the AUTHOR).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: orig-include-exclude: Move bulk into a lib-*

We're going to want to re-use it as-is.  Well, modulo the introduction
of $test_push_2_hook.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: orig-include-exclude: Refactor: introduce test-push-[12]

No functional change (diffed two test logs to check).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: orig-include-exclude: Tests orig auto-inclusion/exclusion

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-archive-process-incoming: Process everything

The previous rune wouldn't process any new .origs.  And in fact there
is no good reason to restrict this.

certainly not right now, because right now we are going to want to
process whatever is in incoming so that the orig-include-exclude test
can easily see what dgit push uploaded.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Automatically calculate which .origs are required

* Spot if any of our .origs have different hashes to the archive's
* Update the .changes to have the set of .origs that the archive doesn't

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Switch by default archive access method to dummycatapi

Now dgit can, during the test suite, make file_in_archive queries,
without exploding.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dummyapicat: New archive query method

Like ftpmasterapi but knows that actually behind the scenes it's just
mostly cat.  Ths is relevant because file_in_archive_dummycatapi has
to be different.

(This is because the test suite can't sensibly predict what queries
dgit might make, because the relevant bits of the test suite don't
actually really know what .origs might be supposed to exist.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Some archive queries: Remove prototypes

That makes these easier to reuse for another method.  No functional
change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dopush: Break out $changes variable

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dopush: Move $upstreamversion setting out to give it wider scope

No functional change yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

@files_csum_info_fields: Provide ftpmaster api database field name

No functional change as nothing looks here yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

archive_query: Provide file_in_archive methods

No functional change as no callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

archive_query: Support further arguments to methods

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

api query: Provide ability to tolerate 404

This is useful for backward-compatibility with earlier ftpmaster api
versions.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Test --new in split brain mode

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain mode: Fix --new.  Closes:#842354.

Test $archive_hash for undef early enough.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.9~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build modes handling: Check for stray .changes on all split brain build entries

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build changes handling: Break out midbuild_checkchanges

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: build-mode-*: Check that right .changes comes out.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build changes handling: Run mergechanges when needed in non-sbuild build modes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

build changes handling: Introduce postbuild_mergechanges

Affects only the `sbuild' subcommand.

Mostly code motion.

Other changes include:
 * Provide in_parent, and use it.
 * "sbuild" changed to "build" in the msg_if_onlyone failure message
 * Unapply patches before merging changelogs
 * Reorganise so that it is possible to use this code when there
   is only one output changes file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: bm-*: turn bm-report-source into bm-report-test

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

absurd git-apply fallback: defend against debian/patches/series being an unusual object, in case dpkg-source doesn't.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.8~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Provide absurd-gitapply test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Allow --force to enable/disable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Ignore unknown force options (with a warning)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-dsc-changes-mismatch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-unsupported-source-format

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Docuemnt absurdity in changelog

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Import: Use absurd `git apply' emulation if gbp pq import fails

gbp import can fail due to git apply not understanding patches.
This is #841867 (against dgit).

The underlying problem is #841866 (in gbp pq) which exposes things
like #841865 and #829067 (in git).  I imagine there are other lurking
incompatibilities between git-apply and dpkg-source.

We could in principle reimplement the gbp patch metadata extraction.
But that would be quite tiresome and have its own compatibility
problems.

The real problem is just `git apply'.  (Indeed gbp already tries git
apply without, and then with, a whitespace fix option.)  We work
around the trouble by providing our own implementation of `git apply'.

Specifically:

We try to do things the sane way (just running gbp pq import) first.
If that works, great.  If it doesn't, we put /usr/share/dgit/absurd on
the PATH.  That contains only a sh script called `git'.  This sh
script figures out whether the caller is trying to invoke `git apply'.
If not, it runs the real git.

If the caller wanted git-apply, the absurd git script emulates it
using dpkg-source --before-build.  Conveniently, we know that the
series file will not be touched by patches.  So we can write just the
patch we care about into the series file, and run --before-build,
which applies just that one patch.

The results are committed (minus the .pc), and for the next patch,
dpkg-source sees again a tree with simply a single patch to apply.

We try ordinary gbp pq first because our absurd approach is very slow
on a big tree.  Also we would like to maximise our chances of the
import working.  If git and/or gbp ever work better by themselves, all
of this craziness will simply not happen.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Build system substitutions, for absurdities

* Introduce absurddir, which is where the absurdities will go
* Substitute the absurddir into the dgit script during make all
* Install the substituted scripts rather than the originals in make install
* Provide a default in dgit, based on $0
* Clean the substitutions up in make clean

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Break out failedcmd_waitstatus

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --force-unrepresentable.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: --force-*: Provide framework

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Some more info about --deliberately.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Reorder the options, moving more important ones earlier.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 2.7~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: ftpmaster api fetch: Check http error code (!)

We are going to want to handle 404 specially so do not just use -f.

Handle file:/// specially since curl just always invents 000 for the
http error code in that case.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: failedcmd_report_cmd: Break out of failedcmd

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: archive-api-query subcommand: pass -f to curl

This makes it fail properly when it should.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: Always honour @curl

Use --curl= and --curl: in ftpmaster api queries and in http git
checks.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

curl invocation: Remove -f from @curl, and add it at the call site

No significant functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.6~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 2.5

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Provide --gbp and --dpm as aliases for --quilt=gbp and --quilt=dpm.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Tidy up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

solit brain: Do not crash in when the two brains are actually identical.

The dgit-view ref is created by quiltify_splitbrain_needed, which is
not always called.  But, quiltify_splitbrain_needed's other function
is to switch branches.  If it was called then HEAD will refer to
dgit-view.  If it wasn't then the original HEAD is fine.

Closes:#841770.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

default config: Switch to new archive/ tag format

Even in non-split-brain mode.
Also change the test suite to expect this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: overwrite junk: Use $tagpfx where applicable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Introduce setup/gnupg

To work around #841143 and improve performance by amortising gnupg
migration cost.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Propagate tmpbase into nested (setup) runs

This stops t-setup-import wrongly deciding we are in ADT non-shared
mode.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: setup import: Cope with nested and multiple imports

Name the IMPORT file after the setup name.

This is important when running under autopkgtest, where all the setups
share the tmp directory with each other and with the main test.  The
setups need distinct IMPORTS.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Add paragraph split

Suggested-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Move remarks on upstream orig

See discussion in #840153.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-merge(7): Tweak wording

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

debian/changelog: Document dgit-maint-merge(7)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Fiercer instructions about legally dangerous git history

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Fix a typo "upsteram"

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Mention what to do if upstream make good tarballs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpages: Add note about other contributors to dgit-maint-merge(7)

I am going to edit this manpage myself in a moment.  In general we
don't want to have to update the AUTHOR section manually.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpage: add dgit-maint-merge(7) to SEE ALSO

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

debian/copyright: add myself

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Manpage: add dgit-maint-merge.7.pod

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Makefile: build and clean prospective *.7.pod

Build the pod section 7 manpages in "make all"; and install them,

There aren't any yet, so this does nothing.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.5

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Document which --ch: options are a good idea.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Add a dummy ref update when necessary

git-update-ref does not make a new reflog entry if the ref is not
actually being changed.

This can mean that `dgit build' might fail to find a good cache entry,
but also fail to generate one that push will accept.

In this case, generate a dummy reflog update.  We make a spurious
child of the $dgitview, as that's straightforward.  Giving it the same
tree and a parent means that human use of reflog commands will produce
empty results rather than lots of confusing output.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Use git log -g

git-reflog is

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

split brain cache: Fix a wrong implicit reference to $_. Closes:#841383.

Otherwise we can get:

  Use of uninitialized value $_ in pattern match (m//) at /usr/bin/dgit
  line 4364.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-cache reflog: Make note in comment about --create-reflog

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit gbp-build: No longer invent a --git-debian-branch option.

Usually the user is a maintainer using split brain, and we should rely
on their own gbp configuration to specify the right check.
Closes:#841100.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document fixes to #841101

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: overwrite-splitbrains: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: Suppress checks in split brain mode with --overwrite

If the user has specified an appropriate --overwrite (=VERSION with
the archive's version, or without a version and the archive's version
being in debian/changelog), do not do all the automatic checks.

Instead, in this case, simply make the dgit view be a descendant of
the archive, as instructed.

Closes:#841101.

When generating split brain pseudomerge with --overwrite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: When generating pseudomerge in quilt split brain mode due to --overwrite, actually include the version number in the commit message.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-gbp-example-prep-no-ff: break out from t-gbp-example-prep

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: Abolish confusing $dgitview variable

This seems mostly to have existed to avoid writing `my $cachekey'
separately.  It has no separate function from $dgithead AFAICT.
Abolish it.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: ovewrite-junk: New test

Check that we can overwrite things even if the tags are broken.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: ovewrite-version: Add missing "echo ok."

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

pseudomerge: In non-splitbrain, do not look at previous tags

We only want to know: is the current archive version what was
specified in --overwrite=VERSION, or mentioned in the changelog with
--overwrite (without VERSION).  In that case we want to overwrite
whatever it is, even if the tags are missing or broken.

Prompted by #841101.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 2.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Detect SIGPIPE (and SIGCHLD) being blocked or ignored. Closes:#841085.

In fact, it seems in my test that Perl resets SIGCHLD itself, printing
something to stderr, so that trip does not actually fire.  But it
makes sense to keep it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian config; actually make split brain mode work.  Closes:#841085.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 2.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add test that #840989 is fixed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Import handling Do not crash due in clone

Previously we failed to handle dpkg-parsechangelog SIGPIPE.
Closes:#840989.

Fixes:
  dgit: failed command: dpkg-parsechangelog --format rfc822 --all
  dgit: subprocess died due to fatal signal PIPE

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Filter out .../git-core from PATH so that we catch git-foo

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document git- prefix changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Test suite: fix tests/tests/quilt

Broken since 20ee7135bcbb "Test suite: quilt: Test that our patch
makes it through dgit and gbp pq".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git- prefixes: Test suite: Fix t-gbp-example-prep

Broken ever since it was introduced.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>