dgit: Break out `infopair' functions

We are going to want to reuse these in a moment.

No functional change other than to debug output.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain pseudomerge: Test $overwrite_version for definedness

Technically, `0' is a valid version number, but it is falsish.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Only send rpush maint-view param if $maintviewhead

Fixes a bug where rpush would always make two tags even for a
non-`3.0 (quilt)' package.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Have quilt-splitbrains test now

Test suite: quilt-splitbrains: Comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-splitbrains: Rename

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Test --quilt=unapplied and --quilt=dpm

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-gbp-pushed-good: Refactor (nfc)

Provide various t-splitbrain-*.  These will be needed for tests of
--quilt=unapplied and --quilt=dpm.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Some extra error reports for wrong split brain mode

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Allow --quilt=dpm

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add two "echo ok."s

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: oldnewtagalt: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add spelling test

Fail this test if the tree contains the misspelling of "pseudo".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Fix spelling of "pseudo"

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Turn gc.auto off in our hidden working tree setups

This avoids potential accidents: git gc --auto might otherwise run and
do very undesirable things to the user's working tree.  (I haven't
been able to trigger such a situation deliberately and none has been
reported, luckily...)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: examplegit: Importers inherit majorv too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Add a todo list item

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Add t-tstunt-parsechangelog to a couple of tests

inarchivecopy and the quilt-gbp tests just use the `example' package.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Done drs-quilt-gbp (actually debpolicy-*)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gbp-build-modes-sbuild: Split out

Now the other tests can run even if the sbuild chroot is not
available.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gbp-build-modes: Lift bulk into lib-build-modes

To split the sbuild test out, we want this to be reusable.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Gardening (reorg only)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: drs-push-rejects: Test push of same version with different dgit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: drs-push-rejects: Break out re-prep (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Some things done

Split brain: Test suite: quilt-gbp: Test 2nd push (ie, test pseudomerge)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Make pseudomerge

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Test suite: quilt-gbp: Check that DEP-14 tag was pushed

Test that we pushed the maintainer view as well as the dgit view.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Always push the maintainer view tag

Our algorithm for calculating whether to do a pseudomerge to make the
dgit view fast forward from the archive contents depends on the
maintainer view tag (effectively, it uses it to go back from the
archive contents dgit view back to the corresponding maintainer view).

So we need to push it too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: dgit-repos-server: Permit pushing maintainer tag too

Split brain is going to need the maintainer tag on the dgit server.
Allow clients to push the _both_ the DEP-14 and new dgit archive/ tag.

In this case dgit-repos-server can largely ignore the DEP-14 tag.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Add some debug relating to tagwants

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: fail: Strip any trailing newline

This makes it sensible to call fail() with a here doc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: debpolicy-quilt-gbp: New test

Check that drs and Debian policy permit split brain.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Changelog handling: Replace a call to parsechangelog with commit_getclogp

This is less code, and takes advantage of the memoisation.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Changelog handling: No not check out last push any more during raw import

We now get the changelog with mergeinfo_getclogp (which just fetches
the changelog blob).  This working tree is not used for anything else,
and is deleted at the end of the function.  So there is no need to
switch branches.

There should be no overall behavioural change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Changelog handling: Use mergeinfo_getclogp during raw import

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Changelog handling: Provide commit_getclogp (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Push reorg: Have FF check use dgit view, not HEAD

Quilt fixup may generate a dgit view, which is not HEAD, and it is the
dgit view which will actually be pushed to the archive, so it is that
that should be checked.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

del_lrfetchrefs: Improve debug output a bit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Push reorg: Postpone FF check

We need to do this after the quilt fixup, because we are going to make
quilt fixup perhaps generate a pseudo-merge.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: run-all: Ensure tests/tmp/ exists

ca007b4d "Test suite: run-all: record output" introduced a regression:
if tests/tmp/ does not exist on entry, tests/run-all fails.

So create it first.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Push reorg: Introduce $archive_hash in dopush (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

fetch_from_archive: Mention that git_fetch_us must be called first

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: changedir: use confess, to get a stack trace

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

fetch_from_archive: Return the hash (and update the comment)

All the call sites currently treat the return value as a boolean, and
the truthiness and falsishness of return values is unchanged, so there
is no overall functional change.

Also, clarify the head comment somewhat.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Add a couple of comments

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Push reorg: Move ff check out of cmd_push into dpush (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: add some desires

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gb-build-modes: Test --clean=dpkg-source too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gb-build-modes: Fixes, passes now

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: lib-build-modes: Remove a couple of hardcoded versions

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gbp-build-modes: New test, which does not pass yet

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: With sbuild, unapply patches in right dir

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: With gbp-build, provide gbp with patches-unapplied tree

We may still need to apply, and then unapply, the patches, so as to
clean the tree.  But gbp itself expects a patches-unapplied tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Make maybe_unapply_patches_again reset $patches_applied_dirtily

That makes it idempotent.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Some more todo items

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: TODO re fetch patterns

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Add a new desire

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: quilt-gbp: Provide and use t-gbp-pushed-good

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Break out t-incoming-dsc (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Break out t-pushed-good-core (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-output: support expecting empty files

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Push correct view to dgit repos server

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Test suite: quilt-gbp test: More

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: lrfetchrefs: Delete unneeded fetched refs, etc.

Fetch only the refs we (think we) are going to use, and delete the
lrfetchrefs (`refs/dgit-fetch/...') versions of those we give another
name to.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Implement `git-fetch --no-insane'

git-fetch has some very poor behaviours.  Implement a non-insane
version.

We are going to want to not fetch things we aren't going to use, and
this is necessary for that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: tag-updates: Do not assert a that dgit-fetch/ ref exists

We don't actually want dgit to leave these lying around unless it has
to.  So don't insist on that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: inarchivecopy: Is finished

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: inarchivecopy: use t-sametree-parent

Otherwise different history between fetch and clone breaks the test.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Provide t-sametree-parent

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: need to clean up lrfetchrefs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: inarchivecopy: More work

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: inarchivecopy: More work

Select specifically 1.3 for first test, and do a clone

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: inarchivecopy: New test (does not pass yet)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: provide for x-dgit-unfinished restriction

This will allow us to add tests which fail or are broken, without
causing the test suite as a whole to fail.  When the test is good, the
restrction should be removed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Add a comment explaining import discard

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Clarify some messages about archive/local skew

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: New fetch algorithm - try to tolerate in-archive copies

Big comment explains.

Amongst other things:
 * Introduce the mergeinfo concept
 * Abolish fetchspec in favour of lrfetchref
 * Synthesize a pseudo-merge linking the archive's .dsc with the dgit
   server's git branch, as appropriate.
 * Introduce mergeinfo_version.

I have tried to ensure that where the import done by previous versions
of dgit would be correct, the new import is the same.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Remove refs/remotes/dgit/dgit/sid from pari-extra worktree

This, effectively, simulates a previous fetch of something.  Do this
in the actual tests instead: in this case, only necessary in
fetch-localgitonly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: examplegit: make "new" branch be 2.x

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-commit: honour $majorv for major version

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Honour t_dgit_xopts

No callers.  This may come in useful, though, at some point.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: examplegit: Make two pushes to unstable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: examplegit: Save mirror as part of setup

incoming has nothing in it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-setup-*: Fix lock path

Needs to be not in a directory which may not exist yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: Provide t-setup-* machinery

Setups are, essentially, prefixes to test cases.  We arrange to be
able to reuse their state (as explicitly recorded in t-setup-done), so
that manual runs of the test suite can be faster.

We save the tmp base directory (the directory shared by tests), if
there is one.  That will be used for cacheing of setups.  And we save
the environment in form we can use xargs and env to restore.

We use a conventional with-lock-ex and stamp file pattern for the
cache.  The IMPORT file (containing shell variables) doubles as the
stamp.

The setup execution rune is exciting.  This mostly results from the
fact that we want xargs to pass all the env vars to env, and that
xargs cannot be persuaded to append fixed arguments to the arguments
read from its input file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: examplegit setup script

We are going to have `setup' scripts which explain how to set up the
preconditions for one or more tests.  This is the first one.

Right now it runs to completion, generating its deliverables in $tmp,
but then crashes due to the as-yet-unimplemented t-setup-done call.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-commit: Permit specification of the suite in $3

This also means that we need to default $2 with ${2:-} since callers
want need to pass an empty value for the version, to get the default.

No functional change since none of the call sites use this yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-pushed-good: Support other suites

Honour $2 being the suite codename, defaulting to sid.

No functional change, since no-one passes this argument right now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Test suite: t-archive-none: Support multiple suites

t-archive-none now honours `suitespecs', which defaults to just
`sid:unstable'.

No overall functional change since no-one sets suitespecs to a
non-default value.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Synthetic ff is not to do with $tag, really

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Have checked all uses of $split_brain

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: Update following split tag work

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

TODO.BRANCH: More notes about our plans

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: Test case: Do push

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split brain: TODOs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split tags: Push the maintainer view tag, where supported

We introduce a new `maint' possibility in to the dgit-tag-format
access cfg list, presence of which indicates that the receiver is
content to receive DEP-14-style maintainer-view (ie, non-dgit0 tags.

For now, we default this to true.  However, repos handled by
dgit-repos-server cannot currently cope with this, because:
 * dgit-repos-server currently tries to be compatible with old
   dgits which send DEP-14-named dgit tags;
 * dgit-repos-server doesn't currently permit the creation of
   `extraneous' (ie, non-dgit) tags and branches.

So for Debian, we do not set this flag.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split tags: Genrate maintainer-view tag too

push_tagwants gets a new argument, $maintviewhead, which is defined
iff the quilt mode means we want split tags.  It then specifies the
generation of both tags.

push_mktags gains the ability to make the new `maint' view tag (and
checks that the view is not something else unexpected).

Introduce a function debiantag_maintview for calculating the
maintainer's idea of the tag.  Currently we use DEP-14 encoding (and
this function is a clone-and-hack of Dgit.pm::debiantag_old, but
perhaps we should do something more complicated (depending on the
quilt mode).

For now we do not push the maint view tag anywhere.  The ability to do
that, where appropriate, will come in a moment.

This commit introduces protocol version 4, which includes the new
`maint-view' param.  When have a split brain quilt mode, we need to be
sure that we're using protocol version 4 or out peer may not honour
this, resulting in an annoying failure later (the responder sending a
different number of tags to the number expected).

We also have to check that we're using the new tag format (or we might
want to try to generate two different tags with the same name, which
is madness).

When the quilt mode is not a split brain one, this whole commit should
produce no overall functional change (even though a higher protocol
version may be negotiated).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split tags: Preparation: Rename tag variables etc. in dopush

We are going to have different branches at once.  Rename some
variables to permit this distinction.

We have $actualhead (whatever HEAD is), $dgithead (the dgit view), and
$maintviewhead (the maintainer's view if different, or undef if not0.

Change the existing references.  Also change one reference to HEAD to
$dgithead: when we update the local tracking ref for the dgit view, we
need to use $dgitview.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Split tags: Preparation: Reorganise tagwants and mktags

We are going to want to generate two tags.  The current code structure
is not really set up for this.  Also the knowledge of what tags are
being made needs to be used both in dopush and in the push responder.

So:
 * Introduce push_tagwants, which calculates which tags we are going
   to be making, including some details of them.
 * `View' indicates which tag this is.  For now there is only one,
   `dgit'.  But the tags are going to be in a defined order - or,
   at least, the `dgit' tag will come first.
 * Have push_parse_changelog no longer return the single tag name,
   and abolish the corresponding $i_tag varaiable.
 * push_mktag bcomes push_mktags and takes much of its instruction
   from @$tagwants.
 * push_mktags does more than just making tags - it also updates
   the dsc, and does some checks, so it needs to fish the relevant
   object id out of the dgit view tag.
 * $mktag in push_tagwants can now operate on a tagwant, and
   the tag-making part of push_mktags just iterates over the tagwants.
 * The filename plumbing in dopush is somewhat generalised: we
   collect the filenames out of push_mtags or responder_receive_files,
   and then substitute them into the tagwants.
 * The tag checking iterates over the tagwants.
 * The push spec computation iterates over the tagwants.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>