dgit: move suite name check to after suite rmap

The Ubuntu Cloud Archive has Codenames of the form
bionic-updates/train (the / in which suitere doesn't match); this can
be corrected by suite rmap, so apply that before doing the suitere
sanity check.

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tag-to-upload tests: Tolerate a changed git message

git changed
  Couldn't find remote ref
to
  couldn't find remote ref
and this broke our test case.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Closes: #932475

test suite: tag-to-upload tests: Make message checks regexps

No functional change yet.  All the existing message strings are
already equivalently valid when interpreted by egrep.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tag-to-upload tests: Set LC_MESSAGES in with-mangled

t-expect-fail would do this but we aren't using it here, so we need to
do it explicitly.

This will avoid the tests failing in some locales.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 9.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalis 9.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document changes for tagupl-gbp

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl-gbp: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-debpolicy: mkdir -p the git repo

This makes t-debpolicy compatible with (eg) setup imports that already
made a $tmp/git.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: splitbrain: "bad b/p" prints more info

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: setup/gbp: Make the result the master branch

This means we can drop --git-ignore-branch from gbp-orig.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: break out tests/setup/gbp

This creates a fairly usual set of gbp style git branches/tags.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-gbp-example-prep: Drop t-tstunt-parsechangelog

And hoist it into all callers.  In fact there is only one caller that
doesn't have it already.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document dgit(1) archive-query doc

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1) document the presence of the aptget archive-query method

This is already used for debian-security, and might be useful for
other archives that like madison or similar (e.g. the Ubuntu Cloud
Archive).

Closes: #932321
Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>

git-debpush: refactor to avoid more than one push command

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: update for tag-to-upload server-side fixes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: update for git-debpush improvements

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: check that upstream tag is ancestor of $branch

Closes: #932096
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: gather some information earlier

For use by sanity checks.

Pure code motion.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: cope with spaces in user-supplied upstream tag

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: check for target suite change since last upload

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: check for pushing the dgit view to the maintainer view

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: factor out find_last_tag()

Also determine the last debian/ tag earlier, for use by sanity checks.

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: introduce and call fail_check

Without this, passing --force to override an earlier sanity check
might cause the user to miss the output of a later sanity check, which
would never get run.

We don't want to have multiple forcing options to override different
sanity checks, as that is too much complexity for a script like this.

So always run all checks, and error out afterwards if at least one of
them failed.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: tagupl-baredebian: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: baredebian: Break up baredebian-test-core

For a baredebian tag-to-upload test, we're going to want to replace
the push.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: printcmd: Print in one go

Sometimes this message is used to report failure of `git fetch' etc.
But when such operations fail, they may interleave output from the
remote git, which typically prints
  fatal: The remote end hung up unexpectedly
(referring to the local end as the `remote' end, from its point
of view), after the the local end printed its message and failed.

This sets up a race between Dgit.pm's printcmd and the message from
the remote.  If we're really unlucky, the remote's message interrupts
the output from printcmd.

We can avoid the within-line interleaving - at least, the interruption
of our message - by printing the whole message in one go, so do that.
(Strictly, the approach we use here only ensures that the message
appears in a single write(2) call if it fits in the stdio buffer.)

One of our test cases (tagupl) depends on this message not being split
and has a small chance of failing without this fix.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: baredebian: Move the quilt fixup forward

We are going to split this function up so that our caller can do the
push via tag-to-upload.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tag2upload: baredebian: Pass dgit refs/tags/, not the commit hash

This will enable git to report the upstream tag in its messages.

The tag name is safe to pass here since dgit only uses its referent,
unless it is nice enough to print.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: resolve_upstream_version: Report tag, where sensible

This affect what is show in commit messages (including the parseable
parts) made by dgit and git-debrebase.  Now they prefer to refer to
the tag name if the caller specified refs/tags/ and the tag name is
a nice one.

This is not expected to make much difference for human callers (who
will probably not explicitly qualify their tag refs).  But it will
make a difference for scripts.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tag2upload: Actually recognise and honour --quilt in tag

Non-default quilt modes were completely broken before.

We only didn't notice because the existing tests worked with the
default.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: parsetag: Print a bit more debug about junk

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: Don't crash on --quilt=baredebian

The die needs to be qualified.  This little bit of fcode has never
worked.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: reflog_cache_insert: Use ensurepath for the reflog

Otherwise quilt fixup can fail in a too-fresh git tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Provide `ensurepath'

Like `ensuredir' but also a bit like a more controlled `mkdir -p'.

We use make_path from File::Path.  It dies on failure.

No caller yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl-*: Move t-pushed-good out of t-tagupl-test

We are going to want to do this separately in some cases.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 9.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 9.2 (no source changes)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 9.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 9.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Minor adjustment to git-debpush discussion

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Add a couple of missing entries

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: Remove comment saying it's a prototype

We have done everything to make it safe to use.  Obviously it still
needs some web service queue thing around it to invoke it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush(1): Tweak AUTHOR section

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: Don't pretend to support --baredebian+tarball

Instead of just deleting the option from the argument parser, accept
it, but print a message explicitly stating that the *quilt mode* is
not supported by this workflow (i.e. it's not just git-debpush passing
the quilt mode on to tag2upload that we don't support).

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: Update

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush(1): Fix POD formatting

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: unset CDPATH

This can cause tests to pass which should fail (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test git-debpush --upstream=

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush: Defend against git-deborig #931509

Without this, when git-deborig fails and #931509 isn't fixed,
git-debpush will silently exit with a nonzero exit status which is
very unfriendly.

With this change it does something like this:
  git-debpush: git-deborig failed; maybe try git-debpush --upstream=TAG
  fatal: Invalid object name 'refs/tags/couldn't find any of the following tags'.
It then exits status 127.  This is obviously daft, but making this
any better is even more complex.

When git-deborig is fixed it will do this:
  couldn't find any of the following tags: 1.0, v1.0, upstream/1.0
  tell me a tag or branch head to make an orig.tar from: git deborig --just-print '--version=1.0-1' COMMITTISH
  git-debpush: git-deborig failed; maybe try git-debpush --upstream=TAG
which is still not brilliant but I guess it will do.

The downside, if it is one, is that we lose git-deborig's original
exit status.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Acked-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: tagupl: Drop an unused variable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush: Support --upstream=TAG (and document it)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush: Defend against unexpected git-deborig output

If git-deborig prints something other than a tag name, for example an
error message (see #931509), this approach will bomb out, roughly
appropriately.

This will also become useful when we make upstream_tag overrideable.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Acked-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: Change comment heading

IMO "Principles of Operation" would be a comprehensive reference
detailing the model and functionality.  I think "Design Principles"
is the conventional name for semi-self-imposed choices/constraints.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Acked-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: t-tagupl-test: Check that salsa master was pushed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl-native: Drop obsolete todo comment

We do now test all these, in tests/tests/tagupl.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-tagupl-test: Use "salsa" as the source

This checks that push pushed the tag(s) there.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-tagupl-run-drs: New source parameter

Change all callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl*: Drop use of --no-push

Now we do actually push.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-tagupl-setting: Provide a "salsa" remote

This will give git-debpush somewhere to push to.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl*: Run t-tagupl-settings in $p

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush: Fixes to help with pushing native packages

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Entry for new "git-push-to-upload" workflow tools

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-repos-server: tag2upload: Clarify key situation comments

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Check Source mismatch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test changelog version

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Check changelog info

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Update version when needed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test lack of signature

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test signature verification fail.

This test case is broken because it doesn't call t-tagupl-run-drs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: raw-mangled: Use 1.0-2 tagname in raw-mangled

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: raw-mangled: Fix mangled tagger test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: raw-mangled: Break out

No functional change.

This test case is broken because it doesn't call t-tagupl-run-drs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: A few more error case tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Better message from upstream mismatch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Check the upstream tag syntax only once

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Verify the tag slightly earlier

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: A few more error case tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Improve a message

Could be multiple distro=, in which case any matching is OK.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Drop an obsolete comment

The usage comment says this script should be retried.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test an email: version mismatch

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Improve a reject message

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: log when we fail and email

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Use a different strategy to avoid fsck fail

This leaves us with a $p directory which is often convenient

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test a really broken tag

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Break out and soup up log moving

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Test some wrong tag names

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Break out next-mangle

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: test one bad tag

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl-*: Break out t-tagupl-run-drs

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Drop a debug $_ from quit message

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Use a bpd and clean up origs

This prevents us keeping forever the junk from uploads, and it
prevents us reusing it too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl-native: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-server: tag2upload: Cope with native packages

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush: Numerous improvements

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debpush: Tolerate not specifying a quilt mode if not needed

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Break out t-tagupl-*

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Drop a now-irrelevant chunk of commented code

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: tagupl: Use git-debpush

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>