changelog: start 9.10

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 9.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document changes since 9.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

dgit-maint-bpo(7): Mention occasional need for --new

Closes: #935443
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debpush(1): warning about use of forcing options

Suggested-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Closes: #935084

git-debpush(1): note that --quilt=baredebian+git is an alias

Suggested-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Relates-to: #935084

infra: dgit-mirror-ssh-wrap: Fix $package_re

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add missing dependency on liburl-perl

This is only a Recommends in dgit.deb so we need to pull it in
explicitly for the tests that need it.

We add the dependency unconditionally to the test suite tests that
depend on dgit, since that avoids having to track exactly when it's
needed.  (This module has an Installed-Size of 228.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Add missing dependency on liburl-perl

dgit uses URI::Escape for archive_query_aptget.

This is a Recommends, because if you never use the aptget method you
don't need it.  (But few installations can do without the aptget
method entirely.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test dgit-mirror-ssh-wrap

This script will break if rsync changes the way it passes arguments to
the peer.  We need to catch that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

infra: New script dgit-mirror-ssh-wrap

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

lintian: Override source format warnings

I also don't agree that missing source format is a problem.
The semantics are well defined.

There are difficulties with 3.0 native (for example, that it forbids
non-native versions) so 1.0 must continue to be used so that it
continues to exist.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

lintian: Override some checks

The mannpage message on line 51 is
  warning: file `<standard input>', around line 51:
    table wider than line width
and refers to this line
  dgit-maint-gbp(7) for maintainers already using git-buildpackage
which does in fact fit (just) in an 80-column xterm.

I hope dgit-badcommit-fixup is never going to be needed enough again
to need a manpage.

I don't agree that trailing whitespace in changelog is a problem.

Remaining messages from

  lintian -EI --pedantic --no-tag-display-limit --suppress-tags=syntax-error-in-debian-changelog,changelog-empty-entry ../bpd/dgit_9.9~_multi.changes

are as follows:

  W: dgit source: missing-debian-source-format
  I: dgit source: older-source-format 1.0

I don't agree that source format 1.0 is wrong.  There are difficulties
with 3.0 native (for example, that it forbids non-native versions) so
1.0 must continue to be used so that it continues to exist.

I also don't agree that missing source format is a problem.  The
semantics are well defined.

  P: dgit source: package-uses-old-debhelper-compat-version 9
  W: dgit source: unnecessary-testsuite-autopkgtest-field

I like to keep things backwards compatible for a much longer time than
most people.  jessie is currently oldoldstable and has dh 9.

  P: dgit source: no-dep5-copyright

I don't have effort for this.  I think it is largely busywork but I
would accept a patch.

  I: dgit: package-contains-empty-directory usr/share/man/man5/

Avoiding this is awkward.

  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-mirror-rsync
  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-repos-admin-debian
  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-repos-policy-debian
  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-repos-policy-trusting
  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-repos-server
  W: dgit-infrastructure: binary-without-manpage usr/bin/dgit-ssh-dispatch

In principle manpages for some of all of these would be good.  But the
usefuless of these programs is largely limited to people whose
requirements are extremely similar to Debian's.  Most people can just
use a normal git server.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

control: update Vcs-Git to end in .git

In
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=888809
Paul Wise writes
  Salsa redirects git to URLs ending in ".git/" and redirects browsers
  to the URL not ending in ".git/" or ".git".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

control: Drop redundant Priority field

Found by lintian.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Extend extended description.

Mentioning what the competition is may help both discoverability and
useability.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix a typo (found by lintian)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

control: Bump Standards-Version

I have been through the upgrading checklist.

I came across two things which were mentioned but which I haven't
implemented:

 * nodoc build option.  It's not clear to me whether this is
   superseded by the nodocs build profile.  If we wanted to bootstrap
   using dgit, the build profile would probably be better.

 * using ${perl:Depends}.  I looked into this briefly and dh_perl
   doesn't seem to DTRT right now: it doesn't spot any of the modules
   we use.  I'm not sure why and have decided to punt on this for now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

control: Change priority of dgit-infrastruture to extra

Pursuant to policy upgrading checklist.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-{merge,debrebase}(7): Use git fetch --tags upstream

We recommend using a remote called 'upstream' and so we can recommend
a command which will be faster and less likely to error out.

Closes: #939679
Suggested-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: start 9.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 9.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document changes since 9.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

test suite: Check that archive skew machinery reports warning

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Check that archive skew machinery succeeds

This is actually the point of this test case.

perling $tmp/aq/package... is a bit of a bodge, but fine.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Once again cope with archive skew

Since our http rearrangements, we have lost the ability to try
multiple dsc versions which means we cope badly with archive skew.

This was an editing error: in 7821907696db
   dgit: Abolish url_get in favour of url_fetch
we omitted to notice that url_get implicitly does what url_fetch needs
Ok404 to do.  So we didn't add the Ok404 parameter.

Fix this by adding it now.

Closes: #935874
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: clone-skew: Test for accessing mirror via http

This test is misnamed right now, but really we are going to make it do
something interesting.

Right now it just sets everything up to access the mirror via http
rather than file://.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-archive: Honour t_archive_ln_s

This will let us arrange to use cp instead.

No functional change for existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: curl: pass CURLOPT_FOLLOWLOCATION

This was accidentally dropped in the 9.6 http changes.

Closes: #939564
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-{merge,debrebase}(7): Use git fetch --all --tags

`git remote update` doesn't fetch tags unless remote branches include
the commits at which those tags point.  Thus, if upstream pushes their
release tag but fails to push their master branch, `git remote update`
will not fetch the release tag.

I've been in this situation more than once when following the
workflows detailed in these manpages, so let's just recommend a
command which will definitely try to fetch the latest release tag.

Closes: #939504
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test already-tagged check

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fail early when the archive/ tag already exists.

Suggest using a new version number.

We need a new --force option to override this check.

Closes: #935802
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Add --force-reusing-version to many tests

This is going to be needed in a moment.  Unknown --force options are
harmless and just generate warnings, so we do this in advance.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Another semantic linefeed

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Warn when we see what look like our options passed too late

Closes: #934807
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Use distro-info-data to resolve unknown suites to distros

Closes: #931212
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Some semantic linefeeds

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Use $tarball_f_ext_re for test_source_only_changes

The old ad-hoc regexp doesn't match .tar.gz.{asc,sig}.

Closes: #939280
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit: Break out $tarball_f_ext_re.

We are going to reuse this.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase(5): Tiny typo fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 9.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 9.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document chages since 9.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

dgit: vcs-git handling: Strip [...] a la Haskell Team

The final syntax of this is not yet agreed but this regexp ought to do
the right thing in all reasonable cases.  (Ie, no [ ] inside the [ ].)

See also #932696 against policy, which is discussing the spec.

Closes: #932699
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: update-vcs-git: Use vcs_git_url_of_ctrl

So we strip -b fragments as we ought.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: update-vcs-git: Print a less confusing message if unchanged

Previously, we would say it was "already configured" which is
unspecific and sounds a bit like an error.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: update-vcs-git: Do not crash if url is unchanged

Previously, we would pass an empty command array to runcmd, causing a
mysterious-looking failure.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out vcs_git_url_of_ctrl

This should have been done before.  No functional change so far.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix on NFS (avoid leaking an open file in a directory to rm)

The file debian/source/format was open by "dgit fetch" but never
closed, causing failure to remove the extracted tree on NFS.

Closes: #933827
Signed-off-by: Stéphane Glondu <glondu@debian.org>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: alternating-dgit: New test

This tests alternating dgit and non-dgit uploads.  These were broken
by the bug #934126 (check_for_git always returns false), in the http
rework, and not detected by the test suite (!)

This test *does* break if check_for_git always returns false.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Use http git check for three existing tests

At the very least we want at least one test which fails when
check_for_git is nobbled ad-hoc [1] to return 1.  "clone-nogit" fits
that bill (verified experimentally).

Pick two other tests to enable this for, roughly arbitrarily.
(Two tests which chain to clone-nogit are affected too.)

Of course we also want a test which fails when check_for_git falsely
returns 0.  We don't have one right now.  Coming up...

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: WWW::Curl: Double check that $response_body was set

We are making some assumptions about how WWW::Curl behaves. This
confess, with appropriate tests, should catch any future problems.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: WWW::Curl Set $response_body to '' explicitly

check_for_git calls url_fetch with a WWW::Curl parameter
CURLOPT_NOBODY which suppresses saving the body.  The result is that
CURLOPT_WRITEDATA does not update $response_body.  Then, even on
success, url_fetch returns undef (which ought to mean 404).

Strictly this is a bit sneaky of check_for_git.  But, rather than
trying to make this interface more formal, just make url_fetch set
$response_body to ''.  It does return this except in the success case,
so the error cases are still handled right.

Closes: #934126
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: examplegit: Call t-select-package

This can save our importers from havig to t-select-package themselves.

Drop the now-redundant call from the one importer that had it.
Various of the importers call t-archive-* or t-git-* that also call
t-select-package, but this is not a problem.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Break out t-non-dgit-upload

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: http-git-check: New provision

Importing this makes us use a local http server for the git check.
It's a setup so that it can imply the http-git-check dependency.

Right now nothing imports this so no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Honour new DGIT_TEST_KEEP_MUSTCLEAN

FTR, using this typically results in runes like

  DGIT_TEST_KEEP_MUSTCLEAN=y tests/using-intree tests/tests/ftpmasterapi-http 2>&1 | tee tests/tmp/log

hanging instead of exiting, because the http server is still running
and its stderr is still going to tee.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Rename http-static-server from ftpmasterapi-

We are going to use this for other things too.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

http[s] tests: Add http test case

Maybe we will turn this into a TLS test case in due course.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

http[s] tests: Fix stdout handling in ftpmaster test server

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

http[s] tests: More-or-less functional ftpmaster http server

Not used anywhere yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

http[s] tests: Import and slightly hack example script

This is basically a copy of some of the example code from the
HTTP::Server::Simple docs, hacked up with my own (so far) nonsense.

Add copyright info to file and d/copyright.  Upstream licence is
Artistic or GPL-1+.  We choose GPL-1+ and upgrade to GPL-3+.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib: Provide $tmp/must-clean directory to tests

This is a directory for things which should be removed on exit.
Currently there are none.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Slightly more debugging for check_for_git with http[s]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Fix a typo in the entry for 9.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document changes to enumerate-tests

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: enumerate-tests: Spot t-alt-test

And find the dependencies of the referred-to test.  No functional
change since currently no chained-to test has any nonstandard
dependencies.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: enumerate-tests: gencontrol-add-things: restrictions

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: enumerate-tests: gencontrol-add-things: delim feature

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: enumerate-tests: Break out gencontrol-add-things

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Start 9.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Write and finalise for 9.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-playtree-create: Drop an unwanted set -x

Closes: #932959
Reported-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/control: Drop now unused dependency on libwww-perl.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Drop mention of LWP::UserAgent since we don't use it now

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Abolish url_get in favour of url_fetch

This removes use of LWP from dgit.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Use libcurl, not @curl, for git check

This means that when we sort out our TLS problems this will be fixed
too.

The remaining use of command-line curl is just to download additional
.dsc compoments.  I intend to leave that as-is.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: url_fetch: Honour new CurlOpts

No functional change with the existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: url_fetch: Honour new AccessBase

No functional change with the existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Reorganise url_fetch

This was archive_api_query_curl.  We're working on making it good for
other things too.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: libcurl usage: Install SIGPIPE handler ourselv

We don't want a global SIGPIPE setting.  In particular we do not want
to run any of subprocesses with SIGPIPE ignored.

Just in case libcurl gets this wrong, tell it CURLOPT_NOSIGNAL, and do
it ourselves.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Set $SIG{INT} to DEFAULT

There is a bug in libdpkg-perl, where Dpkg::Source::Package installs a
handler on loading.  This has the effect of making in-process C
calls (eg to libcurl) uninterruptible by ^C.

Work around it by unconditionally resetting SIGINT.  (Anyone who
deliberately invoked us with SIGINT ignored deserves to keep all the
resulting pieces.)

See #932841.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: archive_api_query_curl: Add a bit of debug

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Use WWW::Curl rather than invoking curl(1) for archive api

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Drop support for old attempts at cert/key pinning

I don't think anyone is using these.  We are going to do something
different to try fo fix #932570.

When #790093 is fixed we may need something like archive-query-tls-key
(or even exactly that) but we would have to reimplement it for
WWW::Curl anyway.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

tag2upload: Use the non-readonly method for dgit fetch

The principal effect in the Debian configuration is to access the
master dgit git server via ssh, rather than the mirror via https.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Provide --for-push option

This is much more conveient and in some cases sensible than
 -cdgit-distro.DISTRO.readonly=0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Make cmd_archive_api_query call api_query_raw

In support of #932570.  Now there is only one caller of
archive_api_query_cmd.

It is OK to pass `undef' because
  - archive_

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Move $data check into api_query

In support of #932570.  Our intended other caller doesn't have a
$data.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out api_query_raw

In support of #932570.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Add missing Closes of #932476 to changelog entry for 9.5

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 9.6

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: finalise 9.5

changelog: Bug close syntax fix

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

changelog: Minor fixes

Mention two omitted bug Closes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Mention Vcs-Git change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>