chiark / gitweb /
dgit.git
5 years agodgit: build_source mostly uses a playtree, not $maindir
Ian Jackson [Thu, 26 Jul 2018 10:08:45 +0000 (11:08 +0100)]
dgit: build_source mostly uses a playtree, not $maindir

In particular, it uses a playtree unless we are trying to include
uncommitted changes in the source package (which is not supported in
split brain mode).

This change allows build_source to respect $buildproductsdir.  It will
also enable, later, building a source package from HEAD, or arbitrary
git ref, when the working tree is dirty.

This change also requires modifying build_prep to only conditionally
clean $maindir.  It should clean $maindir only when it's $maindir that
we're packing into a source package, or when we are running any
builder in $maindir.

When we are using a playtree we must build the changes file there too,
because dpkg-genchanges cannot be persuaded to look for the .dsc
anywhere but ../.

Right now build-source still cleans the tree needlessly.

We need to change the test suite to not expect a clean unless running
a builder:

dgit now cleans the tree only when including uncommitted changes in a
source package, and when running a builder in $maindir.  The code in
lib-build-modes does not test the including of uncommitted changes in
a source package.  Thus, it should expect the tree to be cleaned only
when running a builder.

Also, we need to not expect a clean with sbuild, for the same reason.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Drop alwayssplit tests
Ian Jackson [Thu, 26 Jul 2018 11:48:42 +0000 (12:48 +0100)]
test suite: Drop alwayssplit tests

Since
  dgit: make $need_split_build_invocation always true
this option has had no effect, so these tests are redundant.

And change the comment in dgit near the option parsing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: quilt-splitbrains: Drop non-wgf dpm test of gbp package
Ian Jackson [Thu, 26 Jul 2018 07:55:32 +0000 (08:55 +0100)]
test suite: quilt-splitbrains: Drop non-wgf dpm test of gbp package

Soon, dgit push-source will no longer clean the tree.  So this test
will be of no use.  Also, it will fail because the error message changes.

Delete it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: build_prep: Provide it with $wantsrc
Ian Jackson [Thu, 26 Jul 2018 09:36:01 +0000 (10:36 +0100)]
dgit: build_prep: Provide it with $wantsrc

No functional change yet as it doesn't use it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: build_source: drop call to build_prep
Ian Jackson [Thu, 26 Jul 2018 07:09:32 +0000 (08:09 +0100)]
dgit: build_source: drop call to build_prep

We want the callers to do this so they can control it better.  They
mostly already do.  In detail:

* cmd_push_source

  This one is most complicated.  It already calls prep_push, which
  does a variety of things which overlap with build_prep.  The things
  which are in build_prep but not in prep_push are:
   - notpushing (via build_prep_early), which seems to be a mistake
   - clean_tree(), which is needed, so we add it (for now...)
   - build_maybe_quilt_fixup, which is needed, so we add it
   - remove old changes, which is not actually needed

* cmd_build, cmd_gbp_build

  build_prep was already called by the if arm which didn't call
  build_source; so we can just move it up outside the if.  No
  functional change here.

* cmd_build_source, cmd_sbuild

  We need to replace the call.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: build_source: Drop call to build_prep_early
Ian Jackson [Thu, 26 Jul 2018 07:02:09 +0000 (08:02 +0100)]
dgit: build_source: Drop call to build_prep_early

build_prep starts with a call to build_prep_early so this is redundant
(and confusing).

Effect is to sometimes no longer duplicate some effort.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Abolish build_source_for_push
Ian Jackson [Thu, 26 Jul 2018 06:59:30 +0000 (07:59 +0100)]
dgit: Abolish build_source_for_push

This sub has only one caller and the indirection is not really
helpful IMO.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: build_source_for_push does not try to unapply patches
Sean Whitton [Wed, 25 Jul 2018 11:36:52 +0000 (19:36 +0800)]
dgit: build_source_for_push does not try to unapply patches

build_source_for_push should now always build in a worktree, so
patches should never be applied dirtily to $maindir.  Die if it turns
out they were.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
5 years agodgit: Move up $sourcechanges
Ian Jackson [Thu, 26 Jul 2018 07:20:54 +0000 (08:20 +0100)]
dgit: Move up $sourcechanges

This is not really ideal, but cmd_push_source needs it.  For now do it
like this.  I may move cmd_push_source further down the file, later,
and then we can revert this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Support run-all -p, DGIT_TESTS_PROGRESSIVE
Ian Jackson [Thu, 26 Jul 2018 05:36:51 +0000 (06:36 +0100)]
test suite: Support run-all -p, DGIT_TESTS_PROGRESSIVE

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Drop dependency arrangement for tests/tmp
Ian Jackson [Thu, 26 Jul 2018 08:56:37 +0000 (09:56 +0100)]
test suite: Drop dependency arrangement for tests/tmp

run-all makes the directory now adays so this is not needed.  It's
going to be harmful for --progressive, since it would make make want
to rerun things every time tests/tmp/ is updated.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: run-all: Minor refactoring etc.
Ian Jackson [Thu, 26 Jul 2018 05:35:31 +0000 (06:35 +0100)]
test suite: run-all: Minor refactoring etc.

* Makefile: Move the DGIT_TEST_RETRY_COUNT if/else into the rule.
  This removes a small amount of duplication.

* run-all: Document the option to pass explicitly list of scripts.

* run-all: Add an option parser, which currently supports no options.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Honour DGIT_SCHROOT_CHROOT to set the schroot to use for the sbuild tests.
Ian Jackson [Thu, 26 Jul 2018 04:00:51 +0000 (05:00 +0100)]
test suite: Honour DGIT_SCHROOT_CHROOT to set the schroot to use for the sbuild tests.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: make $need_split_build_invocation always true
Sean Whitton [Wed, 25 Jul 2018 13:05:28 +0000 (21:05 +0800)]
dgit: make $need_split_build_invocation always true

dgit will always build the source package.

One minor consequence is that dgit will usualliy generate
*_multi.changes rather than *_$arch.changes, so we need to update the
one test that makes the contrary assumption.

Bump the dgit major version number as this is quite a significant
change in implementation and also a behavioural change.

This change makes a lot of code dead.  Removing that is left as a task
for the future.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Move WANTSRC_* definitions up the file
Ian Jackson [Wed, 25 Jul 2018 14:11:04 +0000 (15:11 +0100)]
dgit: Move WANTSRC_* definitions up the file

We are going to want these a bit earlier.  NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Disregard *_multi.changes for .changes ambiguity purposes
Ian Jackson [Thu, 26 Jul 2018 02:15:24 +0000 (03:15 +0100)]
dgit: Disregard *_multi.changes for .changes ambiguity purposes

The changes file ambiguity problem arises because dgit does not know
what architecture changes file the build is going to generate.  (To
know that it would have to delve even more into the command line
options the user is passing through dgit to the builder.)

dgit --always-split-build generally makes a _multi.changes file,
because it merges source changes with binaries from the build.  We are
going to make --always-split-build the only way things are done.  This
would result in lots more situations where --rm-old-changes is needed.

However, actually, we can assume that the builder does not generate a
*_multi.changes.  That will allow us to spot the builder-generated
changes file even if there is already a dgit-generated *_multi.changes
file from a previous build.

So: disregard _multi.changes, both when pre-checking for confusing
files, and when actually figuring out what the build produced.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: push-source dies if user tried to include uncommitted changes
Sean Whitton [Wed, 25 Jul 2018 11:34:56 +0000 (19:34 +0800)]
dgit: push-source dies if user tried to include uncommitted changes

This does not make sense because you cannot dgit push uncommitted
changes.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
5 years agodgit: don't let the user pass --include-dirty in split brain mode
Sean Whitton [Wed, 25 Jul 2018 13:04:00 +0000 (21:04 +0800)]
dgit: don't let the user pass --include-dirty in split brain mode

It is not supported.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Rename --ignore-dirty to --include-dirty
Ian Jackson [Thu, 26 Jul 2018 03:22:42 +0000 (04:22 +0100)]
dgit: Rename --ignore-dirty to --include-dirty

Leaving the old name supported as a deprecated alias.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: linkorigs: Fix error handling when bpd missing
Ian Jackson [Thu, 26 Jul 2018 03:11:10 +0000 (04:11 +0100)]
dgit: linkorigs: Fix error handling when bpd missing

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: bpd_abs(): Fix return value when bpd is absolute.
Ian Jackson [Thu, 26 Jul 2018 03:10:47 +0000 (04:10 +0100)]
dgit: bpd_abs(): Fix return value when bpd is absolute.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: massage_dbp_args: Make names for $wantsrc values
Ian Jackson [Wed, 25 Jul 2018 13:59:27 +0000 (14:59 +0100)]
dgit: massage_dbp_args: Make names for $wantsrc values

   0 - source will NOT need to be built separately by caller
  +1 - source will need to be built separately by caller
  +2 - source will need to be built separately by caller AND

So:

   0 => WANTSRC_BUILDER
  +1 => WANTSRC_SOURCE | WANTSRC_BUILDER
  +2 => WANTSRC_SOURCE

No functional change except to debugging output.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: Document some recent changes
Ian Jackson [Wed, 25 Jul 2018 09:46:44 +0000 (10:46 +0100)]
changelog: Document some recent changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: rename $ignoredirty -> $includedirty
Sean Whitton [Wed, 25 Jul 2018 09:17:08 +0000 (17:17 +0800)]
dgit: rename $ignoredirty -> $includedirty

This better reflects the effects that the option has.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
5 years agodgit: rename two quilt_fixup_ functions
Sean Whitton [Thu, 28 Jun 2018 10:25:56 +0000 (11:25 +0100)]
dgit: rename two quilt_fixup_ functions

They are useful for doing things other than quilt fixup.

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
5 years agodgit: actually respect $buildproductsdir in many places
Sean Whitton [Wed, 25 Jul 2018 09:02:16 +0000 (10:02 +0100)]
dgit: actually respect $buildproductsdir in many places

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Provide $bpd_glob
Ian Jackson [Wed, 25 Jul 2018 09:01:28 +0000 (10:01 +0100)]
dgit: Provide $bpd_glob

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: add dgit.default.build-products-dir git configuration key
Sean Whitton [Mon, 23 Jul 2018 04:10:02 +0000 (12:10 +0800)]
dgit: add dgit.default.build-products-dir git configuration key

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agomessages: Be a lot more friendly about NEW in particular
Ian Jackson [Wed, 25 Jul 2018 06:23:18 +0000 (07:23 +0100)]
messages: Be a lot more friendly about NEW in particular

And also add a couple of `please's.  Closes:#904448.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoapt-get method: when apt does not update release files, print hint
Ian Jackson [Mon, 23 Jul 2018 10:00:09 +0000 (11:00 +0100)]
apt-get method: when apt does not update release files, print hint

Unconditionally print hint about noatime.  This problem seems to occur
quite rarely (noatime is out of fashion nowadays, and I haven't seen
any other causes) so it doesn't seem worth fiddling about writing code
to determine the mount options.  Instead, print a hint and let the
user investigate.

Closes:#851873.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: unset VISUAL, which interferes.
Ian Jackson [Mon, 23 Jul 2018 09:38:30 +0000 (10:38 +0100)]
test suite: unset VISUAL, which interferes.

Closes:#904308.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: start 5.11
Ian Jackson [Sat, 14 Jul 2018 17:13:49 +0000 (18:13 +0100)]
changelog: start 5.11

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.10 archive/debian/5.10 debian/5.10
Ian Jackson [Sat, 14 Jul 2018 17:07:16 +0000 (18:07 +0100)]
changelog: finalise 5.10

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoinfrastructure: Run git gc --auto before mirroring. Closes:#841414.
Ian Jackson [Sat, 14 Jul 2018 16:23:37 +0000 (17:23 +0100)]
infrastructure: Run git gc --auto before mirroring.  Closes:#841414.

This would not have avoided the trouble reported here
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=841414#74

Empirically, I managed to work around the problem by running `git gc'
(rather than `git gc --auto') and then rsyncing the result to the
mirror.  I will request that DSA increase the RAM (or swap) size on
the relevant host.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Drop a couple of useless test log output lines.
Ian Jackson [Sat, 14 Jul 2018 15:58:04 +0000 (16:58 +0100)]
test suite: Drop a couple of useless test log output lines.

We don't need these (((((((((( )))))))))) now that it's just one line
inside them.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: rework for merging experimental tests into master
Ian Jackson [Sat, 14 Jul 2018 15:57:17 +0000 (16:57 +0100)]
changelog: rework for merging experimental tests into master

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.9+exp4 archive/debian/5.9+exp4 debian/5.9+exp4
Ian Jackson [Fri, 13 Jul 2018 11:10:37 +0000 (12:10 +0100)]
changelog: finalise 5.9+exp4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: tar up the artifacts, and name the tarball after the test
Ian Jackson [Fri, 13 Jul 2018 10:38:36 +0000 (11:38 +0100)]
test suite: tar up the artifacts, and name the tarball after the test

Naming the output after the test works around #903696.

tarring the whole lot up turns out to make it a lot smaller.
Before:
    $ du -sk ; du -sk --apparent-size
    1640    .
    853     .
After:
    236     .
    235     .

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: chmod a+r $AUTOPKGTEST explicitly. archive/debian/5.9+exp3 debian/5.9+exp3
Ian Jackson [Thu, 12 Jul 2018 23:54:27 +0000 (00:54 +0100)]
test suite: chmod a+r $AUTOPKGTEST explicitly.

5 years agotest suite: Do not save non-world-readable artifacts archive/debian/5.9+exp2 debian/5.9+exp2
Ian Jackson [Thu, 12 Jul 2018 23:42:26 +0000 (00:42 +0100)]
test suite: Do not save non-world-readable artifacts

Save things to $AUTOPKGTEST_ARTIFACTS without preserving access
control information.

Works around #903680.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.9+exp1 archive/debian/5.9+exp1 debian/5.9+exp1
Ian Jackson [Thu, 12 Jul 2018 12:45:17 +0000 (13:45 +0100)]
changelog: finalise 5.9+exp1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Save much stuff to $AUTOPKGTEST_ARTIFACTS.
Ian Jackson [Thu, 12 Jul 2018 11:57:32 +0000 (12:57 +0100)]
test suite: Save much stuff to $AUTOPKGTEST_ARTIFACTS.

We take some care to make this of reasonable size.  In particular,
we delete raw tarballs (typically, IMPORT) and .git's sample hooks.

And, we git gc the git trees.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Introduce playtree-save-refs
Ian Jackson [Thu, 12 Jul 2018 12:32:11 +0000 (13:32 +0100)]
test suite: Introduce playtree-save-refs

Arranges for all the refs in each playtree to be saved in the parent's
ref namespace.

This is going to be used when running git-gc when saving artifacts.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Drop t-ok from setup/*
Ian Jackson [Thu, 12 Jul 2018 12:27:19 +0000 (13:27 +0100)]
test suite: Drop t-ok from setup/*

t-ok is going to save artifacts, which we don't want these to do.

It is not actually needed.  All it does that is useful is the fsck,
which the actual tests will do.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Use dch -r -D sid '' not dch -r sid.
Ian Jackson [Thu, 12 Jul 2018 11:48:32 +0000 (12:48 +0100)]
test suite: Use  dch -r -D sid ''  not  dch -r sid.

Bizarrely,
   dch -r [-D suite]
runs $EDITOR but
   dch -r [-D suite] weevils
ignores `weevils' but does not run $EDITOR.  This is #903598.

Apparently, this is well known, and convention is to pass ''.
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=903598#8.

So use that.

Closes:#903441.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Factor out t-dch-commit-r
Ian Jackson [Thu, 12 Jul 2018 11:37:45 +0000 (12:37 +0100)]
test suite: Factor out t-dch-commit-r

The pattern `dch -r sid' is actually wrong and we will want to change
it.  See #903441 and #903598.  For future-proofing, factor this out
before we fix it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: start 5.9+exp1
Ian Jackson [Thu, 12 Jul 2018 12:43:07 +0000 (13:43 +0100)]
changelog: start 5.9+exp1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.9 archive/debian/5.9 debian/5.9
Ian Jackson [Thu, 12 Jul 2018 12:37:29 +0000 (13:37 +0100)]
changelog: finalise 5.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase(5): Suggest `gitk -- :.' too
Ian Jackson [Mon, 9 Jul 2018 00:21:11 +0000 (01:21 +0100)]
git-debrebase(5): Suggest `gitk -- :.' too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase(5): Document best gitk options. Closes:#901927.
Ian Jackson [Sun, 8 Jul 2018 23:53:54 +0000 (00:53 +0100)]
git-debrebase(5): Document best gitk options.  Closes:#901927.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase(5): Add new subsection DISCUSSION
Ian Jackson [Sun, 8 Jul 2018 23:51:19 +0000 (00:51 +0100)]
git-debrebase(5): Add new subsection DISCUSSION

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoNOTES.dgit-downstream-dsc.7.pod: WIP for #810829
Ian Jackson [Sun, 8 Jul 2018 23:47:02 +0000 (00:47 +0100)]
NOTES.dgit-downstream-dsc.7.pod: WIP for #810829

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit-downstream-dsc(7): New manpage.
Ian Jackson [Sun, 8 Jul 2018 23:44:09 +0000 (00:44 +0100)]
dgit-downstream-dsc(7): New manpage.

Closes:#842643,#851194.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: Document reorganisations
Ian Jackson [Sun, 8 Jul 2018 23:30:23 +0000 (00:30 +0100)]
changelog: Document reorganisations

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: gdr-convert-gbp: drop --overwrite
Ian Jackson [Sun, 8 Jul 2018 17:37:56 +0000 (18:37 +0100)]
test suite: gdr-convert-gbp: drop --overwrite

With #903132 properly fixed, this is no longer needed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase convert-from-gbp: Add ffq check
Ian Jackson [Sun, 8 Jul 2018 19:05:26 +0000 (20:05 +0100)]
git-debrebase convert-from-gbp: Add ffq check

Check that the result will not count as having diverged.  This will
usually turn failures to make the ff pseudomerge into -fdiverged.

Related to #903132.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase convert-from-gbp: Make it ff of dgit view, if we can.
Ian Jackson [Sun, 8 Jul 2018 17:37:34 +0000 (18:37 +0100)]
git-debrebase convert-from-gbp: Make it ff of dgit view, if we can.

Look for dgit-generated tags so we can usually make the new branch ff
of the dgit view.

Specifically, we fish the most recent released version out of
debian/changelog and look for a DISTRO/VSN tag and a corresponding
archive/DISTRO/VSN tag.  If this all looks good, we declare ff of
archive/.

This should avoid the need for dgit push --overwrite in most cases.

We do not check the DISTRO name.  In principle dgit could find the
distro name from the suite name, but that makes the software layering
quite complicated.  (For example, we would have to provide a way to
tell gdr what program to run for dgit, for the benefit of the test
suite.)  I think the current approach is good enough; there are not
likely to be any tag pairs that look like the right kind of thing but
are in fact something else entirely, so we're not likely to
accidentally trash some wanted changes.

Closes:#903132.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: Set $|
Ian Jackson [Sun, 8 Jul 2018 22:57:01 +0000 (23:57 +0100)]
git-debrebase: Set $|

This will avoid pepeering everything with calls to flush.  Without it
some messages from ffq_check, say, can arrive after the stderr output
from a subsequent snags_maybe_bail, which is quite confusing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: Provide snags_maybe_bail_early
Ian Jackson [Sun, 8 Jul 2018 18:43:06 +0000 (19:43 +0100)]
git-debrebase: Provide snags_maybe_bail_early

No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: Replace `die badusage' with `badusage'.
Ian Jackson [Sun, 8 Jul 2018 18:42:08 +0000 (19:42 +0100)]
git-debrebase: Replace `die badusage' with `badusage'.

badusage contains a call to finish.  So the die is redundant and never
gets called.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: ffq_check: Move defaults for $ff and $notff into sub
Ian Jackson [Sun, 8 Jul 2018 18:40:35 +0000 (19:40 +0100)]
git-debrebase: ffq_check: Move defaults for $ff and $notff into sub

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoDgit.pm: Move debinatag_maintview from dgit
Ian Jackson [Sun, 8 Jul 2018 17:32:09 +0000 (18:32 +0100)]
Dgit.pm: Move debinatag_maintview from dgit

git-debrebase is about to want this for convert-from-gbp dgit split
brain fixup.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoDgit.pm: Break out parsechangelog_loop from dgit
Ian Jackson [Sun, 8 Jul 2018 19:01:50 +0000 (20:01 +0100)]
Dgit.pm: Break out parsechangelog_loop from dgit

git-debrebase is going to want this.

No functional change except to some debugging messages.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agoDgit.pm: Move parsecontrol etc. out of dgit
Ian Jackson [Sun, 8 Jul 2018 18:51:39 +0000 (19:51 +0100)]
Dgit.pm: Move parsecontrol etc. out of dgit

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Fix a typo in a comment
Ian Jackson [Sun, 8 Jul 2018 16:54:42 +0000 (17:54 +0100)]
dgit: Fix a typo in a comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: Break out ff_check (for reuse by convert-from-gbp)
Ian Jackson [Sat, 7 Jul 2018 21:44:09 +0000 (22:44 +0100)]
git-debrebase: Break out ff_check (for reuse by convert-from-gbp)

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: record_ffq_prev_deferred: Move get_head();
Ian Jackson [Sat, 7 Jul 2018 21:27:42 +0000 (22:27 +0100)]
git-debrebase: record_ffq_prev_deferred: Move get_head();

No functional change.  This will make ff check easier to split out.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: start 5.9
Ian Jackson [Sun, 8 Jul 2018 10:44:27 +0000 (11:44 +0100)]
changelog: start 5.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.8 archive/debian/5.8 debian/5.8
Ian Jackson [Sun, 8 Jul 2018 10:42:53 +0000 (11:42 +0100)]
changelog: finalise 5.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: Improve grammar if one blocking snag.
Ian Jackson [Fri, 6 Jul 2018 23:50:17 +0000 (00:50 +0100)]
git-debrebase: Improve grammar if one blocking snag.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: gdr-newupstream: Test adding revision
Ian Jackson [Fri, 6 Jul 2018 23:31:09 +0000 (00:31 +0100)]
test suite: gdr-newupstream: Test adding revision

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase new-upstream: Add a -1 revision if the user didn't supply one.
Ian Jackson [Fri, 6 Jul 2018 23:41:56 +0000 (00:41 +0100)]
git-debrebase new-upstream: Add a -1 revision if the user didn't supply one.

Closes:#903127.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: status: Reorganise commit reporting.
Ian Jackson [Fri, 6 Jul 2018 23:15:46 +0000 (00:15 +0100)]
git-debrebase: status: Reorganise commit reporting.

Replace the references to @_ with a hash.  Using a hash is much less
confusing than all this numerical indexing.  Also, \@_ is wrong
because there is only one @_, so by the time it is dereference, the
commit being printed is the anchor.

This is why in #903131 we see

      branch is unlaundered
      found mixed upstream/packaging commit (7badba627162337c7057002f37e9a1a593d07d08)
        abcc6260 Update to upstream 2.9.7

which appears to sugggest that the anchor is a mixed commit.  Actually
the mixed commit is 7badba627162 but the subsequent message is based
on @_ which has different information in it by then.

And, set $kcmsg from the main message, not including the commit
information.  We are the more sophisticated caller mentioned in the
previous commmit.

Closes: #903131.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agogit-debrebase: keycommits: Callbacks get separate $mainwhy
Ian Jackson [Fri, 6 Jul 2018 23:13:02 +0000 (00:13 +0100)]
git-debrebase: keycommits: Callbacks get separate $mainwhy

Separate out the extra contextual info, naming the hash of the commit
being complained about, into a separate argument to $x->().  This
allows us to pass the message without that as an additional argument
to the callbacks.

This way we can still pass \&snag as a callback but other callers can
do something more sophisticated involving $cl.

No functional change for any existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: makepatches7: Add a nasty .gitignore
Ian Jackson [Fri, 6 Jul 2018 20:14:05 +0000 (21:14 +0100)]
test suite: makepatches7: Add a nasty .gitignore

Prior to the fix for #903130 this prevents both dgit and gdr from
making patches properly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit, git-debrebase: Properly make patches with nasty .gitignores.
Ian Jackson [Fri, 6 Jul 2018 20:17:18 +0000 (21:17 +0100)]
dgit, git-debrebase: Properly make patches with nasty .gitignores.

Specifically:

 * When running git status --porcelain, pass --ignored so it
   lists ignored files, and handle the ! output that it then
   produces.

 * Add missing -f to a few git add invocations.

I have done some greps ('git.*add', 'git.*status', 'porcelain', etc.)
to try to find other missed cases and none turned up.

git diff is OK because normally we pass two treeish arguments, in
which case the ignores are ignored by git diff.  When we are asking it
to look at the working tree, we are expecting it to ignore untracked
files (whether ignored or not), and diff tracked ones, which is what
it does.

Closes:#903130.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: Document dgit(1) build unscrambling
Ian Jackson [Fri, 6 Jul 2018 17:44:18 +0000 (18:44 +0100)]
changelog: Document dgit(1) build unscrambling

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
squash! changelog: Document manpage fix

5 years agodgit(1): Unscramble push and push-source descriptions
Sean Whitton [Fri, 6 Jul 2018 10:59:59 +0000 (11:59 +0100)]
dgit(1): Unscramble push and push-source descriptions

These got mixed up in a recent commit.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
5 years agochangelog: start 5.8
Ian Jackson [Thu, 5 Jul 2018 14:04:21 +0000 (15:04 +0100)]
changelog: start 5.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.7 archive/debian/5.7 debian/5.7
Ian Jackson [Thu, 5 Jul 2018 14:02:31 +0000 (15:02 +0100)]
changelog: finalise 5.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: sourceonlypolicy: New test
Ian Jackson [Thu, 5 Jul 2018 00:27:09 +0000 (01:27 +0100)]
test suite: sourceonlypolicy: New test

Test the fix for #801435.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Check that entirely-new uploads to Debian are not source-only-uploads.
Ian Jackson [Sun, 1 Jul 2018 10:08:21 +0000 (11:08 +0100)]
dgit: Check that entirely-new uploads to Debian are not source-only-uploads.

Avoids the user pushing things that will inevitably be REJECTed.

We end up introducing some new config and force options to support
this.

We do not reuse test_source_only_changes, as it doesn't have quite the
right shape (in particular, it sometimes blithers on stdout).  Also
arguably it is better to look, specifically, for .debs, for the
purpose of this test.

This requires a new archive protocol method.  We implement it for
ftpmasterapi (where it is actualliy needed) and dummycatapi (where we
need it for tests.)  Implementing it for madisonish methods would be
easy.  For aptget, it would probably involve iterating over suites,
and not be adviseable.

Closes:#801435.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Test dput-ng compatibility.
Ian Jackson [Thu, 5 Jul 2018 00:08:03 +0000 (01:08 +0100)]
test suite: Test dput-ng compatibility.

Add dput-ng as a dependency to trustingpolicy-replay.

This is the test which went wrong with dput-ng before.  It also has a
number of pushes, so it will do.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Fix trustingpolicy-replay & dput-ng.
Ian Jackson [Thu, 5 Jul 2018 00:06:26 +0000 (01:06 +0100)]
test suite: Fix trustingpolicy-replay & dput-ng.

Specifically:

 * Only dput-ng actually complains about the existence of .upload
   files.  dput itself just says there is nothing to do and calls that
   success.

 * The call to t-rm-dput-dropping was in the wrong place.  t-commit
   bumps the version to 1.1 but it's 1.0 that needs removing.

Closes:#903007.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: checkout: Check we end up on the right branch each time
Ian Jackson [Wed, 4 Jul 2018 23:56:36 +0000 (00:56 +0100)]
test suite: checkout: Check we end up on the right branch each time

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: checkout: Do not leave detached HEAD!
Ian Jackson [Wed, 4 Jul 2018 23:56:04 +0000 (00:56 +0100)]
dgit: checkout: Do not leave detached HEAD!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: Always pass LC_COLLATE=C to sort(1).
Ian Jackson [Wed, 4 Jul 2018 23:15:44 +0000 (00:15 +0100)]
test suite: Always pass LC_COLLATE=C to sort(1).

Specifically, by replacing every invocation of sort with the new
function t-sort.  The one case where we already took care of this, we
change LC_ALL to LC_COLLATE, which should be sufficient.

Closes:#903006.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit(7): Mention git-debrebase and gbp pq alongside git-dpm
Ian Jackson [Wed, 4 Jul 2018 01:19:05 +0000 (02:19 +0100)]
dgit(7): Mention git-debrebase and gbp pq alongside git-dpm

in the comment about handling patch stacks.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agotest suite: checkout: new test
Ian Jackson [Wed, 4 Jul 2018 00:54:13 +0000 (01:54 +0100)]
test suite: checkout: new test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit(1): checkout: Document when this acceses the network
Ian Jackson [Wed, 4 Jul 2018 00:27:42 +0000 (01:27 +0100)]
dgit(1): checkout: Document when this acceses the network

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: checkout: New command.
Ian Jackson [Sun, 1 Jul 2018 11:41:40 +0000 (12:41 +0100)]
dgit: checkout: New command.

Closes:#878443.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: fetch: Introduce new dofetch()
Ian Jackson [Wed, 4 Jul 2018 00:21:43 +0000 (01:21 +0100)]
dgit: fetch: Introduce new dofetch()

This actually does all that is needed.  It replaces two now-identical
calling patterns.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: fetch: Reorganise cmd_fetch
Ian Jackson [Wed, 4 Jul 2018 00:18:25 +0000 (01:18 +0100)]
dgit: fetch: Reorganise cmd_fetch

This slightly odd construction is actually how the return value from
fork_for_multisuite should be handled.  It now looks much more like
the calling pattern in pull().

No functional change: if $multi_fetched is falseish, we call
fetch_one, and then, no matter what, we finish 0.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: fetch: Slightly refactor pull()
Ian Jackson [Wed, 4 Jul 2018 00:16:19 +0000 (01:16 +0100)]
dgit: fetch: Slightly refactor pull()

When for_for_multisuite returns '0', meaning we are a child and should
do one suite, use finish 0 rather than returning, after doing the
fetch.

This has no functional change because the one call site is at the end
of cmd_pull, after which we finish 0 anyway.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: fetch: Rename fetch_one from fetch (nfc)
Ian Jackson [Wed, 4 Jul 2018 00:12:52 +0000 (01:12 +0100)]
dgit: fetch: Rename fetch_one from fetch (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit(1): Document as-yet-unwritten `dgit checkout'
Ian Jackson [Sun, 1 Jul 2018 11:00:07 +0000 (12:00 +0100)]
dgit(1): Document as-yet-unwritten `dgit checkout'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: update-vcs-git: Honour --package properly.
Ian Jackson [Sun, 1 Jul 2018 11:20:50 +0000 (12:20 +0100)]
dgit: update-vcs-git: Honour --package properly.

Specifically, use package_from_d_control rather than open-coding
a substandard implementation.

The consequent abolition of $sourcep does mean that if the user
specifies the suite `.', and does not specify -p, we needlessly parse
debian/control twice.  This doesn't really matter.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agodgit: Break out package_from_d_control (nfc)
Ian Jackson [Sun, 1 Jul 2018 11:18:22 +0000 (12:18 +0100)]
dgit: Break out package_from_d_control (nfc)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: start 5.7
Ian Jackson [Sat, 30 Jun 2018 23:43:27 +0000 (00:43 +0100)]
changelog: start 5.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
5 years agochangelog: finalise 5.6 archive/debian/5.6 debian/5.6
Ian Jackson [Sat, 30 Jun 2018 23:41:46 +0000 (00:41 +0100)]
changelog: finalise 5.6

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>