changelog: finalise 5.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-viagit, gdr-newupstream: do not auto-import t-setup-gnupg.

We provide a new mechanism for individual tests to disable the
automatic t-setup-import of gnupg.

We need to add this to all the other setup/ that these import, too.
gdr-convert-gbp is particularly awkward because it wants gnupg if
$ifarchive isn't `:'.  There we arrange to suppress its automatic
inclusion, but add it again by hand if appropriate.

This fixes these two tests in chroots without gnupg.

Closes:#902559.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Better description of --overwrite.

In particular, be clear that --overwrite (without previous-version) is
quite a weak promise: that the version you are uploading contains
everything in your changelog.  It won't overwrite willy-nilly.

Somewhat apropos of discussion in #902534.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Fix bug ref to #865444 in previous changelog entry.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 5.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 5.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(7): Add discussion of quilt fixup error messages

And add cross-references to dgit(1) and the actual error.
Somewhat apropos of #842614.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Better message formatting when --overwrite may be needed

Closes:891031.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: --overwrite test: Be more tolerant of exact message

We're about to change this, apropos of #891031.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Mention in --overwrite that it is usually needed for first dgit push

This is part of #891031.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test ill-advised repeat dgit push after git push fails

This demonstrates #869146, which is about dgit repeatedly adding the
same .orig to the .changes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Do not introduce duplicate origs in .changes files

The .dsc and .changes matching code contained roughly
   $fieldbody =~ / \Q$item\E\n/m
but this is wrong when the $fieldbody matches $item in its last line,
and there is no trailing newline (as would be usual).  This code
is obviouslu confused because m affects only ^ $ which don't appear
in the regexp.  Change these to / \Q$item\E$/m which is what ought
to have been, and probably was, intended.

In one case we are trying to delete a line.  In that case we need to
remove the leading newline.  The field body always starts with a
newline so the use of \n rather than ^ is fine.

Closes:#869146.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test GIT_REFLOG_ACTION setting by git-debrebase

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase new-upstream: Provide better reflog entries

By setting GIT_REFLOG_ACTION.

Closes:#901925.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

ref updates: Break out git_reflog_action_msg

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

ref updates: Honour GIT_REFLOG_ACTION everywhere.

Specifically, every time we call git_update_ref_cmd.

Closes:#901935.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

ref updates: Introduce update_ref_cmd

Currently the comment about `message may be modified' is rather
misleading, since there is no code to do that - but it explains the
semantics of the new function.

We change the appropriate subset of the git update-ref call sites.
These are precisely the ones where we previously specified a message
with -m, except for the quilt cache (where we use the reflog message
as the cache lookup key).

The calls are updates of internal refs etc.

We end up using qw(git) rather than @git.
Otherwise no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 5.3~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 5.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: dgit/gdr interop: test make-patches exiting 7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib-gdr: t-some-changes: Honour $2 `dum'

This allows for less thorough but faster test cases.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: git-debrebase interop: Actually tolerate git-debrebase status 7

make-patches exiting with status 7 causes $?==7*256, not 7.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: git-debrebase interop: Add a missing debugcmd debugging print.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: warn if vcs-git seems out of date

Print a warning to stderr on `dgit fetch sid', if your vcs-git remote
url disagrees with what's in sid's .dsc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Provide `update-vcs-git' subcommand

This is for creating and adjusting the vcs-git remote url.  Useful for
transition from alioth to salsa.  Closes:#902006.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Suggest sbuild-debian-developer-setup(1)

This will only create the chroot for building packages for the
machine's architecture, because sbuild-debian-developer-setup(1) does
not have an --architecture option, so sbuild-createchroot(1) will be
needed too, hence the 'and'.

The nice thing about sbuild-debian-developer-setup(1) is that does
fiddly things that a non-expert might forget about, like adding the
current user to the sbuild group.  So it is still worth recommending
even though it does not have an --architecture option.

Closes:#895779.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: Test that --new --overwrite works properly.

Here --overwrite is not needed, so is a no-op.  I chose this test for
no particular reason, out of the results of git-grep -e --new.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Clarify that --overwrite does nothing if not needed.

Closes:#863578.  Related to #863576 too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: --overwrite now no longer crashes if there is no $archive_hash

Ie there is nothing to overwrite (eg, when used with --new).  Instead,
it is simply ignored, as it is ignored in situations where the push is
fast forward.

Closes:#863576.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Use nproc(1) rather than Sys::CPU.

This is more portable and does not depend on libsys-cpu-perl being
installed.

Closes:888496.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Non-noop git fetch is not available with --dry-run, so fail

When fetch or push wants git fetch (other than in a situation where it
happes to be a noop) but --dry-run was specified, we would loop with a
false coplaint about git fetch.  False, because we didn't actually run
git fetch so of course it didn't do anything.

Instead, fail with an explanation.  Closes:#871317.

This does not occur, and we do not fail, if git-fetch would be a
no-op.  We can tell when this is the case, because we use
git-ls-remote too.  This leads to one suggestion for a workaround for
the user, which is to do a not --dry-run fetch first.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit, Dgit.pm: Clarify error framing a bit

Add a newline and "error:" before most error messages.  Thius makes
the output clearer, by distinguishing informational output (which
might or might not relate to any error which is occurring) from things
which are definitely wrong.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Improve message for discrepant quilt fixup

When quilt fixup fails because of discrepancies, print a git diff rune
which will show them.  Closes:#865446.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quiltify_splitbrain: Pass $oldtiptree (o+d/p) too

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: test suite: quilt-singlepatch: Test that we can remove everything

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: commit_quilty_patch: Handle other file statuses too

In particular, R can occur if all Debian changes have been removed in
a package with single-debian-patch, in which case we would fail to
commit the patch queue removal.  I don't think dpkg-source will remove
files in other cases, so this is probably the only actually buggy
case.

I don't think the other letters are particularly likely but if they
occur they should be committed too.  I have deliberately excluded U
which should definitely not occur.

Closes:#893263.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit build: Better message when network is offline.

When build fails because the archive access fails, mention that this
was tried because --since-version was not specified.

Closes:#883340.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Mention under `dgit build' that it uses the network.

Part of the fix for #883340.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 5.2~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 5.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: document changes so far

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: suppress gbp pq export output

Except in case of error.  Closes:#901809.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move shell_cmd from dgit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Trigger on indirect dependencies

As discussed on debian-ci.  Bug report to request documentation for
this in the autopkgtest spec is TBD.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

control file: Fix a typo in the git-debrebase description

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: check that dotfiles work properly too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Test that old macros are updated

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Rename a sums file to the right name

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: gitattributes: Defuse working-tree-encoding

* Add -working-tree-encoding to $negate_harmful_gitattrs.

* Add new arrangements for updating an existing dgit-defuse-attrs
  macro definition:
    - is_gitattrs: say whether the macro is up to date
    - setup_gitattrs: update an existing out-of-date macro
    - minor message changes
    - document the new behaviour

One consequence is that we have had to breach the promise about how to
stop future dgit setup-new-tree runs messing with this: it is now
necessary to disapply the macro, rather than redefine it.

Closes:#901900.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: gitattributes: Minor refactoring to prep for adding new squash

* is_gitattrs_setup: Break out $gitattrs_ourmacro_re
* is_gitattrs_setup: Reorganise very slightly
* is_gitattrs_setup: Add a debug print
* setup_gitattrs: Introduce $new
* setup_gitattrs: Add a blank line

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Do not try to set working-tree-encoding to true

Part of the fix to #901900.  This suppresses the sort-of-spurious test
failure which occurs because the working-tree-encoding attribute is
discovered and then set to an inappropriate value.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Test working-tree-encoding suppression

Put a UTF-8 BOM in the middle of the test files, and some gitattribute
settings for working-tree-encoding.  `ISO-8859-1' produces file
corruption.  `UTF-16' produces an error since it doesn't specify BE or
LE, and the file lacks a BOM at the start.

Part of the fix to #901900.  This introduces the desired test failure.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: new-upstream: fix ff check handling of multi-piece upstreams

Multi-piece combine commits may have n or n+1 parents, depending on
whether the previous new-upstream invocation was snag-free.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase manpages: Fix typos and etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 5.1~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 5.0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Delete some obsolete NOTES

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: git-debrebase: Rename gdr-newupstream-v0 test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Rename new-upstream-v0 command to new-upstream

This UI is pretty good now, I think, for single-piece upstreams.

For multi-piece ones, which are quite tricky anyway, it is tolerable,
and we it doesn't steal much of the available syntax space.  So we
don't need the -v0 any more.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Refer to dgit-maint-debrebase

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Abbreviate some of the tutorial descriptions

This will avoid linewrap from `man dgit' in an 80-column terminal,
when we introduce a reference to dgit-maint-debrebase.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Merge branch 'gdr-manpages' into wip.rebase

dgit-maint-debrebase(7): Add cross-refs to git-debrebase(1) and (5)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: add dgit-maint-debrebase.7 to .gitignore

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: git-debrebase: Do some tests without dgit installed

We provide a new psuedo-dependency NO-DGIT which throws away the usual
default dependency list.

We abolish NO-DEFAULT which could not be useful because the tests
pretty much all depend on setup/gnupg which (currently, as a
workaround for the gnupg2 races) requires chiark-utils-bin for
with-lock-ex.

There are two tests which don't actually need any interaction with the
archive.  Mark them accordingly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Split into its own package

* Provide and use Debian::Dgit::GDR which does the same for @INC as
  Debian::Dgit:Infra, only for GDR.  It's not *entirely* similar as
  its doc comment talks about ExitStatus too.

* In Makefile, split off lots of GDR_* variables and the
  two targets install-gdr and installdirs-gdr
  We take no care that only the required manpages are built for
  each install target; instead, they all go in MANPAGES and `all'.

* Add a control file stanza.  git-debrebase takes over the dependenc
  on libfile-fnmatch-perl.  We Recommend dgit and gbp.  (Even though
  if you're working with neither the Debian archive nor `3.0 (quilt)',
  you need neither.)

* In rules, add specpkg_install_gdr, to run the Makefile machinery.

* In the test dependencies, GDR now means to ask for git-debrebase and
  also git-buildpackage (for make-patches).  libfile-fnmatch-perl is
  handled via the dependencies of git-debrebase.deb.

* Add a .gitignore for the staging area in debian/

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: git-debrebase: Use gdr-convert-gbp-noarchive

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-convert-gbp-noarchive: new setup kind

Introduce $ifnoarchive in gdr-convert-gbp, which allows us to run it
without any of the operations which touch the simulated archive.

This is faster.  Also in the future after we split the packages this
will make it possible to run this new setup without having dgit or the
archive management tools installed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Get perl -I for source tree right

We wanted the Dgit.pm in the source tree (ie, the test suite's).
Previously this would work by accident, because Dgit.pm would be
installed in /usr.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: t-setup-done: get $import right setup using t-chain-test

We need to honour DGIT_TEST_NAME, rather than just using $0, in that
case.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: divergence checking: Do not mind if a ref we are checking does not exist

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: git-debrebase: provide GDR "macro" for dependencies

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: enumerate-tests: Refactor dependencies, support NO-DEFAULT

This will allow more flexibility in a moment.

No users of NO-DEFAULT yet, and no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

debian/rules: Refactor dgit-infrastructure perl arrangements

We are going to want to do this for another package too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document gdr, and bump to 5.0~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

documentation: Discuss gdr make-patches vs dgit quilt-fixup, etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Use git-deborig style upstream commitish finding

As discussed in email with Sean.

We could use git-deborig --just-print --version= but its error
handling is rather troublesome.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Add some plans for future, in a comment

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-edits: Rely on dgit interop, and test that it DTRT

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: t-gdr-good: Introduce etypes, a sequence

Rather than having the next type recorded in netype, record it in
etypes.  This is more orthogonal, and will allow more subtle
specifications of what is expected.

No functional change for now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Interoperate with git-debrebase, automatically

* Recognise gdr branches, by the presence of corresponding
  ffq-prev or debrebase-last refs.

* Before trying to do our own multi-patch linear quilt fixup, on a gdr
  branch, see if gdr can do it.  If so that is faster and better.

* If our branch is an unstitched gdr branch, which would be ff from
  the archive if we were to stitch it, then do so.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Prepare for calling git-debrebase

Provide the --git-debrebase= option, and pass it in the test suite.
Nothing uses this yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: gdr_ffq_prev_branchinfo: move from git-debrebase

dgit is going to want this too, for interoperability.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: make-patched: Provide --quiet-would-amend

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: provide X checkletter

No user yet.  I wanted this for a test which actually isn't going to
work like that, but I thought I would commit it anyway.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: use xopts for N*

This stylistic change improves the orthogonality.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: split off XX

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: honour new $xopts

This allows the introduction of chaining checkletters which modify
subsequent attempts.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: document XX

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Change exit status for patch amendments, to 7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: make_patches; Move get_differs into cmd_...

We want to make the error handling dependent on options, etc.  Moving
the differs analysis into cmd_make_patches avoids having a complex
calling convention for make_patches.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Provide failmsg, previously buried in fail

Most of `fail' is error message construction.  This change allows
callers to do something other with the message, than pass it to die.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: run gdr status

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Provide status subcommand

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: keycommits: Now $fatal hook

This allows a caller to avoid having keycommits crash on
unrecognisable stuff.  This will be used in a moment.

No functional change, because the default implementation calls fail
the same way it did before.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: keycommits abolish use of last etc.

Previously, presumably, this little fragment was within an anonymous
sub.  Now it isn't any more, and we can simply return.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: keycommits: Pass $cl to callbacks

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: keycommits: Move up $cl (nfc)

We are going to want this within the scope of the anon sub $x.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Fix multi-orig handling

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Fix snagging

* Rename $snags_checked to $snags_summarised and make it a counter
  of the snags we have summarised in snags_maybe_bail.
* Introduce all_snags_summarised, which uses arithmetic to see if
  we have had new snags since the most recent snags_maybe_bail,
* Use this in run_deferred_updates rather than the existing
  approach; this means we can crash if there were new snags for
  which we should have bailed.
* Make snags_maybe_bail not produce a redundant identical summary.
* Initialise counters to 0 so arithmetic does not give undef warnings.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>