git-debrebase: Do not try split brain gdr make-patches

Print a helpful error message instead.

Closes: #906908.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: test suite: gdr-subcommands: test forget

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: forget-was-ever-debrebase: New subcommand

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-import-nostitch: Test -fseries-comments

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-*: snag on discarding comments in series

series files can contain #-comments.  The conversion process will
throw them away, so complain about that.

Closes: #907198.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Make example_1.1 contain a patch with Name and Topic

This triggers #907202 and #907206.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: classify: Do not misclassify patches in subdirs

The old get_differs algorithm would recurse in debian/patches.
The new one failed to do that.  Fix that.

Closes: #907206.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: trees_diff_walk: Support recursion

No-one specifies this yet, so NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Fix get_tree's memoisation wrt $recurse

$recurse needs to be part of the key to the memo hash!

Luckily, there is only one caller which calls with $recurse==1,
check_series_has_all_patches.  It ends up mishandles this situation by
complaining about the directory names, as if they were unused patches.

And after that no-one needs to call get_tree.

Closes: #907202.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: trees_diff_walk: Refactor $all to \%opts

No callers pass this so NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 6.10

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: fianlise 6.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Remove unsubstituted $changesfile from message

Closes:#906787.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document docs changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-debrebase(7): Add crossreference to "ILLEGAL OPERATIONS"

There are three items under "ILLEGAL OPERATIONS":

1. We already refer to "OTHER MERGES".

2. This commit adds a crossreference to the second item.

3. The information under "Editing debian/patches" is already basically
   included in dgit-maint-debrebase(7).

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit-maint-debrebase(7): Rewrite "Converting an existing package"

Closes:#905573

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: Update debian/tests/control following dependency fix.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: do not crash on push of a new gdr package.  Closes:#906784.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 6.9

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Fix dependencies of new gdr-merge-conflicts test.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 6.8

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Finalise 6.7

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Reorganise and tidy up the current entry

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fixup: Maybe mention gitattributes

These are a potential cause of quilt linearisation failure.

Closes:#906199.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1), dgit(7): Document that we do not suppress export attributes

Ie, ones which affect git-archive.  This is related to #906199.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fixup errors: Refactor and reword very slightly

Make this two calls to push so we can put things in between.
Also remove `to specify'; now the message fits in 80 columns.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fixup: Do not suggest quilt modes if we see gdr

If `stopped at' is a git-debrebase commit, then a non-default quilt
mode is probably not the answer.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fiuxp: linearisation failure: Add a blank line

The `stopped at' commit report can be long, if there are several, or
complicated, commits.  Add a blank line.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fixup: @failsuggestion: Include a machine-readable reason

Each entry is now an arrayref.  This will let quilt_fiuxp filter them
later.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(7): Mention bad origs as possible cause of quilt fixup failure

And document our error message changes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt fixup error messages: Change hint emphasis

* No longer hint --quilt=smash or dpkg source --commit.  These are
  rarely the right thing.

* Hint about bad origs, instead.  This is fairly common.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Remove a spurious space

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-unprocessable-hints: New test

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Provide hints for unprocessable commits

Depending on the apparent branch ffq state.  Closes:#905005.

In particular, if it doesn't look like this branch was ever used with
gdr, suggest that the user is on the wrong branch, or needs to use
convert-from-*.  Closes:#905279.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Introduce new `fail_unprocessable'

This is called when we want to call `fail' because we have found an
unprocessable commit.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-*: leave debrebase-last refs

To hint to everyone that this is now a gdr branch.  This will make
dgit push use gdr make-patches, for example.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: begin_convert_from: Break out $gdrlast_obj

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Snag on repeated conversions

If debrebase-last says we are ahead of some previous gdr work, it is
carrying on is unlikely to be helpful.  It would only be sensible if
the package is vacillating between branch formats, and this is
reflected and visible in the branch history.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Refuse to start conversion if ffq-prev exists

Trying to rerun git-debrebase convert-from-something, on an unstitched
gdr branch, would end in tears.  Forbid it utterly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: record_gdrlast: Honour $oldvalue

NFC for existing callers.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Move ffq_prev_branchinfo up the file

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Break out begin_convert_from & complete_...

$gdrlastinfo is undef for now, and will be used in a moment.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Break out record_gdrlast

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: get_tree: Check we got an object name

Otherwise the cache is dangerous.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Merge: Make a todo about gbp not round tripping

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Merges: Enable with command-line option instead of env var

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: New walk mechanism: Drop obsolete $opt_careful

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Merge: Put off some xxx's, making them MERGE-TODO

We are going to release with this machinery in this experimental
state.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Document merged-breakwater metadata tag

This is generated only by the experimental merge machinery and is
subject to possible future change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Merge: Remove some xxx's

git-debrebase(5) already has ILLEGAL OPERATIONS (which forbids many
git-rebase operations) and also of course LEGAL OPERATIONS.

The limitations on what kind of merges we can resolve and how will be
documented when we document the merge machinery properly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document changes to convert-from-gbp messages.

This is part of #906641.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-gbp: Check separately for $dtag

This allows us to print a more sensible error message, Previously
git would print a fatal error, after which we carry on.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-gbp: Add a missing newline to a die

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-gbp: handle missing maintainer tag

Look for / in $mtag, not in $mtag_pat.  Obviously it will be in the
latter.  The result, before, is that we would look for a completely
bogus $dtag and print daft error messages.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-import-nostitch: New test

Generate this situation and check it's OK.  Also check that the
messages contain the expected hint.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-gbp; on stitch failure, clarify message

Clarify what the preceding failure messages mean, and what the
consequences are.

This is part of #906641.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-convert-gbp: Break the start out into lib-gdr

This will allow us to test other things to do with this conversion
without making tests/setup/gdr-convert-gbp more complicated.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-gbp: Print a success message.

This is part of #906641.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Add close note for #905400 to changelog entry for 6.5.

The commit
  git-debrebase: Properly reject bare dgit dsc imports.
fixed #905400.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-convert-gbp: test snag on unused patches

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: convert-from-*: Snag on unused patches

Patches in d/patches which are not in series will be deleted, and not
represented in the output.

So call them a snag.

Closes:#904997.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: get_tree: Support $precheck and $recurse

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-subcommands: Test raw conversion from gbp

Make a branch which strips off the pseudomerge generated by gdr to
stitch the dgit history, and then run some tests on that.

These new tests detect #905975.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-subcommands: Use $peel, not $work, for a few checks

The definition of P says
    P        result is add-patches, remaining letters apply to result~

But this was not hnoured by N n Z.  Honour it.

No change with existing tests, which do not follow P with any of
those.  But, this will be important in a moment,

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gdr-convert-gbp: Save converted-from-gbp

This new branch will be useful in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: On rebase, always save ffq-prev

Even if we were both stitched and laundered.

Closes:#905975.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: `prepush' is now a silent no-op if unstitched.

This is more friendly.  Update the docs accordingly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Add perf numbers

This is from tests on my laptop with the branch the Debian kernel
folks shared with me, ie c6eb63c6ed0694b0d22fbe5aaff953d209a3fc4e.

                                     avoid        new
   runtimes in [s]         before    read-tree   get_differs

  git-debrebase             78.574    3.699       1.419
  git-debrebase status       3.359    unchanged   0.818

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Rewrite get_differs in terms of trees_diff_walk

As noted, git-diff-tree is quite slow.  We can do much better, by
stepping across the two trees ourselves.

There should be no particularly significant change: the answers should
be the same for reasonable inputs.  Unreasonable inputs (particularly,
executable files etc.) may be handled better, even.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: get_tree: Support '' for nonexistent objects

This is going to be convenient in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: New get_tree machinery: Fix debugging

When we localise, we get a new variable, rather than localising the
binding imported from Debian::Dgit.  How annoying.

Also, add two new debug prints at level 3.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Fix comment documenting D_DEB_*

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Actually avoid read-tree in walk when not rewriting

See
   git-debrebase: Prep for avoid read-tree in walk when not rewriting
for the motivation.

Here we try to check that the commit we would make, if we were
actually calling read-tree, write-tree and hash-object, would be
identical to the one we are reusing.

In particular, we check that the upstream files, and the debian/
directory, have the contents we would feed to read-tree.  And we check
that the parent list is the same.

With these checks this new algorithm is, I think, safe enough to just
use always.   So set $opt_careful to 0.  We will abolish this
variable in a moment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Provide new get_tree and trees_diff_walk

These are ways to avoid calling git-diff-tree, which is very slow even
with --name-only --no-renames and without recursion.  I think it is
doing all the work to make nice actual diffs, and then throwing them
away.

Two calls to git-ls-tree is about 20x faster than git-diff-tree
on the Linux kernel, for example.

No functional change in this commit, since no callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: unset GIT_EDITOR, so it works if user has that set.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Rename $want_tree_ to $want_

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Prep for avoid read-tree in walk when not rewriting

Because the git index is a flat list of files, not a directory
hierarchy, git-read-tree can be very slow.  We want to avoid it
if possible.

Introduce a new way of working, where we defer calls to read-tree.

In detail:
 * $read_tree_upstream and $read_tree_debian, which are the only ways
   that walk invokes read-tree, simply record their argument;
 * The actual read-tree is done just before the write-tree and
   commit generation.  read_tree_upstream, conveniently, does both
   halves, if it gets both arguments.
 * Put all of the read-tree and commit regeneration in a branch
   triggered if we are rewriting or "trying to be careful".

Right now we are always careful, but I have tested this with a hack
setting $opt_careful to 0, and the performance on the example branch
in #905995 is much improved: 3.4s rather than 77s for `git-debrebase'.

This is not suitable for enabling in its current form.  I want to
actuallly abolish the $opt_careful, and, instead, check with
get_differs or git-cat-file that the tree segments, and the parent
lists, are identical.

When we do that we won't be checking any more that the commit
generation can generate identical commits when not rewriting.  But it
doesn't really matter much any more provided the commits are
well-formed and right, and the test suite will check that.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Encourage --overwrite rather than --overwrite=version.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: walk: Introduce $read_tree_debian

We are going to make these do something more efficient, sometimes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: walk: Move $read_tree_upstream into the loop

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Drop a redundant call to $read_tree_upstream

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: conflict: Some xxx's now done

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: test conflict: Test resolution via patch

This is a really horrible way to do things.  We need a better
way to let the user help resolve this kind of merge.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Actually use merge resolutions

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Actually record merge resolutions

And, destroy it on scrap.

Nothing consumes this yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: scrap works properly when it does only rebase --abort

We use @deferred_updates to see if we actually did anything for
--noop-ok purposes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: test conflict: Break out no-wreckage

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Change wreckage to refs/debrebase/wreckage/*

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move reflog_cache_* from dgit

We are going to want this for git-debrebase merge resolutions.

NFC other than the usual change to literal `git'.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move $maindir* up the file.

We're going to want these for the reflog cache functions.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move make_commit_text from dgit

NFC other than the usual change to literal `git'.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: New record-resolved-merge subcommand, WIP

Right now this seems to be able to generate some commit, but
it doesn't know how to record it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Split off merge_series_patchqueue_convert

We're about to reuse this.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Record the new-base in the wreckage.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Use a fresh branch and check out again

We are going to want this in a moment.

No overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Introduce $merged_pq

We are going to want this in a moment.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Use a couple of globals instead

We are going to make some of this into its own sub, and copying the
convenience functions seems annoying, so do this.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Introduce merge_attempt_cmd global

NFC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: merge: Do wreckage cleaning separately

git-update-ref hates us if we delete a ref and reintroduce it, in the
same transaction.  In this case we don't mind doing it in two goes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: fresh_workarea: Support $subdir

And use this in merge_series.

NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>