dgit: Remove foolish uses of $b in unpack_playtree_linkorigs etc.

Evidently I had forgotten that $b is weird in Perl and should not by
used for things other than sorting.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: generate_commits_from_dsc: Move up $upstreamv

We will want this in a moment.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: generate_commits_from_dsc: Factor out $bpd_abs

And add a newline.

We will want this in a moment.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test --clean-git[-ff],always

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Document new --clean=git behaviours

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean: Provide -wg[f]a short aliases

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean: Provide new --git[-ff],always clean mode

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: cleaning: Improve handling of note about ignores

* Provide a new $ignmessage parameter to clean_tree_check_git and use
  that.  This makes it a bit easier to:

* Only print the additional message if the clean mode actually
  honoured ignores.  If it didn't then the problem is not `due
  to missing .gitignore entries' because they would not have helped.

* Change all callers other than clean_tree_check_git_wd to
  pass the empty string, so no other overall functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean: Do check the tree even when building in playtree

If we might be doing quilt fixup, then the quilt fixup's attempt to
merge back into master might fail if there are uncommitted
debian/patches.  So it is sometimes wrong to just not clean at all.

We don't want the behaviour to depend on the source package format,
and ideally not on the clean mode.

Also, the user may have forgotten to `git add', in which case they
will thank us for spotting their mistake.

In the original design table in
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=910705#56
says this for this case
/: disregarded
   would be fine to delete but
   ** better to trip ? **
** want opt to disregard **
but this was not implemented at the time.

So implement that now.

I think I have concluded that with -wg and -wgf the right way to turn
these new failures into successes is to have a way to have the clean
actually done.  If that is not wanted, one can say -wn instead.

This is particularly true given that ignoring the problem can produce
confusing failure, as seen in #914317.

So there will be a new --clean=git[-ff],always shortly.

Closes: #914317
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: exmaplegit: Do not leave the tree dirty

This test setup facility passed a dirty working tree to all of its
consumers.  That was not really intended or useful, and is about to
start causing trouble.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean mode: Refactor -wg parsing to prep for suffixes

This will make it convenient to add new suffix letter(s) to -wg.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean: Prepare cleanmode_re for suffixes on git and git-ff

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean: Change matcbing of git to regexps

We are going to have some ,... suffixes on this, so we need more
general matching.

No functional change with the current set of clean modes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix a formatting typo

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Allow uppercase (ascii) letters in multi-orig components

We didn't recognise files like
   firefox-esr_60.4.0esr.orig-l10n-sv-SE.tar.bz2
as orig files.  As a result, dgit would see all the things in the
corresponding subdirectory as having been added in Debian changes,
leading it making inaccurate complaints and bombing out.

See also #918438 against policy, where I (effectively) argue that this
is a design error in the `3.0 (quilt)' format.

Closes: #916926
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-policy-debian: Remove duplicated text from force message

$ovwhy is from SELECT deliberately ... WHERE ... OR deliberately = ?
where ? is a key in %deliberately which came from the command line
where it already contains --deliberately=.

Closes: #913676
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Check that the build-products-dir is accessible

In fetch/clone, and build.  Others are less likely to be an irritating
problem.

Closes: #913648.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clone: Create destination directory before using network

We are going to want to check the bpd (apropos of #913648), which
might be a relative path.  We should fail quickly in this case.

No significant functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-debrebase(7): handle DFSG-filtering for a new package

Closes: #915973
Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

dgit: Pass --no-source-only-changes to pbuilder and cowbuilder

This will break with older versions of pbuilder and cowbuilder,
unfortunately.  That breakage can be worked around by passing dgit one
or both of the new options:
  --pbuilder!:--no-source-only-changes
  --cowbuilder!:--no-source-only-changes

Closes: #904862.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Provide --program!:option

Apropos of #904862.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: pretend-pbuilder: Reject unknown options

The failure to do this was simply a bug.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix reference to -wdn/-wddn in a message.

It's not -wdu/-wddu.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix documentation of .quilt-mode config

In
  0457c2d771ef6fe6c0883344add9dd4da0c60917
  dgit: Honour new .clean-mode-newer access config option
this was accidentally deleted.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po/README: Fix reference to dgit-user_7.pot

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 8.2~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: fianlise 8.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-debrebase(7): Minor corrections to #903377 wording

Do not suggest trying without --overwrite first, since if you were
going to pass it anyway you may as well do so right away.

And mention that git-debrebase will normally prompt you to say
dgit --overwrite.

Closes: #903377
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-debrebase(7): account for case discussed in #903377

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

git-debrebase: Provide convert-from-unapplied aa alias for -gbp

Closes: #905433.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: Document scope of convert-from-gbp

In fact this works with any kind of patches-unapplied (or patchless)
package.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

git-debrebase: usage message: Mention only convert-from-*

There is convert-from-dgit-view which was not mentioned, and we're
going to make another one in a moment.  We don't want to list the
whole lot here.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: pairwise-pocheck: Better handling for missing tcl

Mention in the README, and add a specific error check.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

NOTES.podchecker: Document why I'm not using podchecker

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: manpages-format: pass --warnings

Suggested-by: Colin Watson <cjwatson@chiark.greenend.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Run everything with LC_CTYPE=C.UTF-8

Things are not likely to work otherwise.

(If someone wants to try to run dgit in another ctype then I think
things will mostly work...)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: Add -LUTF-8 to the config

Apparently this is not the default.

Suggested-by: Colin Watson <cjwatson@chiark.greenend.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: pairwise-pocheck: Force it to be run sometimes

 * when building a binary package
 * when preparing a translation preview

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: pairwise-pocheck: Provide make targets to invoke it explicitly

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: Fix gitignore to actually ignore right .po4a.LANG.cfg

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po4a: New pairwise-pocheck script

This currently just detects if a translator has dropped the B or I
or some such from POD constructs like B<...> and I<...>.

Suggested-by: Colin Watson <cjwatson@chiark.greenend.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 8.1~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 8.0

The not-entirely-compatible change means a bump of major version I
think.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

po/README: Give a pointer to salsa; remind the translator to commit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po/README: Mention -k10 threshold

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

po/list-documents: Set translation threshold to 10%

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

README.md: Add this document for the benefit of Salsa

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

i18n-commit - autogenerated

Makefile: Provide i18n-commit target

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: push-source-with-changes: Make it test -wdda

We have plenty of other places where we used -wddn or ,no-check.

Here we have an opportunity to test -wdda: create a ~ file (which
would be ignored), clean everything else, and check that -wdda fails
but the default (-wdd) succeds.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: push-source*: Drop --dpkg-buildpackage:-d

These have been here since these test was created.  I think this must
have been done by copying the example of some other tests - but those
other tests didn't use the `example' test package and actually needed
it.  `example' does not.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Demand that all clean modes are handled by clean_tree_check

We've dealt with all the cases now, so we can add the default.

Also, in the test suite.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean_tree: confess rather than die on unknown clean mode

That would be an internal error.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: -wn clean check is a no-op

Handle this explicitly in the clean_tree_check conditional and in the
test suite.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: -wg / -wgf clean check is a no-op

Document this, and handle it explicitly in the clean_tree_check
conditional and in the test suite.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Provide --clean=dpkg-source[-d],all-check aka -wda / -wdda

This seems logically necessary and could help debug a clean target.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: dgit: Test that -wd catches the uncommitted changes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: --clean=dpkg-source: Check for untracked unignored files

We would like to spot if the user forgot to `git add' a file.
This can be done by calling clean_tree_check_git, after rules
clean (if applicable).

We need to make this configurable.  We do so via the clean mode, with
a comma-separated checking control suffix (and short aliases) like we
did for --clean=git,ignores.

The default should be cautious, ie to do this check, but often the
user will want to disable it because the source package has a buggy
clean target or no or insufficient .gitignore.  Existing users should
probably get the new check until they choose otherwise (which we have
made easier for them with the .clean-mode-newer config option).

So we change the meanings of -wd and -wdd to include the new check,
and provide new build modes ...,no-check aka -wdn / -wddn to disable
it.

To implement this we introduce a new clean_tree_check_git_wd function
to do the actual work, particularly because both during cleaning and
cleanliness checking, we want to print some hints to the user if the
check fails.

We can't do the new check if we applied patches dirtily to run the
rules target, because it will trip over the result of patch
application.  This way of working is just too poor to support this new
check.

The test suite generally tests the default versions, not the no-check
versions.  We must teach the test to expect the new check.  This is
most easily done with a separate case for the check side of the -wd
clean modes.  And we need to support the no-check variant too,
because:

The push-source-with-changes test does in fact work with a built tree
and needs to test the no-check variant.

The gbp tests sometimes involve patch application.  Rather than trying
to predict which of them do (in which cases there would be no clean
check), we force them all to ,no-check.

The oldnewtagalt test can use -wgf.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib-build-modes: Honour $cleanmodes_dpkgsource_extra

Nothing sets this now so NFC.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: lib-build-modes: Reformat a couple of lines

This will make forthcoming change a less disruptive diff.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Document that rules clean sometimes not run

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Combine option parsing of -wd and -wdd

We are going to introduce more variants, so we need to refactor this.

No functionalk change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out clean_tre_check_git

This will make it easier to handle the other clean modes more clearly.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Combine descriptions of --clean=dpkg-source and ...-d

We are going to add another dimension to the matrix here, which would
result in a profusion of cases.  Instead, combine the two sections.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Provide new clean mode --clean=check,ignores

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: $cleanmode_re: Use extended syntax

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean_tree_check: Reorganise command construction

This will make it easier to make the command vary.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: clean_tree_check: Add a comment about modifed tracked files

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Use a regexp to match clean mode check

This will make it easier to introduce new variants.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Implement clean_tree_check for mode -wc

This part of the fix for #910705.  This makes -wc actually work with
build_source.

But per the discussion in #910705 we want to implement some checking
in dpkg-source[-d] mode too.

We need to teach the test suite about this notion of checking
cleanlines rather than cleaning.  For now we wildcard C*.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Move clean_tree_check

Code motion.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Rorganise dpkg-source[-d] clean implementation

This will make it easier to introduce new variants.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Abolish obsolete variable $clean_using_builder

Nothing ever sets this to a trueish value.  Abolish it.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Reorganise cleaning in build_prep

1. Replace the open-coded $includedirty with the equivalent
building_source_in_playtree(), because the latter is what we actually
care about here.

2. Reformat into if blocks.

3. Introuce and call clean_tree_check (currently a no-op).  Now,
writing clean_tree_check will be part of fixing #910705.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Honour new .clean-mode-newer access config option

This will allow us to extend the set of clean modes without causing
irresolvable compatibility problems for users.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Put (?: ) around $cleanmode_re and use qr{}.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Drop (?!\n) from a use of $cleanmode_re

The config machinery now defends us from newlines.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Reject all git config options containing newlines

If we find something like this our regexp-based checking approaches
are likely to fail and other strange may will go wrong.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Refuse to work if critical files have uncommitted changes

Notably, d/source/format and options, and the forbidden files.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Factor out debian/source/.. in check_not_dirty

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out @forbid in check_not_dirty

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changedir_git_toplevel: Special error for not in git tree

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Change to git toplevel dir before starting

Closes: #910724.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Move changedir_git_toplevel from git-debrebase

NFC other than hardcoded qw(git).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: rename_link_xf: Avoid copying if src is a link to dst

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: rename_link_xf: Always use cp

If we use mv on a symlink, it moves the link, not the file.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: rename_link_xf: Minor style fix

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Handle EXDEV when putting in place output source

Replace a rename call with rename_link_xf, and change the subsequent
error handling to use $@ iff appropriate.

Closes: #910730.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Add missing error check in single-debian-patch handling

We called rename here without checking the error at all.
Handle ENOENT in case dpkg-source makes no patch.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Handle EXDEV when saving origs

Replace two link calls with rename_link_xf, and change the subsequent
error handling to use $@ iff appropriate.

Part of the fix to #910730.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit.pm: Provide rename_link_xf

This will be used for cross-filesystem support.  No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Forbid source building with --include-dirty non-.. bpd

Right now, this does bizarre damage to ..
Fixing this is very hard without bpd support in dpkg-source.

Closes: #910725.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: quilt linearisation: Stop at debian/source/format changes

Traversing a change of source format will not end well.
Discovered this possibility while investigating #910687.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Replace mention of alioth by salsa

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix spelling errors etc. in messages

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-sponsorship(7): Fix spelling error

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(7): Fix spelling errors

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix spelling errors

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Avoid crash if branch_is_gdr finds an origin commit

Closes: #910687.
Reported-by: Mattia Rizzolo <mattia@debian.org>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>