test suite: gitattributes: work around git attribute updates bug

touch af/*, to work around #851713

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: More testing of clone: check sums

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Start to test clone

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Break out sums_check_broken

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Defuse gitattributes in private working area, always

Even if we don't do it in the user's tree because config
setup-gitattributes=false.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Improve comment left in .git/info/attributes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Check that warning does not appear when unexpected

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Break out t-dgit-warn-check

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Break out wd= setting

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Test that checkout is good too

test suite: gitattributes: Break out sums_check

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Rename $branch (from $refname)

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Move setting of sums

Fixes some of the working filenames.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Call t-git-none

This is preparatory to testing clone.  No significant effect right
now.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Break out origtar= setting

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Fix a ----- message

No significant change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Break out dscf= setting

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 3.5~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: worktrees: Replace references to /home/ian

Replace with references to /nonexistent, to catch inadvertant
accesses.

Now `git-grep /home/ian' produces only one hit - in a commit message,
where it's harmless.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite; drs-push-rejects: Set origin's url

Use an ad-hoc expression which produces the right ext:: rune, as dgit
would.

Closes:#851580.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 3.4~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.3

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: try to guess some attr names from manpage

This may allow the test suite to detect if git gains new attributes.
The worst case is that we add unknown attributes to our test package.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document gitattributes handling

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): gitattributes: Document dgit-distro.distro.setup-gitattributes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: Docs and message wordsmithing

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Check for expected warnings

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: Issue a warning on un-defused .gitattributes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Remove .git from orig

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: test unsetup tree, too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Test source format 1.0 with diff

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: Reorganise and improve, still wip

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: gitattributes: new test, still wip

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: dgit: setup-gitattributes: New action

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: Suppress transformations in our cloned trees

The user wants a checkout of the actual source tree, so that their
checkout and the source package are identical.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: Suppress transformations in ud

We want all of our own manipulations to be unaffected by gitattributes
transformations, so that the imported git *trees* are identical to the
source packages.  (Ie, if transformations are in effect, the
*checkout* of such a tree might not be identical to the source
package.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: gitattributes: Provide setup_gitattrs (internal function)

This is the idempotent function which adds the appropriate runes to
.git/info/attributes.  No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

gitattributes: dgit(7): Document our approach and rationale

This is not properly implemented yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: import_dsc: Break out import_dsc_result

No functional change other than slight change to messages.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Remove redundant use of List::Util qw(any).  Closes:#851280.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: protocol-compat: check that we use the right distro

* Set distro to '' when running the import.  This makes dgit fall back
  to the default distro.

* Set the default distro to `foreign' (which has no settings, so we
  use builtin defaults).  This only has any effect if the shell
  variable `distro' is set to the empty string - so it only takes
  effect during the import.

* Set dgit-distro.foreign.cmd-git to false.  This therefore detects if
  we ever try to do the import with the default distro's `git'
  command: we are supposed to be using the one from the .dsc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Option parsing: Fix undefined $suite in some import-dsc.

* Set $isuite to a dummy value.  In import-dsc we aren't operating on
  a suite.  We have to set it to something (because the config likes
  to look at the suite) and this avoids picking up information
  relating to the default suite.

* Set $idistro (if -d not supplied) to the distro from (or implied by)
  the .dsc.  This is needed so that we do not use any config from the
  default distro.  In particular, the default commands (like what
  `git' command to use to access the dgit git server) need to be found
  via the dsc's distro's cmd-git setting, and this is done by
  parseopts_late_defaults() in pushing().  So we must set $idistro
  before then.

* Move notpushing earlier, so that we call it before we call
  resolve_dsc_field_commit().  (It doesn't matter there for $idistro,
  because it sets that itself, but the git command does.)

Closes:#851213.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Internal change: fix handling of nonempty distro=

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: defdistro-import-dsc: Drop this test.

import-dsc *should* access the default distro for cmd-dgit et al.
(Many of the things in parseopts_late_defaults.)

Checking that we use the right distro config is going to be done
shortly by protocol-compat.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Set default dsc import distro when suppressing Dgit field.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Break out parse_dsc_field_def_dsc_distro as global sub

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Set default dsc import distro when there is no Dgit field.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: parse_dsc_field: Break out $def_dsc_distro

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: default dsc distro config setting: Move to lib.

We need this for the .dscs we have in tests/pkg-srcs/.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Remove a leftover debugging print.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Packaging: Remove redundant Recommends on libtext-iconv-perl.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 3.3~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.2

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: tidy up

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix a warning message about ref (mainly, tag) updates.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Fix a typo in a comment.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Fix some more typos

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

option parsing: Rework --dep14tag options, introducing `defoptmodes'

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-user(7): Fix some typos.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Mention #850953

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document dgit-maint-merge change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-maint-merge(7): Use git-deborig(1)

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>

test suite: mirror-private: test that package becomes public.  (#849789)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit-repos-policy-debian: Remirror a package when it becomes public

Ie, make the repo available much more promptly when the package passes
NEW.

Closes:#849789.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

infra: mirroring and policy hooks: Provide ((( ))) banners

When we invoke these, this makes their output easier to find in the
middle of the test suite debug scrool.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Do not execute END blocks in children.

END blocks run inside children created by fork or piped open (the
latter, if they run perl code), on exit or (I think maybe) die.  This
is far from appropriate.

So far symptoms of this bug seem to be limited to duplicated error
messages but I have not done a thorough analysis.  Closes:#850052.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Call forkcheck_setup in multisuite children

We want execute our SIGWARN handler in these explicitly created
children.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Dgit: forkcheck_*: break out from setup_sigwarn

Other places are going to want this too, notably END blocks.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 3.2~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Arrange not to overwrite .save-env on t-chain-test

We want .save-env to be the env on entry to the original test, not
on execution of t-chain-test with a modified environment.

Without this the effects of t-defdistro might leak into the execution
of tests/setup/examplegit.  This is no good because
tests/setup/examplegit relies in the -dtest-dummy which is our usual
dgit runes.  (Or to put it another way, examplegit uses many suites
which are not (yet) set up properly by t-defdistro.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: buile_prep_early: Call notpushing() after getting $isuite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: dsc download handling: save as ,fetch if need be

* Pass \$refetched to complete_file_from_dsc, so that we refetch files
  whose hashes are wrong.

* When saving files we refetched, save them as F,fetch if saving them
  as F gives EEXIST.  So we can save refetched files.

* When looking for files to reuse, look for F,fetch first.

We do this only here in generate_commits_from_dsc, not in
ensure_we_have_orig.  This is because, in practice, this all won't be
very effective for origs which need to be referred to by various
.dscs.

The main use case is when the user did a build without changing the
version number.  This would otherwise break fetch: Closes:#850824.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: complete_file_from_dsc: Introduce $refetched

Make have a option which, if set, causes hash mismatches to result in
a refetch rather than a crash.

No functional change because no call sites pass it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: complete_file_from_dsc: Reorganise $fetchhash

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: complete_file_from_dsc: Introduce $checkhash

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: generate_commits_from_dsc: Introduce $upper_f

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Fix entry referring twice to refs/dgit-fetch/DISTRO

Reported-by: Sean Whitton <spwhitton@spwhitton.name>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: After downloading .debian.* files, save them in `..', too

Ie do this not just for .origs.  We don't want the user needlessly
re-downloading them on each fetch.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: defdistro-import-dsc: New test

Add test for import-dsc with default distro.  (Detects #850781.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: import-dsc: Add missing call to notpushing().

Avoids crashing with undefined $isuite.  Closes:#850781.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Do not nedlessly re-fetch the rewrite map.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: start 3.1

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: finalise 3.0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: defdistro-mirror: Add missing dependency

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: defdistro-rpush: Remove workaround for missing isuite param

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: rpush: Use $isuite for $isuite, not $csuite

Send isuite as a parameter, and use it if it's provided, instead of
csuite.  This means config lookups like
    dgit-distro.SUITE.distro
only need the user-provided suite name (eg `unstable'), and don't need
to be duplicated for the canonical name (eg `sid').

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Test that default distro works

Ie check that dgit-distro.SUITE.distro works

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

test suite: Provide t-defdistro (for use with t-alt-test)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: $isuite handling: Defer calls to access_* until first `want'

pushing, rpush_handle_protovsn_bothends, and quiltmode, all depend on
access_*.  For that we need $isuite, but we only get that (in the form
of $csuite) from the build host.

We can move all of these calls into i_resp_want, which is called the
first time the build host actually wants something from us.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: $isuite fixes: defer pushing and rpush_handle_protovsn_bothends in rpush

We mustn't call pushing until we have parsed the arguments enough to
find $isuite.  So remove the call to pushing from
cmd_remote_push_build_host.  Now we rely on cmd_push's call to
pushing, which it does after the argument parsing.

We then need to move rpush_handle_protovsn_bothends (which implicitly
calls accesss_*) too, until we have got $isuite and called pushing().
We can move that into dopush.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: rpush: suppress changelog tag format check

The changelog might be received (and therefore parsed) before we have
been told the suite.  So we can't check the tag format because we
cannot know the distro yet.

Simply suppress this early check.  If the tag format is wrong, we will
discover this later.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: Option parsing: Move pushing call in push

Otherwise $isuite might not be set when we call access_something.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit: rpush: Provide $we_are_initiator

No functional change yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Tidy a bit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Mention #850519 re docs updates

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

changelog: Document dgit(1) changes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Fix --force-import-dsc-with-dgit-field

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

dgit(1): Clarify --no-chase-dsc-distro

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>