chiark / gitweb /
dgit.git
2 years agoworktree support: dgit: Fix for .git/info/attributes
Ian Jackson [Wed, 2 Aug 2017 18:22:28 +0000 (19:22 +0100)]
worktree support: dgit: Fix for .git/info/attributes

Experimentally, this file is in the common dir, not the per-worktree
dir.  Honour $maindir_gitcommon everywhere.

Rename `open_main_gitattrs'.  Previously it could in theory be used in
the playtree (but, it isn't).  Now it always operates on the main
tree.

Indeed, as we require, the `setup' stuff is only done in the main
tree.  The gitattributes fixes for playtrees are done,
unconditionally, with the much simpler hammer in playtree_setup.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoworktree support: Fix playtree_setup
Ian Jackson [Wed, 2 Aug 2017 18:19:38 +0000 (19:19 +0100)]
worktree support: Fix playtree_setup

In a worktree, .git is not a directory.  We need to use
$maindir_common.  Add a comment, too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory handling: dgit: Introduce dgit_privdir and use it
Ian Jackson [Wed, 2 Aug 2017 17:38:32 +0000 (18:38 +0100)]
directory handling: dgit: Introduce dgit_privdir and use it

This removes another pile of open-coded references to .git/

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: sbuild-gitish: Insist on using "build" chroot
Ian Jackson [Thu, 3 Aug 2017 19:34:11 +0000 (20:34 +0100)]
test suite: sbuild-gitish: Insist on using "build" chroot

In "test suite: sbuild-gitish: Find sbuild rune in the manpage" aka
f5c0d33ede8a we accidentally switched from "build" to "jessie".
Randomly messing about with the user's "jessie" chroot is not very
nice.  (And it would break in a hypothetical formal test providing the
x-dgit-schroot-build capability.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoworktree support: Fix ensure_a_playground (gives absolute path now)
Ian Jackson [Wed, 2 Aug 2017 18:17:14 +0000 (19:17 +0100)]
worktree support: Fix ensure_a_playground (gives absolute path now)

Make it honour $maindir_gitdir so it will DTRT in a `git worktree'.
The .git/dgit play area ends up in the .git/worktrees/<worktreename>
for this worktree.

It now returns an absolute path.  The only call site that needs
adjusting is the one in fresh_playground.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory handling: dgit: Use $maindir rather than ../../..
Ian Jackson [Wed, 2 Aug 2017 15:46:37 +0000 (16:46 +0100)]
directory handling: dgit: Use $maindir rather than ../../..

Replace all the open-coded ../.. constructs with references to
$maindir.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory refactoring: dgit clone: call record_maintree in multisuite too
Ian Jackson [Thu, 3 Aug 2017 07:31:57 +0000 (08:31 +0100)]
directory refactoring: dgit clone: call record_maintree in multisuite too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory refactoring: dgit clone: call record_maintree
Ian Jackson [Wed, 2 Aug 2017 18:25:34 +0000 (19:25 +0100)]
directory refactoring: dgit clone: call record_maintree

We are making up our own tree, here.  We will need to call
record_maintree in case we need to use information which it obtains.
Currently, we don't.

But git worktrees are complicated: we are going to have to depend on
the new $maindir_git* variables, in setup_new_tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory refactoring: most invocations: call record_maintree
Ian Jackson [Wed, 2 Aug 2017 18:59:53 +0000 (19:59 +0100)]
directory refactoring: most invocations: call record_maintree

We need to call record_maintree so that $maindir and its friends are
set, since we're going to rely on them heavily.  Most operations will
need them.

Any operation that is invoked in a git tree should definitely have
this called.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodirectory refactoring: Rename no_local_git_cfg
Ian Jackson [Wed, 2 Aug 2017 18:56:40 +0000 (19:56 +0100)]
directory refactoring: Rename no_local_git_cfg

This is actually called when we don't know that we are starting in a
git tree.  We want to use this knowledge to control an automatic call
to record_maindir, too.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoworktree support: Dgit.pm: Introduce $maindir_gitdir and _commondir
Ian Jackson [Wed, 2 Aug 2017 18:14:40 +0000 (19:14 +0100)]
worktree support: Dgit.pm: Introduce $maindir_gitdir and _commondir

A "git worktree" separates out some of the things which used to be
found in .git, into "common" things and "gitdir" things.

In this patch we simply collect the relevant informaation.  No-one
uses it yet so there is no significant functional change.

However, while we are here, we do improve an error message slightly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: Dgit.pm: Provide ensure_a_playground
Ian Jackson [Wed, 2 Aug 2017 17:34:02 +0000 (18:34 +0100)]
playground refactoring: Dgit.pm: Provide ensure_a_playground

dgit wants to make a lot of temporary things in .git/dgit.
That's like a playground, but dgit doesn't want it wiped.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: Dgit.pm: Honour $maindir in fresh_playground
Ian Jackson [Wed, 2 Aug 2017 14:52:41 +0000 (15:52 +0100)]
playground refactoring: Dgit.pm: Honour $maindir in fresh_playground

Previously, fresh_playground expected that the playground argument has
exactly the depth .git/FOO/BAR.

Now, it expects that $maindir is set.  In dgit (the only caller right
now) this is always true because dgit now uses fresh_playground.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: dgit: Use playground facilities
Ian Jackson [Wed, 2 Aug 2017 11:28:09 +0000 (12:28 +0100)]
playground refactoring: dgit: Use playground facilities

* Replace implementation of prep_ud with a call to fresh_playground
* Replace references to $ud with $playground
* Replace hardcoded changedir ../../.. etc. with $maindir

The overall functional change is fairly minimal.  We now call getcwd
more often, and when we want to return to the maindir we changedir to
an absolute path instead of a relative one.

Later, we may rename prep_ud, mktree_in_ud_here, etc.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: Dgit.pm: Provide fresh_playground etc.
Ian Jackson [Wed, 2 Aug 2017 14:50:03 +0000 (15:50 +0100)]
playground refactoring: Dgit.pm: Provide fresh_playground etc.

These functions will shortly replace the similar, but less-flexible,
ad-hoc functions in dgit proper.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: Dgit.pm: Expose $local_git_cfg
Ian Jackson [Wed, 2 Aug 2017 14:09:12 +0000 (15:09 +0100)]
playground refactoring: Dgit.pm: Expose $local_git_cfg

This is now the default for playtree_setup.  The caller can simply set
it explicitly.

We make this part of a new :playground export tag for Dgit.pm,
which is going to gain other bits shortly.

And, we add some comments, and move the global; more code and
commentary will appear around this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground refactoring: dgit: Make prep_ud no longer take a directory
Ian Jackson [Wed, 2 Aug 2017 11:17:32 +0000 (12:17 +0100)]
playground refactoring: dgit: Make prep_ud no longer take a directory

Nothing passes this argument, so no functional change.
This will make forthcoming changes more obviously correct.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoRevert "Dgit: Introduce in_workarea and fresh_workarea"
Ian Jackson [Wed, 2 Aug 2017 13:20:32 +0000 (14:20 +0100)]
Revert "Dgit: Introduce in_workarea and fresh_workarea"

This reverts commit aa073c5bdde68a1ab5026ba4daaf29c8bf361532.

These functions are not used on this branch.  I originally put them
here because I thought dgit would want to use them, but actually it
doesn't really.

There are no tests for them here and we are about to engage in
refactoring which will probably accidentally break them.  Instead,
delete them.

This revert should itself be re-reverted when these functionsn are
going to be used.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoplayground: playtree: rename from workarea
Ian Jackson [Fri, 28 Jul 2017 17:27:36 +0000 (18:27 +0100)]
playground: playtree: rename from workarea

We are going to use "playground" for containing temporary directory,
and playtree for git trees within it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit: break must_getcwd out into Dgit.pm
Ian Jackson [Fri, 28 Jul 2017 17:33:52 +0000 (18:33 +0100)]
Dgit: break must_getcwd out into Dgit.pm

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: document refactoring
Ian Jackson [Wed, 26 Jul 2017 22:30:44 +0000 (23:30 +0100)]
changelog: document refactoring

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: factor out build_or_push_prep_early sub
Sean Whitton [Sun, 12 Feb 2017 23:14:24 +0000 (16:14 -0700)]
dgit: factor out build_or_push_prep_early sub

This is to permit code deduplication between the commands for pushing
and building.  Both these sets of commands deny usage of -p, and set
$isuite, $package and $version based on the changelog.

In particular, a command that will perform a build /and/ a push can
first perform the check for -p, set $package and then call functions
to do the build.  These latter functions will not complain that
$package is set thanks to the $build_or_push_prep_early_done logic.

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agocmd_push: Factor out all prep code
Sean Whitton [Fri, 3 Feb 2017 04:15:15 +0000 (21:15 -0700)]
cmd_push: Factor out all prep code

Will be re-used by cmd_push_source.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Refactor to create human-readable $subcommand
Sean Whitton [Fri, 3 Feb 2017 04:11:09 +0000 (21:11 -0700)]
dgit: Refactor to create human-readable $subcommand

$cmd,$subcommand now global vars.

No functional change.

Signed-off-by: Sean Whitton <spwhitton@spwhitton.name>
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: in_workarea: Use changedir
Ian Jackson [Tue, 18 Jul 2017 20:44:33 +0000 (21:44 +0100)]
Dgit.pm: in_workarea: Use changedir

This produces better debug output.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: changedir: Break out from dgit
Ian Jackson [Fri, 14 Jul 2017 15:36:22 +0000 (16:36 +0100)]
Dgit.pm: changedir: Break out from dgit

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit: Introduce in_workarea and fresh_workarea
Ian Jackson [Fri, 3 Feb 2017 16:33:57 +0000 (16:33 +0000)]
Dgit: Introduce in_workarea and fresh_workarea

No callers yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: git_slurp_config_src: Break out from dgit
Ian Jackson [Fri, 14 Jul 2017 09:53:33 +0000 (10:53 +0100)]
Dgit.pm: git_slurp_config_src: Break out from dgit

We are going to want this so we can provide better subtree setup
functions in Dgit.pm, including the config transfer.

No functional change, except that we now run just "git" from the path,
rather than honouring dgit's @git (which is not available in Dgit.pm).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: workarea_setup: Break out from dgit
Ian Jackson [Mon, 10 Jul 2017 10:30:51 +0000 (11:30 +0100)]
Dgit.pm: workarea_setup: Break out from dgit

This subroutine needs a hash of the main tree's git config, so that
some of the config settings which affect the object store can be
properly honoured.

Purely code motion, except for the changed reference to $gitcfgs.
No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: simplify gitattrs setup in mktree_in_ud_here
Ian Jackson [Mon, 10 Jul 2017 10:19:27 +0000 (11:19 +0100)]
dgit: simplify gitattrs setup in mktree_in_ud_here

We are going to want to break the meat of mktree_in_ud here out into
Dgit.pm.  Dgit.pm doesn't have access_cfg, is_gitattrs_setup, and so
on.

On the other hand, we don't need any of that here: this is a fresh
tree of our own (so there is no existing .git/info/attributes to
combine with our new settings), and we ignore the config already.

So overall, little functional change: the .git/info/attributes in our
private working areas are now slightly simpler: they don't indirect
via git-defuse-attrs.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: $negate_harmful_gitattrs: Break out from dgit
Ian Jackson [Fri, 10 Mar 2017 17:40:35 +0000 (17:40 +0000)]
Dgit.pm: $negate_harmful_gitattrs: Break out from dgit

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDgit.pm: runcmd: Break out from dgit
Ian Jackson [Fri, 3 Feb 2017 21:54:56 +0000 (21:54 +0000)]
Dgit.pm: runcmd: Break out from dgit

Move runcmd into Dgit.pm.

We need to change dgit-repos-server too, but luckily its definition is
essentially equivalent.

No functional change to dgit.  dgit-repos-server now prints somewhat
better error messages when subprocesses fail.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: sbuild-gitish: Find sbuild rune in the manpage
Ian Jackson [Sun, 16 Jul 2017 18:08:24 +0000 (19:08 +0100)]
test suite: sbuild-gitish: Find sbuild rune in the manpage

This avoids duplicating this rune (so the manpage can't get out of
date).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit-user(7): Provide information about how to use sbuild
Ian Jackson [Sun, 16 Jul 2017 17:27:16 +0000 (18:27 +0100)]
dgit-user(7): Provide information about how to use sbuild

Quite ugly due to #868527.  Closes:#868526.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit-user(7): Fixed example rune to use curl
Ian Jackson [Sun, 16 Jul 2017 17:26:07 +0000 (18:26 +0100)]
dgit-user(7): Fixed example rune to use curl

curl prints to stdout, as the rune expects.

Reported-by: Simon Tatham
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agosbuild-gitish: New test case to check running sbuild from git
Ian Jackson [Sun, 16 Jul 2017 16:51:15 +0000 (17:51 +0100)]
sbuild-gitish: New test case to check running sbuild from git

This involves creating a new "gitish-only" branch in the example
worktree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoMerge branch 'stable' into HEAD base.pbuilder-config
Ian Jackson [Sun, 16 Jul 2017 20:40:29 +0000 (21:40 +0100)]
Merge branch 'stable' into HEAD

2 years agochangelog: finalise 3.12 archive/debian/3.12 debian/3.12
Ian Jackson [Sun, 16 Jul 2017 20:37:03 +0000 (21:37 +0100)]
changelog: finalise 3.12

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoquilt fixup: Check that funny changes are represented properly
Ian Jackson [Sun, 16 Jul 2017 11:55:54 +0000 (12:55 +0100)]
quilt fixup: Check that funny changes are represented properly

Specifically, do dgit push --dry-run.  This will check that the source
package and git tree agree - ie, that what we have produced can
round-trip through dpkg-source.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoquilt fixup: Check we can delete files with funny modes
Ian Jackson [Sun, 16 Jul 2017 11:54:11 +0000 (12:54 +0100)]
quilt fixup: Check we can delete files with funny modes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: unrepres. changes: Tolerate creation of symlinks
Ian Jackson [Sun, 16 Jul 2017 10:41:14 +0000 (11:41 +0100)]
dgit: unrepres. changes: Tolerate creation of symlinks

Experimentally, dpkg-source on stretch will create patches to create
new symlinks.  (It will fail on attempts to modify existing symlinks
and it ignores attempts to change plain file executability.)

Implementation: add an alternative which tolerates the git symlink
mode.  This replaces the check on $oldmode, which in this context we
know is all 0s and can therefore never match.

While we're here, change the error message.

Closes:#857382.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: unrepres. changes: Tolerate deletion of executable files
Ian Jackson [Sun, 16 Jul 2017 10:28:50 +0000 (11:28 +0100)]
dgit: unrepres. changes: Tolerate deletion of executable files

We don't care what the old mode was; if we tell dpkg-source to record
the deletion it can do so.

But we do care that it was a file.  Experimentally, dpkg-source on
stretch ignores attempts to delete symlinks.

The removal of the check for $newmode has no functional change,
because in this context we know that $newmode is all 0s.  If it
wasn't, we would have been in "both old and new files exist", above.
So that limb of the test will never match and should be removed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: unrepres. changes: Separate out creation and deletion
Ian Jackson [Sun, 16 Jul 2017 10:26:32 +0000 (11:26 +0100)]
dgit: unrepres. changes: Separate out creation and deletion

We are going to want to handle these cases separately because the
behaviour of dpkg-source is different.

In this commit, simply clone the existing code (and add a few
comments), so no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: unrepres. changes: Prepare to tolerate symlinks
Ian Jackson [Sun, 16 Jul 2017 10:19:29 +0000 (11:19 +0100)]
dgit: unrepres. changes: Prepare to tolerate symlinks

Permit symlinks (which have mode 120000 in git) to make their way
through for more detailed checks.

No functional change except to error messages, because:

 * If neither thing was a symlink, then the existing regexps
   still match and the new "modified symlink" clause will not,
   so the flow is unchanged.

 * Otherwise, if both $oldmode and $newmode match [^0], ie,
   this is a modification rather than deletion or removal,
   we insist that $oldmode=$newmode, and then, fail the
   new "modified symlink" check.

 * Otherwise, we fail the check for default mode.

So in the case we are allowing to proceed further, we fail as before.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: Test symlink modification and deletion, not creation
Ian Jackson [Sun, 16 Jul 2017 10:50:40 +0000 (11:50 +0100)]
test suite: Test symlink modification and deletion, not creation

We are going to tolerate symlink creation, because dpkg-source can
consume patches to create symlinks (even though it cannot create
them).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: run git gc on tests/worktrees/example_1.0.tar
Ian Jackson [Sun, 16 Jul 2017 11:35:10 +0000 (12:35 +0100)]
test suite: run git gc on tests/worktrees/example_1.0.tar

Specifically,
  git-gc --aggressive --prune=all
This shrinks it quite a lot.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: Introduce example 1.1 orig containg some interesting objects
Ian Jackson [Sun, 16 Jul 2017 11:31:49 +0000 (12:31 +0100)]
test suite: Introduce example 1.1 orig containg some interesting objects

We are going to want to test alternation/deletion of symlinks and
non-644 files.

To test this we need to switch to using example_1.1.orig.tar.gz (and
the corresponding git branch in the worktree), so that we have
such existing objects.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit-repos-server: Do not reject commits with no author/committer email
Ian Jackson [Sun, 16 Jul 2017 01:23:26 +0000 (02:23 +0100)]
dgit-repos-server: Do not reject commits with no author/committer email

But still insist on date, and hence on the actual committer and author
commit header fields.  Peter Green reports that eg 66c65d90db100435 in
upstream linux.git is such a commit (and is accepted by github).
Closes:#863353.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: import-dsc: Test missing files, particularly in ..
Ian Jackson [Sat, 15 Jul 2017 23:47:01 +0000 (00:47 +0100)]
test suite: import-dsc: Test missing files, particularly in ..

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: t-report-fail: print $PWD as part of failure message
Ian Jackson [Sat, 15 Jul 2017 23:46:16 +0000 (00:46 +0100)]
test suite: t-report-fail: print $PWD as part of failure message

This is really helpful when debugging.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit import: Avoid making broken symlinks in ..
Ian Jackson [Sat, 15 Jul 2017 22:27:29 +0000 (23:27 +0100)]
dgit import: Avoid making broken symlinks in ..

In most cases we could carry on and fail later.  But creating a broken
symlink is undesirable, particularly because it might prevent dgit
from trying to make a non-broken symlink pointing elsewhere in future
(or prevent dget from downloading the file).

Even worse, if the dsc is in .. but an absolute path was provided, we
would make a circular symlink!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit import: Right error message for missing files in ..
Ian Jackson [Sat, 15 Jul 2017 22:21:13 +0000 (23:21 +0100)]
dgit import: Right error message for missing files in ..

Close examination of this code path reveals that:
 * The error is generated only if $there contains no slash.
 * This can only occur if $dscfn matches the first regexp,
   ie $dscfn is [./]../X in which case $there becomes X
 * So in this situation, $there is simply the dsc filename
   which is supposed to be in ..
 * What we should be testing is ../$f but that is $here
   which is what are trying to create and which we statted
   earlier and got ENOENT for.

So this occurs when the dsc is in .. and a file it refers to is
missing.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit import: Defend against broken symlinks in ..
Ian Jackson [Sat, 15 Jul 2017 22:15:54 +0000 (23:15 +0100)]
dgit import: Defend against broken symlinks in ..

Provide a special error message if lstat succeeds but lstat fails.

This is not hypothetical - currently even dgit import ../blah.dsc can
generate this situation !

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoMerge tag 'archive/debian/3.11_deb9u1' into stable
Ian Jackson [Sat, 15 Jul 2017 21:58:53 +0000 (22:58 +0100)]
Merge tag 'archive/debian/3.11_deb9u1' into stable

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: finalise stable update archive/debian/3.11_deb9u1 debian/3.11_deb9u1
Ian Jackson [Tue, 11 Jul 2017 16:12:08 +0000 (17:12 +0100)]
changelog: finalise stable update

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoMerge branch 'stable'
Ian Jackson [Sun, 9 Jul 2017 15:29:38 +0000 (16:29 +0100)]
Merge branch 'stable'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Regularise patch filenames, and defend against funny commit subjects
Ian Jackson [Sun, 9 Jul 2017 13:45:17 +0000 (14:45 +0100)]
dgit: Regularise patch filenames, and defend against funny commit subjects

* Do not specified patch names which look like series filenames
* When we invent a filename based on a commit message, add ".patch".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agousing-these: New script to help with ad-hoc testing
Ian Jackson [Sun, 9 Jul 2017 15:13:44 +0000 (16:13 +0100)]
using-these: New script to help with ad-hoc testing

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: start 4.1~
Ian Jackson [Sun, 9 Jul 2017 15:13:14 +0000 (16:13 +0100)]
changelog: start 4.1~

2 years agodgit: Pass --no-renames to git diff-tree -z, avoiding potential trouble
Ian Jackson [Sun, 9 Jul 2017 12:54:02 +0000 (13:54 +0100)]
dgit: Pass --no-renames to git diff-tree -z, avoiding potential trouble

Without this option, git-diff-tree might detect a rename (or possibly
even a copy).  If it does it prints a different output format with a
status of C or R *and a separate filename*.  The latter is an
additional nul-terminated record and would get dgit's interpreter of
the git diff-tree output out of step.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: start 3.12~
Ian Jackson [Sun, 9 Jul 2017 12:48:13 +0000 (13:48 +0100)]
changelog: start 3.12~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: finalise 3.11 archive/debian/3.11 debian/3.11
Ian Jackson [Sat, 8 Jul 2017 21:42:47 +0000 (22:42 +0100)]
changelog: finalise 3.11

2 years agoActually understand foo,-security (!)
Ian Jackson [Sat, 8 Jul 2017 21:31:09 +0000 (22:31 +0100)]
Actually understand foo,-security (!)

There does not in fact seem to have been any code which implements
this line from dgit(7):
    If a specified subsuite starts with - then mainsuite is prepended.
The code which canonicalises it back to the version with ",-" does
work, though.

Closes:#867189.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: mkdir .git/info in setup_gitattrs
Ian Jackson [Fri, 10 Mar 2017 17:43:57 +0000 (17:43 +0000)]
dgit: mkdir .git/info in setup_gitattrs

This might be necessary if setup_mergechangelogs were disabled.
(This is something of a latent bug, since `git init' creates
.git/info.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: Garden and reformat
Ian Jackson [Sat, 8 Jul 2017 20:20:33 +0000 (21:20 +0100)]
changelog: Garden and reformat

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: Test multisuite clone without --rm-on-error.
Ian Jackson [Sat, 8 Jul 2017 20:18:29 +0000 (21:18 +0100)]
test suite: Test multisuite clone without --rm-on-error.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: clone multisuite works even without --no-rm-on-error.
Ian Jackson [Sat, 8 Jul 2017 20:11:52 +0000 (21:11 +0100)]
dgit: clone multisuite works even without --no-rm-on-error.

This is slightly subtle.

We need $rmonerror in cmd_clone to be cleared.  cmd_clone does that
only when clone itself returns.  The multisuite plumbing means that
currently, clone returns only in the child which set up the tree and
set up the first suite.  Unsetting $rmonerror there is correct because
we want to keep the tree on success, and the parent will remove it if
there is a later failure.  (Ie, the child remains responsible for
removing the tree if it itself fails.)

In the parent, we get $multi_fetched==1 if not only our original clone
child succeeded, but all the other fetches worked too.  We are now
responsible for the tree.  If our final tasks are successful, again,
we need not to delete the tree.

Closes:#867434.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: badcommit-fixup: Check core.sharedRepository works
Ian Jackson [Sat, 8 Jul 2017 19:47:23 +0000 (20:47 +0100)]
test suite: badcommit-fixup: Check core.sharedRepository works

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit-badcommit-fixup: Honour core.sharedRepository.
Ian Jackson [Sat, 8 Jul 2017 19:46:12 +0000 (20:46 +0100)]
dgit-badcommit-fixup: Honour core.sharedRepository.

Closes:#867603.

In principle it might be nicer to copy more options.  But we don't
want to duplicate the logic in prep_ud in dgit, and we don't want to
make this script too standalone.  I'm not aware of other options that
are important, rather than nice-to-have tuneables.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: honour more pre-tree git config options in our private trees
Ian Jackson [Sat, 8 Jul 2017 19:21:30 +0000 (20:21 +0100)]
dgit: honour more pre-tree git config options in our private trees

These share the user's object store and we should manipulate the
object store the way the user wants.  In particular,
core.sharedRepository is important.

Prompted by #867603, which is the same bug in dgit-badcommit-fixup.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: tolerate compressor terminating with SIGPIPE.
Ian Jackson [Sat, 8 Jul 2017 19:14:28 +0000 (20:14 +0100)]
dgit: tolerate compressor terminating with SIGPIPE.

Closes:#857694.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: avoid dying with wrong message if compression fails
Ian Jackson [Sat, 8 Jul 2017 19:11:23 +0000 (20:11 +0100)]
dgit: avoid dying with wrong message if compression fails

When dgit intended to report a decompressor had failed, it would
instead crash with no useful message.  This was due to
generate_commits_from_dsc's @compr_cmd being shadowed inside
the if $compr_ext.

This is part of #857694.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoinfrastructure: Cope with new git-receive-pack which has quarantine feature
Ian Jackson [Sat, 8 Jul 2017 18:59:41 +0000 (19:59 +0100)]
infrastructure: Cope with new git-receive-pack which has quarantine feature

Ie, work around #867702.  See the bug there for discussion.

(We are perhaps flying a little close to the wind with our wrapper
script location, but it doesn't seem likely to break, to me.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoinfra: dgit-repos-server: Break out mkscript
Ian Jackson [Sat, 8 Jul 2017 18:56:22 +0000 (19:56 +0100)]
infra: dgit-repos-server: Break out mkscript

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Add many pre_* to call no_local_git_cfg
Ian Jackson [Wed, 28 Jun 2017 16:00:19 +0000 (17:00 +0100)]
dgit: Add many pre_* to call no_local_git_cfg

For each operation which can meaningfully be run outside a git tree,
arrange to call no_local_git_cfg and thus avoid running
  git config --local

There is one slight infelicity: some subcommands (notably
archive-api-query) could in theory be run within a git tree and expect
that git tree to influence their output.  However, this seems
unlikely.  In fact, I think there are probably only in-tree callers
and the in-tree callers do not do this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Provide no_local_git_config
Ian Jackson [Wed, 28 Jun 2017 15:59:00 +0000 (16:59 +0100)]
dgit: Provide no_local_git_config

Part of the fix to #865863.  No callers yet, so no functional change.

This function not only arranges not to call git config --local.
Removing 'local' from @gitcfgsources also prevents various config
lookup machinery from trying to find information in $cfg{local}
now of course does not exist.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: rpush: Do argument parsing and chdir in pre_...
Ian Jackson [Sat, 8 Jul 2017 17:46:18 +0000 (18:46 +0100)]
dgit: rpush: Do argument parsing and chdir in pre_...

The only thing in between pre_... and cmd_... os git_slurp_config.

The code now in pre_... does the following things, all of these should
be done before git_slurp_config:

 * Sets some global variables affecting debugging and logging output.
   If git_slurp_config is told to print debugging, it should be
   affected by these changes.

 * Rearranges its file descriptors so that stdin/stdout are as the
   rest of dgit expects, and the protocol is on PI/PO.  If
   git_slurp_config were to use stdin/stdout, it ought to use the
   "normal" versions, and not access the protocol streams.

 * Checks and calculates the negotiated protocol version.  This is not
   affected by the config, only by the arguments form the caller and
   our own idea of the protocol versions we support.

 * Changes to the appropriate working tree.  Doing this before
   slurping the config arranges to honour the local git config from
   build host working tree.  (It also avoids rpush failing on newer
   git due to asking for git config --local in the wrong place.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Postpone call to git_slurp_config
Ian Jackson [Wed, 28 Jun 2017 15:39:59 +0000 (16:39 +0100)]
dgit: Postpone call to git_slurp_config

This is needed to fix #865863: we need to know what our operation is,
before we can decide whether to look for --local git config.

The code which now runs earlier is:
 * The messages about $dryrun_level (which is set only by
   the command line, and not by configuration - verified by
   searching for $dryrun_level).
 * Usage failure if @ARGV empty.  This is not affected by
   configuration.  (parseopts does the argument parsing and
   already runs before git_slurp_config.)
 * Extracting the $cmd from @ARGV.
 * Calling $pre_fn.  There is only one pre_* sub, which is
   pre_gbp_build.  It provides the default for $quilt_mode.
   $quilt_mode is indeed somewhat entangled with the git config,
   but this takes place in parseopts_late_defaults, which is
   called much later.

Therefore there is no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: curl --proto-redir settings: do not use qw() for ...,...
Ian Jackson [Sat, 8 Jul 2017 17:37:33 +0000 (18:37 +0100)]
dgit: curl --proto-redir settings: do not use qw() for ...,...

This syntax generates the following spurious warning with buster's
perl (but not with stretch's):
  Possible attempt to separate words with commas at /home/ian/things/Dgit/2dgit/dgit line 103.

It seems better to keep this warning everywhere else, and using a
different syntax is easier than disabling it around this construct.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: Cope with git restricting ext:: protocols.
Ian Jackson [Sat, 8 Jul 2017 17:29:43 +0000 (18:29 +0100)]
test suite: Cope with git restricting ext:: protocols.

buster's git rejects ext:: by default.  See #XXXX and
  man git-config |less +/'protocol.*allow'

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Cope if the archive server sends an HTTP redirect
Ian Jackson [Sat, 8 Jul 2017 17:05:36 +0000 (18:05 +0100)]
dgit: Cope if the archive server sends an HTTP redirect

We achieve this by passing -L to curl.

We also pass an appropriate-seeming --proto-redir, because the curl
manual is not entirely reassuring that following redirections with the
default configuration is safe.

This finally fixes #867185/#867309.  What happens there is that curl
gets a redirect, along with an HTML error document.  curl then exits
with status zero, effectively pretending that the error document is
the resource which was requested.  dgit notices that something is
wrong because the file does not have the expected cryptographic
checksum.

I suspect that there are other download problems which would give a
similar effect.  Sadly the curl manpage doesn't seem to suggest a way
to avoid this.  At least, dgit will never carry on in such a
situation, since it insists that the file has the right hash.  And if
it does have the right hash we don't really care how it was obtained.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: avoid "Use of uninitialized value $got in concatenation"
Ian Jackson [Sat, 8 Jul 2017 16:59:55 +0000 (17:59 +0100)]
dgit: avoid "Use of uninitialized value $got in concatenation"

When dgit wanted to report that a file it downloaded had the wrong
checksum, it would instead crash with this message.  This was due to
complete_file_from_dsc's $got (which is relied on by the callers of
$checkhash) being shadowed inside $checkhash.

This is part of #867185/#867185.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: fix rpush+buildinfo: Transfer buildinfos for signing.
Ian Jackson [Sat, 8 Jul 2017 16:22:53 +0000 (17:22 +0100)]
dgit: fix rpush+buildinfo: Transfer buildinfos for signing.

buildinfos are supposed to be signed.  And, indeed, if they are
present, debsign wants to sign them.  That means they need to be
transferred to the signing end, and back again.

We check that the filename is not totally unreasonable, but do not
attempt to verify it completely.  If there are situations where
unwanted or confusing buildinfos are generated, this is the fault of
the build process.  dgit rpush should, in this respect, do the same as
debsign+dput - ie faithfully sign and upload what the build has
provided.

We do check that the buildinfo doesn't look too much like a .changes,
and mentions the same files as the .changes (insofar as they mention
files in common).  This is a rather nugatory defence against some
kinds of bait and switch attacks.

This is in some sense an incompatible protocol change: if the build
host has a new dgit, and sends buildinfos, an old dgit on the
initiator will declare a protocol violation.  However, the new
protocol elements occur only when needed.  in this situation, the only
way to get things to work at all with the old dgit at either end would
be to strip out the buildinfos, which is obviously undesirable.

Closes:#867693.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agorpush: break out @rfiles in dopush
Ian Jackson [Sat, 8 Jul 2017 15:51:06 +0000 (16:51 +0100)]
rpush: break out @rfiles in dopush

This very slight refactoring makes it easier to add additional files
here, and also factors out the appending of $dryrunsuffix to each one.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agoDeclare fast forward from 0.22-experimental2 archive/debian/4.0 debian/4.0
Ian Jackson [Sun, 12 Feb 2017 22:22:31 +0000 (22:22 +0000)]
Declare fast forward from 0.22-experimental2

[dgit --overwrite=0.22-experimental2]

2 years agochangelog: finalise 4.0
Ian Jackson [Sun, 12 Feb 2017 22:22:43 +0000 (22:22 +0000)]
changelog: finalise 4.0

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: --deliberately-not-fast-forward works properly in split view quilt modes (suppr...
Ian Jackson [Sun, 12 Feb 2017 22:20:46 +0000 (22:20 +0000)]
dgit: --deliberately-not-fast-forward works properly in split view quilt modes (suppressing the pseudomerge).

2 years agochangelog: start 4.0~
Ian Jackson [Sun, 12 Feb 2017 22:06:43 +0000 (22:06 +0000)]
changelog: start 4.0~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: start 3.11~
Ian Jackson [Sun, 12 Feb 2017 22:06:03 +0000 (22:06 +0000)]
changelog: start 3.11~

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: finalise 3.10 archive/debian/3.10 debian/3.10
Ian Jackson [Mon, 6 Feb 2017 17:49:50 +0000 (17:49 +0000)]
changelog: finalise 3.10

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agochangelog: Update for docs fixes
Nicholas D Steeves [Sun, 29 Jan 2017 23:01:27 +0000 (16:01 -0700)]
changelog: Update for docs fixes

Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
2 years agodocs: Fix typos
Nicholas D Steeves [Sun, 29 Jan 2017 23:00:16 +0000 (16:00 -0700)]
docs: Fix typos

Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
2 years agoimport-maintmangle: New test for changelog Maintainer mangling.
Ian Jackson [Sun, 5 Feb 2017 20:34:40 +0000 (20:34 +0000)]
import-maintmangle: New test for changelog Maintainer mangling.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agolib-import-chk: New import_chk_changelog_massage hook
Ian Jackson [Sun, 5 Feb 2017 20:33:47 +0000 (20:33 +0000)]
lib-import-chk: New import_chk_changelog_massage hook

Nothing sets this yet, so no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: importing: Better handle commas in changelog maintainer fields
Ian Jackson [Sun, 5 Feb 2017 15:22:32 +0000 (15:22 +0000)]
dgit: importing: Better handle commas in changelog maintainer fields

Some maintainers have written commas in the maintainer field of their
changelog entries.  Such changelog entries could not be imported.

clogp_authline had code to replace multiple maintainers (which are
hypothetical right now) into just the first, but that trips on these
maintainers with commas.

Ultimately we should expect that the Maintainer: field from
dpkg-parsechangelog is in (a subset of) RFC5322 recipient field
format.  In that format, any commas would need to be quoted.

So:

If the Maintainer field from dpkg-parsechangelog has a comma which has
no @ or " before it, then we consider it a single old-school
comma-containing maintainer.  If it were intended as multiple
maintainers, then the first maintainer has no email address.  Not
coping properly with that very-hypothetical future seems OK.  We
simply delete the comma, so that the things we record in the git
history are more conservative.

If there is a " we leave things untouched in the hope that this is a
single address, albeit with some quoting.  The alternative would be to
try to use a full RFC5322 parser.  That's quite a risky change.

Perhaps we will revisit this after stretch.

For now this Closes:#852661.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: lib-import-chk: Test commit authorship
Ian Jackson [Sun, 5 Feb 2017 19:45:11 +0000 (19:45 +0000)]
test suite: lib-import-chk: Test commit authorship

Check that commits have smae authorship as appears in the changelog.
(Or, at least, the same authorship set.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Do not fail when run with detached HEAD. Closes:#853022.
Ian Jackson [Sun, 5 Feb 2017 14:41:43 +0000 (14:41 +0000)]
dgit: Do not fail when run with detached HEAD.  Closes:#853022.

Specifically:

* Pass -q to git-symbolic-ref.  That means that it doesn't print
  an error message when HEAD is not a symbolic ref (ie, a
  detached head), and it means that the exit status is then 1
  rather than 128.

* If the return value from cmdoutput_errok is undef, check $? (which
  is the exit status <<8) and then simply pass on the undef.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: branchsuite: Break out @cmd
Ian Jackson [Sun, 5 Feb 2017 14:20:15 +0000 (14:20 +0000)]
dgit: branchsuite: Break out @cmd

We are going to need to reuse this to print a proper error message.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agodgit: Strip initial newline from Changes line from dpkg-parsechangelog
Ian Jackson [Sun, 5 Feb 2017 14:11:10 +0000 (14:11 +0000)]
dgit: Strip initial newline from Changes line from dpkg-parsechangelog

so as to avoid blank line in commit messages.  Closes:#853093.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2 years agotest suite: quilt-useremail: New test for user config copying
Ian Jackson [Sun, 5 Feb 2017 13:17:04 +0000 (13:17 +0000)]
test suite: quilt-useremail: New test for user config copying

Test for #853085.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>