Re: [PATCH] rclone needs a flag to exit nonzero when a file is not found

[nathan todd-stone]

Sure!

I've just switched all my repos from rsync and servers to rclone and s3. In doing so, I encountered an issue with the rclone backend, and fixed it with this patch.

The rclone cli needs a flag to exit nonzero when a file is not found, this flag is --error-on-no-transfer. The help states:

 >> rclone help flags | grep error-on
      --error-on-no-transfer                 Sets exit code 9 if no files are transferred, useful in scripts

The git-remote-gcrypt functions GET and PUT are both expected to exit nonzero on failure, and without this flag, they always exit zero when rclone is used, even when a file is not found.

My initial failure was on line 544: GET "$URL" "$Manifestfile" "$tmp_manifest" 2>| "$tmp_stderr" || {.

GET was returning zero even when the remote file was not found, and later a "No such file or directory" error would occur trying to read "$tmp_manifest", which had never been written.

With this flag, rclone exits nonzero when a remote file is missing for GET, or a local file is missing for PUT. This has fixed rclone so far for me, and is likely all that is needed.

- Nathan Todd-Stone

On Sat, Jun 12, 2021, at 8:52 PM, Sean Whitton wrote:
> Hello,
> 
> On Wed 09 Jun 2021 at 01:33PM -10, nathants wrote:
> 
> > Signed-off-by: Nathan Todd-Stone <me at nathants.com>
> > ---
> >  git-remote-gcrypt | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/git-remote-gcrypt b/git-remote-gcrypt
> > index c519bf8..7e7240f 100755
> > --- a/git-remote-gcrypt
> > +++ b/git-remote-gcrypt
> > @@ -238,7 +238,7 @@ GET()
> >  		(exec 0>&-; rsync -I -W "$(rsynclocation "$1")"/"$2" "$3" >&2)
> >  	elif isurl rclone "$1"
> >  	then
> > -		(exec 0>&-; rclone copyto "${1#rclone://}"/"$2" "$3" >&2)
> > +		(exec 0>&-; rclone copyto --error-on-no-transfer "${1#rclone://}"/"$2" "$3" >&2)
> >  	elif islocalrepo "$1"
> >  	then
> >  		cat "$1/$2" > "$3"
> > @@ -258,7 +258,7 @@ PUT()
> >  		rsync $Conf_rsync_put_flags -I -W "$3" "$(rsynclocation "$1")"/"$2" >&2
> >  	elif isurl rclone "$1"
> >  	then
> > -		rclone copyto "$3" "${1#rclone://}"/"$2" >&2
> > +		rclone copyto --error-on-no-transfer "$3" "${1#rclone://}"/"$2" >&2
> >  	elif islocalrepo "$1"
> >  	then
> >  		cat >| "$1/$2" < "$3"
> 
> Thank you for this, but could you provide more detail for the commit
> message, please?  In what situation are there problems and why is this
> the best way to resolve them?
> 
> -- 
> Sean Whitton
>