server/keyset.c: Return more informative error codes from ks_decrypt.

Also, do what the commentary says and return zero and break the output
buffer if it's not big enough, rather than returning KSERR_DECRYPT.

The new error codes allow callers to make more sensible decisions about
whether to continue to search for other keys which might work better.