--- /dev/null
+Title: Launchpad now supports SSH Ed25519 keys and RSA SHA-2 signatures
+Slug: lp-new-ssh-features
+Date: 2022-02-18 13:49:59 +00:00
+Category: launchpad
+Tags: launchpad, openssh, planet-debian, planet-ubuntu, twisted
+
+As of 2022-02-16, Launchpad supports a couple of features on its SSH
+endpoints (`git.launchpad.net`, `bazaar.launchpad.net`, `ppa.launchpad.net`,
+and `upload.ubuntu.com`) that it previously didn't: [Ed25519 public
+keys](https://bugs.launchpad.net/bugs/907675) (a well-regarded format,
+supported by OpenSSH since 6.5 in 2014) and [signatures with existing RSA
+public keys using SHA-2 rather than
+SHA-1](https://bugs.launchpad.net/bugs/1933722) (supported by OpenSSH since
+7.2 in 2016).
+
+I'm hesitant to call these features "new", since they've been around for a
+long time elsewhere, and people might quite reasonably ask why it's taken us
+so long. The problem has always been that Launchpad can't really use a
+normal SSH server such as OpenSSH because it needs features that aren't
+practical to implement that way, such as virtual filesystems and dynamic
+user key authorization against the Launchpad database. Instead, we use
+[Twisted Conch](https://twistedmatrix.com/trac/wiki/TwistedConch), which is
+a very extensible Python SSH implementation that has generally served us
+well. The downside is that, because it's an independent implementation and
+one that occupies a relatively small niche, it often lags behind in terms of
+newer protocol features.
+
+Catching up to this point has been something we've been working on for
+around five years, although it's taken a painfully long time for a variety
+of reasons which I thought some people might find interesting to go into, at
+least people who have the patience for details of the SSH protocol. Many of
+the delays were my own responsibility, although realistically we probably
+couldn't have added Ed25519 support before OpenSSL/`cryptography` work that
+landed in 2019.
+
+ * In 2015, we did some similar work on [SHA-2 key exchange and MAC
+ algorithms]({filename}/ssh-sha-2-support-in-twisted.md).
+ * In 2016, various other contributors were working on ECDSA and Ed25519
+ support (e.g. [#533](https://github.com/twisted/twisted/pull/533) and
+ [#644](https://github.com/twisted/twisted/pull/644)). At the time, it
+ seemed best to keep an eye on this but mainly leave them to it. I'm very
+ glad that some people worked on this before me - studying their PRs
+ helped a lot, even parts that didn't end up being merged directly.
+ * In 2017, it became clear that this was likely to need some more
+ attention, but before we could do anything else we had to revamp
+ Launchpad's build system to use [pip](https://pip.pypa.io/en/stable/)
+ rather than [buildout](https://www.buildout.org/en/latest/), since
+ without that we couldn't upgrade to any newer versions of Twisted. That
+ proved to be a substantial piece of yak-shaving: first we had to upgrade
+ Launchpad off Ubuntu 12.04, and then the actual [build system
+ rewrite](https://code.launchpad.net/~cjwatson/launchpad/virtualenv-pip/+merge/331388)
+ was a complicated project of its own.
+ * In 2018, I fixed an [authentication
+ hang](https://bugs.launchpad.net/bugs/830679) that happened if a client
+ even tried to offer ECDSA or Ed25519 public keys to Launchpad, and we got
+ ECDSA support fully working in Launchpad. We also discovered as a result
+ of automated interoperability tests run as part of the Debian OpenSSH
+ packaging that Twisted needed to gain support for the new
+ `openssh-key-v1` private key format, which became a prerequisite for
+ Ed25519 support since OpenSSH only ever writes those keys in the new
+ format, and so I [fixed
+ that](https://github.com/twisted/twisted/pull/1193).
+ * In 2019, Python's [cryptography](https://pypi.org/project/cryptography/)
+ package gained support for X25519 (the Diffie-Hellman key exchange
+ function based on [Curve25519](https://en.wikipedia.org/wiki/Curve25519))
+ and Ed25519, and it became somewhat practical to add support to Twisted
+ on top of that. However, it required OpenSSL 1.1.1b, and it seemed
+ unlikely that we would be in a position to upgrade all the relevant bits
+ of Launchpad's infrastructure to use that in the near term. I at least
+ managed to add [curve25519-sha256 key exchange
+ support](https://github.com/twisted/twisted/pull/1202) to Twisted based
+ on some [previous work](https://github.com/twisted/twisted/pull/644) by
+ another contributor, and I prepared [support for Ed25519
+ keys](https://github.com/twisted/twisted/pull/1210) in Twisted even
+ though I knew we weren't going to be able to use it yet.
+ * 2020 was ... well, everyone knows what 2020 was like, plus we had a new
+ baby. I did some experimentation in spare moments, but I didn't really
+ have the focus to be able to move this sort of complex problem forward.
+ * In 2021, I bit the bullet and started seriously working on [fallback
+ mechanisms to allow us to use
+ Ed25519](https://github.com/twisted/twisted/pull/1607) even on systems
+ lacking a sufficient version of OpenSSL, though found myself blocked on
+ figuring out type-checking issues following a code review. It then
+ became clear on the release of [OpenSSH
+ 8.8](https://www.openssh.com/releasenotes.html#8.8p1) that we were going
+ to have to deal with RSA SHA-2 signatures as well, since otherwise
+ OpenSSH in Ubuntu soon wouldn't be able to authenticate to Launchpad by
+ default (which also caused me to delay [uploading 8.8 to Debian
+ unstable](https://bugs.debian.org/996391) for a while). To deal with
+ that, I first had to add [SSH extension
+ negotiation](https://github.com/twisted/twisted/pull/1666) to Twisted.
+ * Finally, in 2022, I added [RSA SHA-2 signature
+ support](https://github.com/twisted/twisted/pull/1692) to Twisted,
+ finally unblocked myself on the type-checking issue with the Ed25519
+ fallback mechanism, quickly put together a [similar fallback mechanism
+ for
+ X25519](https://git.launchpad.net/~launchpad/twisted/+git/twisted/commit/?id=536a8934be619044fc95f51822139b96edea9dcc),
+ backported the whole mess to Twisted 20.3.0 since we currently can't use
+ anything newer due to the somewhat old version of Python 3 that we're
+ running, promptly ran into and fixed a
+ [regression](https://github.com/twisted/twisted/pull/1696) that affected
+ SFTP uploads to `ppa.launchpad.net` and `upload.ubuntu.com`, and finally
+ added Ed25519 as a permissible key type in Launchpad's authserver.
+
+Phew! Thanks to everyone who works on Twisted, `cryptography`, and OpenSSL
+- it's been really useful to be able to build on solid lower-level
+cryptographic primitives - and to those who helped with code review.
~cjwatson / blog.git / commitdiff