--- /dev/null
+Title: Free software activity in July 2025
+Slug: activity-2025-07
+Date: 2025-08-06 11:41:41 +01:00
+Category: columbiform
+Tags: activity, columbiform, freexian, planet-debian, planet-ubuntu
+Summary: DebConf, and a fiddly OpenSSH upgrade problem.
+Mastodon-ID: 114981399646873713
+
+About 90% of my Debian contributions this month were
+[sponsored](https://www.freexian.com/about/debian-contributions/) by
+Freexian.
+
+You can also support my work directly via
+[Liberapay](https://liberapay.com/cjwatson) or [GitHub
+Sponsors](https://github.com/sponsors/cjwatson).
+
+## DebConf
+
+I attended DebConf for the first time in 11 years (my last one was DebConf
+14 in Portland). It was great! For once I had a conference where I had a
+fairly light load of things I absolutely had to do, so I was able to spend
+time catching up with old friends, making some new friends, and doing some
+volunteering - a bit of Front Desk, and quite a lot of video team work where
+I got to play with sound desks and such. Apparently one of the BoFs ("birds
+of a feather", i.e. relatively open discussion sessions) where I was
+talkmeister managed to break the automatic video cutting system by starting
+and ending precisely on time, to the second, which I'm told has never
+happened before. I'll take that.
+
+I gave a [talk about
+Debusine](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-398-using-debusine-to-pre-test-your-unstable-uploads.av1.webm),
+along with [helping Enrico run a Debusine
+BoF](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-669-debusine-workflow-bof.av1.webm).
+We still need to process some of the feedback from this, but are generally
+pretty thrilled about the reception. My personal highlight was getting a
+shout-out in a [talk from
+CERN](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-605-approaching-the-speed-of-light-with-debian-debians-role-in-the-worlds-largest-particle-accelerator.av1.webm)
+(in the slide starting at 32:55).
+
+Other highlights for me included a [Python team
+BoF](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-416-debian-python-bof.av1.webm),
+[Ian's tag2upload
+talk](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-397-tag2upload-upload-simply-by-pushing-a-signed-git-tag.av1.webm)
+and some very useful follow-up discussions, a session on [archive-wide
+testing](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-420-future-of-archive-wide-testing.av1.webm),
+a somewhat brain-melting whiteboard session about the "multiarch interpreter
+problem",
+[several](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-649-salsa-bof.av1.webm)
+[useful](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-624-salsa-ci-bof.av1.webm)
+[discussions](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-560-salsa-ci-now-running-for-27-000-packages-what-have-we-learnt.av1.webm)
+about salsa.debian.org, [Matthew's talk on how Wikimedia automates their Debian package
+builds](https://meetings-archive.debian.net/pub/debian-meetings/2025/DebConf25/debconf25-180-automating-downstream-debian-package-builds-and-updates-in-ci.av1.webm),
+and many others. I hope I can start attending regularly again!
+
+## OpenSSH
+
+Towards the end of a release cycle, people tend to do more upgrade testing,
+and this sometimes results in interesting problems. Manfred Stock reported
+["No new SSH connections possible during large part of upgrade to Debian
+Trixie"](https://bugs.debian.org/1109742), and after a little testing in a
+container I confirmed that this was a reproducible problem that would have
+affected many people upgrading from Debian 12 (bookworm), with potentially
+severe consequences for people upgrading remote systems. In fact, there
+were two independent problems that each led to much the same symptom:
+
+* OpenSSH 9.8 split the monolithic `sshd` listener process into two
+ pieces: a minimal network listener (still called `sshd`), and an
+ `sshd-session` process dealing with each individual session. (OpenSSH
+ 10.0 further split `sshd-session`, adding an `sshd-auth` process that
+ deals with the user authentication phase of the protocol.) This hardens
+ the OpenSSH server by using different address spaces for privileged and
+ unprivileged code.
+
+ Before this change, when `sshd` received an incoming connection, it
+ forked and re-executed itself with some special parameters to deal with
+ it. After this change, it forks and executes `sshd-session` instead,
+ and `sshd` no longer accepts the parameters it used to accept for this.
+
+ Debian package upgrades happen in two phases: first we unpack the new
+ files onto disk, and then we run some package-specific configuration
+ steps which usually include things like restarting services. (I'm
+ simplifying, but this is good enough for this post.) Normally this is
+ fine, and in fact desirable: the old service keeps on working, and this
+ approach often allows breaking what would otherwise be difficult cycles
+ by ensuring that the system is in a more coherent state before trying to
+ restart services. However, in this case, unpacking the new files onto
+ disk immediately means that new SSH connections no longer work: the old
+ `sshd` receives the connection and tries to hand it off to a
+ freshly-executed copy of the new `sshd` binary on disk, which no longer
+ supports this.
+
+ If you're just upgrading OpenSSH on its own or with a small number of
+ other packages, this isn't much of a problem as the listener will be
+ restarted quite soon; but if you're upgrading from bookworm to trixie,
+ there may be a long gap when you can't SSH to the system any more, and
+ if something fails in the middle of the upgrade then you could be in
+ trouble.
+
+ So, what to do? I considered keeping a copy of the old `sshd` around
+ temporarily and patching the new `sshd` to re-execute it if it's being
+ run to handle an incoming connection, but that turned out to fail in my
+ first test: dependencies are normally only checked when configuring a
+ package, so it's possible to unpack `openssh-server` before unpacking a
+ newer `libc6` that it depends on, at which point you can't execute the
+ new `sshd` at all. (That also means that the approach of restarting the
+ service at unpack time instead of configure time is a non-starter.) We
+ needed a different idea.
+
+ `dpkg`, the core Debian package manager, has a specialized facility
+ called "diversions": you can tell it that when it's unpacking a
+ particular file it should put it somewhere else instead. This is
+ normally used by administrators when they want to install a
+ locally-modified version of a particular file at their own risk, or by
+ packages that knowingly override a file normally provided by some other
+ package. However, in this case it turns out to be useful for
+ `openssh-server` to temporarily divert one of its own files! When
+ upgrading from before 9.8, it now diverts `/usr/sbin/sshd` to
+ `/usr/sbin/sshd.session-split` before the new version is unpacked, then
+ removes the diversion and moves the new file into place once it's ready
+ to restart the service; this reduces the period when incoming
+ connections fail to a minimum. (We actually have to pretend that the
+ diversion is being performed on behalf of a slightly different package
+ since we're using `dpkg-divert` in a strange way here, but it all
+ works.)
+
+* Most OpenSSH processes, including `sshd`, check for a compatible version
+ of the OpenSSL library when they start up. This check used to be very
+ picky, among other things requiring both the major and minor number to
+ match. OpenSSL 3 has a [better versioning
+ policy](https://www.openssl-library.org/policies/general/versioning-policy/),
+ and so OpenSSH 9.4p1 [relaxed this
+ check](https://anongit.mindrot.org/openssh.git/commit/?id=b7afd8a4ecaca8afd3179b55e9db79c0ff210237).
+
+ Unfortunately, bookworm shipped with OpenSSH 9.2p1, which means that as
+ soon as you unpack the new `libssl3` during an upgrade (actually
+ `libssl3t64` due to the [64-bit `time_t`
+ transition](https://wiki.debian.org/ReleaseGoals/64bit-time)), `sshd`
+ stops working. This couldn't be fixed by a change in trixie; we needed
+ to change bookworm in advance of the upgrade so that it would tolerate
+ newer versions of OpenSSL. And time was tight if we wanted to maximize
+ the chance that people would apply that stable update before upgrading
+ to trixie; there isn't going to be another point release of Debian 12
+ before the release of Debian 13.
+
+ Fortunately, there's a
+ [`stable-updates`](https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#special-case-the-stable-updates-suite)
+ mechanism for exactly this sort of thing, and the stable release
+ managers kindly accepted my proposal to fix this there.
+
+The net result is that if you apply updates to bookworm (including
+`stable-updates` / `bookworm-updates`, which is enabled by default) before
+starting the upgrade to trixie, everything should be fine. Many thanks to
+Manfred for reporting this with just enough time to spare that we were able
+to fix it before Debian 13 is released in a few days!
+
+## debmirror
+
+I did my twice-yearly refresh of debmirror's [`mirror_size`
+documentation](https://salsa.debian.org/debian/debmirror/-/blob/master/mirror_size),
+and applied a [patch from Christoph Goehre](https://bugs.debian.org/1108274)
+to improve mirroring of installer files.
+
+## madison-lite
+
+I [proposed](https://bugs.debian.org/1109003) renaming this project along
+with the `rmadison` tool in `devscripts`, although I'm not yet sure what a
+good replacement name would be.
+
+## Python team
+
+I upgraded python-expandvars, python-typing-extensions (in experimental),
+and webtest to new upstream versions.
+
+I backported fixes for some security vulnerabilities to unstable:
+
+* python-urllib3: [CVE-2025-50181](https://bugs.debian.org/1108076),
+ [CVE-2025-50182](https://bugs.debian.org/1108077)
+
+I fixed or helped to fix a number of release-critical bugs:
+
+* [bitstruct: autopkgtest regression: invalid command
+ 'test'](https://bugs.debian.org/1108968)
+* [django-pipeline: autopkgtest failure](https://bugs.debian.org/1102611)
+ ([contributed supporting fix
+ upstream](https://github.com/jazzband/django-pipeline/pull/837))
+* [pnopaste: Fails to install with debconf noninteractive
+ frontend](https://bugs.debian.org/1108946) (suggested possible patch)
+* [py3dns: autopkgtest regression: '96.7.128.186' !=
+ '93.184.215.14'](https://bugs.debian.org/1108799) ([contributed
+ upstream](https://code.launchpad.net/~cjwatson/py3dns/+git/py3dns/+merge/488798))
+* [python-marshmallow-dataclass: autopkgtest depends on removed package
+ python-marshmallow-enum](https://bugs.debian.org/1108782)
+* [python-pkgconfig: autopkgtest regression: list index out of
+ range](https://bugs.debian.org/1108795)
+* [python-txrequests: autopkgtest regression:
+ twisted.trial.unittest.FailTest: 200 !=
+ 404](https://bugs.debian.org/1108797)
+
+I fixed some other bugs, mostly `Severity: important`:
+
+* [afew: Unable to remove tags](https://bugs.debian.org/1106896) ([reviewed
+ and merged
+ MR](https://salsa.debian.org/python-team/packages/afew/-/merge_requests/3))
+* [ipy: FTBFS with the nocheck build
+ profile](https://bugs.debian.org/1103313)
+* [paramiko: Does not correctly handle OpenSSH 10
+ version](https://bugs.debian.org/1108434)
+* [python-django-storages: FTBFS with the nocheck build
+ profile](https://bugs.debian.org/1103323)
+* [python-icalendar: Depends on a transitional
+ package](https://bugs.debian.org/1104638) (and follow-up fixes for missing
+ build-dependencies in python-recurring-ical-events, python-x-wr-timezone,
+ and todoman)
+* [python-libais: Stop calling python3 setup.py
+ test](https://bugs.debian.org/1025404) ([contributed supporting fix
+ upstream](https://github.com/schwehr/libais/pull/252))
+
+I reinstated python3-mastodon's build-dependency on and recommendation of
+python3-blurhash, now that the latter has been [fixed to use the correct
+upstream source](https://bugs.debian.org/1101140).
~cjwatson / blog.git / commitdiff