Free software activity in July 2025
About 90% of my Debian contributions this month were sponsored by Freexian.
You can also support my work directly via Liberapay or GitHub Sponsors.
DebConf
I attended DebConf for the first time in 11 years (my last one was DebConf 14 in Portland). It was great! For once I had a conference where I had a fairly light load of things I absolutely had to do, so I was able to spend time catching up with old friends, making some new friends, and doing some volunteering - a bit of Front Desk, and quite a lot of video team work where I got to play with sound desks and such. Apparently one of the BoFs (“birds of a feather”, i.e. relatively open discussion sessions) where I was talkmeister managed to break the automatic video cutting system by starting and ending precisely on time, to the second, which I’m told has never happened before. I’ll take that.
I gave a talk about Debusine, along with helping Enrico run a Debusine BoF. We still need to process some of the feedback from this, but are generally pretty thrilled about the reception. My personal highlight was getting a shout-out in a talk from CERN (in the slide starting at 32:55).
Other highlights for me included a Python team BoF, Ian’s tag2upload talk and some very useful follow-up discussions, a session on archive-wide testing, a somewhat brain-melting whiteboard session about the “multiarch interpreter problem”, several useful discussions about salsa.debian.org, Matthew’s talk on how Wikimedia automates their Debian package builds, and many others. I hope I can start attending regularly again!
OpenSSH
Towards the end of a release cycle, people tend to do more upgrade testing, and this sometimes results in interesting problems. Manfred Stock reported “No new SSH connections possible during large part of upgrade to Debian Trixie”, and after a little testing in a container I confirmed that this was a reproducible problem that would have affected many people upgrading from Debian 12 (bookworm), with potentially severe consequences for people upgrading remote systems. In fact, there were two independent problems that each led to much the same symptom:
-
OpenSSH 9.8 split the monolithic
sshd
listener process into two pieces: a minimal network listener (still calledsshd
), and ansshd-session
process dealing with each individual session. (OpenSSH 10.0 further splitsshd-session
, adding ansshd-auth
process that deals with the user authentication phase of the protocol.) This hardens the OpenSSH server by using different address spaces for privileged and unprivileged code.Before this change, when
sshd
received an incoming connection, it forked and re-executed itself with some special parameters to deal with it. After this change, it forks and executessshd-session
instead, andsshd
no longer accepts the parameters it used to accept for this.Debian package upgrades happen in two phases: first we unpack the new files onto disk, and then we run some package-specific configuration steps which usually include things like restarting services. (I’m simplifying, but this is good enough for this post.) Normally this is fine, and in fact desirable: the old service keeps on working, and this approach often allows breaking what would otherwise be difficult cycles by ensuring that the system is in a more coherent state before trying to restart services. However, in this case, unpacking the new files onto disk immediately means that new SSH connections no longer work: the old
sshd
receives the connection and tries to hand it off to a freshly-executed copy of the newsshd
binary on disk, which no longer supports this.If you’re just upgrading OpenSSH on its own or with a small number of other packages, this isn’t much of a problem as the listener will be restarted quite soon; but if you’re upgrading from bookworm to trixie, there may be a long gap when you can’t SSH to the system any more, and if something fails in the middle of the upgrade then you could be in trouble.
So, what to do? I considered keeping a copy of the old
sshd
around temporarily and patching the newsshd
to re-execute it if it’s being run to handle an incoming connection, but that turned out to fail in my first test: dependencies are normally only checked when configuring a package, so it’s possible to unpackopenssh-server
before unpacking a newerlibc6
that it depends on, at which point you can’t execute the newsshd
at all. (That also means that the approach of restarting the service at unpack time instead of configure time is a non-starter.) We needed a different idea.dpkg
, the core Debian package manager, has a specialized facility called “diversions”: you can tell it that when it’s unpacking a particular file it should put it somewhere else instead. This is normally used by administrators when they want to install a locally-modified version of a particular file at their own risk, or by packages that knowingly override a file normally provided by some other package. However, in this case it turns out to be useful foropenssh-server
to temporarily divert one of its own files! When upgrading from before 9.8, it now diverts/usr/sbin/sshd
to/usr/sbin/sshd.session-split
before the new version is unpacked, then removes the diversion and moves the new file into place once it’s ready to restart the service; this reduces the period when incoming connections fail to a minimum. (We actually have to pretend that the diversion is being performed on behalf of a slightly different package since we’re usingdpkg-divert
in a strange way here, but it all works.) -
Most OpenSSH processes, including
sshd
, check for a compatible version of the OpenSSL library when they start up. This check used to be very picky, among other things requiring both the major and minor number to match. OpenSSL 3 has a better versioning policy, and so OpenSSH 9.4p1 relaxed this check.Unfortunately, bookworm shipped with OpenSSH 9.2p1, which means that as soon as you unpack the new
libssl3
during an upgrade (actuallylibssl3t64
due to the 64-bittime_t
transition),sshd
stops working. This couldn’t be fixed by a change in trixie; we needed to change bookworm in advance of the upgrade so that it would tolerate newer versions of OpenSSL. And time was tight if we wanted to maximize the chance that people would apply that stable update before upgrading to trixie; there isn’t going to be another point release of Debian 12 before the release of Debian 13.Fortunately, there’s a
stable-updates
mechanism for exactly this sort of thing, and the stable release managers kindly accepted my proposal to fix this there.
The net result is that if you apply updates to bookworm (including
stable-updates
/ bookworm-updates
, which is enabled by default) before
starting the upgrade to trixie, everything should be fine. Many thanks to
Manfred for reporting this with just enough time to spare that we were able
to fix it before Debian 13 is released in a few days!
debmirror
I did my twice-yearly refresh of debmirror’s mirror_size
documentation,
and applied a patch from Christoph Goehre
to improve mirroring of installer files.
madison-lite
I proposed renaming this project along
with the rmadison
tool in devscripts
, although I’m not yet sure what a
good replacement name would be.
Python team
I upgraded python-expandvars, python-typing-extensions (in experimental), and webtest to new upstream versions.
I backported fixes for some security vulnerabilities to unstable:
- python-urllib3: CVE-2025-50181, CVE-2025-50182
I fixed or helped to fix a number of release-critical bugs:
- bitstruct: autopkgtest regression: invalid command ‘test’
- django-pipeline: autopkgtest failure (contributed supporting fix upstream)
- pnopaste: Fails to install with debconf noninteractive frontend (suggested possible patch)
- py3dns: autopkgtest regression: ‘96.7.128.186’ != ‘93.184.215.14’ (contributed upstream)
- python-marshmallow-dataclass: autopkgtest depends on removed package python-marshmallow-enum
- python-pkgconfig: autopkgtest regression: list index out of range
- python-txrequests: autopkgtest regression: twisted.trial.unittest.FailTest: 200 != 404
I fixed some other bugs, mostly Severity: important
:
- afew: Unable to remove tags (reviewed and merged MR)
- ipy: FTBFS with the nocheck build profile
- paramiko: Does not correctly handle OpenSSH 10 version
- python-django-storages: FTBFS with the nocheck build profile
- python-icalendar: Depends on a transitional package (and follow-up fixes for missing build-dependencies in python-recurring-ical-events, python-x-wr-timezone, and todoman)
- python-libais: Stop calling python3 setup.py test (contributed supporting fix upstream)
I reinstated python3-mastodon’s build-dependency on and recommendation of python3-blurhash, now that the latter has been fixed to use the correct upstream source.