Enrico
+writes about creating pipelines with Python's subprocess
+module, and notes that you need to take care to close stdout in non-final
+subprocesses so that subprocesses get SIGPIPE correctly. This
+is correct as far as it goes (and true in any language, although there's a
+Python bug report requesting
+that subprocess be able to do this itself), but there's an
+additional gotcha with Python that you missed.
Python ignores SIGPIPE on startup, because it prefers to
+check every write and raise an IOError exception rather than
+taking the signal. This is all well and good for Python itself, but most
+Unix subprocesses don't expect to work this way. Thus, when you are creating
+subprocesses from Python, it is very important to set
+SIGPIPE back to the default action. Before I realised this was
+necessary, I wrote code that caused serious data loss due to a child process
+carrying on out of control after its parent process died!
+import signal +import subprocess + +def subprocess_setup(): + # Python installs a SIGPIPE handler by default. This is usually not what + # non-Python subprocesses expect. + signal.signal(signal.SIGPIPE, signal.SIG_DFL) + +subprocess.Popen(command, preexec_fn=subprocess_setup) ++ +
I filed a patch a while
+back to add a restore_sigpipe option to
+subprocess.Popen, which would take care of this. As I say in
+that bug report, in a future release I think this ought to be made the
+default, as it's very easy to get things dangerously wrong right now.