reservoir
reservoir – delay stage in a pipeline
reservoir [ -o filename | -O filename ]
reservoir's function is to read from its standard input until it sees end-of-file, then to write everything it has seen to its standard output.
It behaves exactly like cat with no arguments, except that it writes none of its outgoing data until all of its input has arrived.
-O filename
reservoir detects end of file on its input.
-o filename
-O, but with one special case: if there is no output at all to be written, reservoir will not open the output file for writing at all. Hence, if the process which is supposed to generate the output completely fails to run, filename will not be overwritten.
-a filename
-o, but this time, the output is appended to the specified file instead of overwriting it.
In this mode, if reservoir has no data to write, it will not even attempt to open the output file for writing, so that its timestamp is not gratuitously updated.
If you have a program which filters its input in some way (for example, a base-64 decoder, or a tr(1) command performing rot13), and you wish to copy a small amount of data into that program using a terminal emulator's paste function, it can be inconvenient to have the output interspersed with the echoed input so that you cannot select and copy the output as a whole.
For example:
$ tr a-zA-Z n-za-mN-ZA-M
Hello, world.
Uryyb, jbeyq.
This is a test.
Guvf vf n grfg.
If your terminal emulator pastes the text line by line, then to copy the transformed output requires you to separately select each line of the output. If the terminal pastes in larger chunks, you may not see the problem quite so quickly, but it will still appear eventually.
You can solve this using reservoir:
$ tr a-zA-Z n-za-mN-ZA-M | reservoir
Hello, world.
This is a test.
(now the user presses ^D)
Uryyb, jbeyq.
Guvf vf n grfg.
A common reason why you might want to buffer data in a pipeline is in order to transform a file in place. For example, you cannot write
$ tr a-zA-Z n-za-mN-ZA-M < temp.txt > temp.txt
because the output redirection will destroy the contents of the file before its original contents can be read. reservoir can help, because it does not begin writing output until after the input has all been read.
You still cannot use output redirection, because the presence of the > operator on your command line will cause the output file to be truncated to zero length before running reservoir, so there is nothing reservoir can do about this. Instead, you can use the -o option provided by reservoir:
$ tr a-zA-Z n-za-mN-ZA-M < temp.txt | reservoir -o temp.txt
Now reservoir will not open temp.txt for output until after the rest of the pipeline has finished reading data from it.
(This is not a reliable means of editing files in place. If something goes wrong half way through writing the output, part of your data will be lost, although the default behaviour of -o will at least avoid overwriting the file if something goes wrong before the output begins to be written. Also, the file is not replaced atomically. This method is very convenient in non-critical situations, such as when the target file is backed up in source control, but is not recommended for critical or automated use.)
Another use for -o is for requesting a list of files using find(1) or ls(1), without the output file appearing in the list:
$ find . -type f | reservoir -o filelist
reservoir is free software, distributed under the MIT licence. Type reservoir --licence to see the full licence text.