[qmail] / RFCQMTP

Quick Mail Transfer Protocol (QMTP)
D. J. Bernstein, djb@pobox.com
19970201


1. Introduction

   The Quick Mail Transfer Protocol (QMTP) is a replacement for the
   Simple Mail Transfer Protocol (SMTP). QMTP eliminates any need for
   end-of-line scanning between hosts with the same end-of-line
   convention. It features automatic pipelining and chunking, 8-bit
   transmission, prior declaration of the message size, and efficient
   batching. It is designed to be very easy to implement.

   QMTP is supported by the qmail-qmtpd and maildir2qmtp programs in the
   qmail package.

   In this document, a string of 8-bit bytes may be written in two
   different forms: as a series of hexadecimal numbers between angle
   brackets, or as a sequence of ASCII characters between double quotes.
   For example, <68 65 6c 6c 6f 20 77 6f 72 6c 64 21> is a string of
   length 12; it is the same as the string "hello world!". Note that
   these notations are part of this document, not part of the protocol.


2. Protocol

   A QMTP client connects to a QMTP server, as discussed in section 7,
   over a reliable stream protocol allowing transmission of 8-bit bytes.

   Protocol outline: the client sends one or more packages; after each
   package, the server sends back some responses.

   The client begins by sending a package. A package contains a mail
   message, an envelope sender address, and one or more envelope
   recipient addresses. See section 4 for the format of a package.

   When the server sees the end of the package, it sends back a series
   of responses, one response for each envelope recipient address, in
   the same order as given by the client. The server is not permitted to
   change the order under any circumstances, even if two addresses are
   the same. See section 5 for the format of a response.

   The server is not permitted to send any portion of its responses to a
   package until the client has sent the final byte of the package. The
   client is permitted to close the connection before sending the final
   byte of the package; in this case, the server must throw away the
   package without attempting to deliver the message. However, the
   server must not throw away previously accepted messages.

   The client does NOT need to wait for a server response before sending
   another package. The server must NOT throw away incoming data when it
   sends a response. It is the client's responsibility to avoid
   deadlock: if it sends a package before receiving all expected server
   responses, it must continuously watch for those responses. The server
   is permitted to delay its responses if further data has already shown
   up from the client; while it is delaying responses, it must not pause
   to wait for further data for the client.

   The server is permitted to close the connection at any time, although
   high-quality servers will try to avoid doing so. Any response not
   received by the client indicates a temporary failure.

   A QMTP session should take at most 1 hour. Both sides are expected
   to close the connection after this time.


3. Messages

   In this document, an ``8-bit mail message'' means a sequence of
   lines. Each line is a string of zero or more 8-bit bytes.

   A message is called ``safe'' if none of its bytes are <0a>.

   Implementation note: Here is the intended interpretation of text
   files as messages under some current operating systems. Under DOS, a
   message is stored on disk as

      first line, <0d 0a>, second line, <0d 0a> ... <0d 0a>, last line.

   Under UNIX, a message is stored on disk as

      first line, <0a>, second line, <0a> ... <0a>, last line.

   Notice that both of these encodings are reversible for safe messages.

   In practice, it is very common for the last line to be empty. Many
   existing utilities refer to the last line as a ``partial line'' and
   ignore it whether or not it is empty.


4. Packages

   A package is the concatenation of three strings:

      first, an encoded 8-bit mail message;
      second, an encoded envelope sender address;
      third, an encoded series of encoded envelope recipient addresses.

   Each envelope address is a string of 8-bit bytes. The interpretation
   of addresses depends on the environment in which QMTP is used and is
   outside the scope of this document. Each address is encoded as a
   netstring, as discussed in section 6. The series of encoded recipient
   addresses is in turn encoded as a netstring.

   A message is encoded as a string of 8-bit bytes in one of two ways:

      Encoding #1 is <0d>, the first line, <0d 0a>, the second line,
      <0d 0a>, the third line, ..., <0d 0a>, the last line.

      Encoding #2 is <0a>, the first line, <0a>, the second line, <0a>,
      the third line, ..., <0a>, the last line.

   This string of 8-bit bytes is in turn encoded as a netstring, as
   discussed in section 6.

   Every server must be prepared to handle encoding #1 and encoding #2.
   A server must not reject a message merely because of its encoding.

   Implementation note: The intent of encoding #1 and encoding #2 is to
   allow very straightforward handling of text files under DOS and UNIX
   respectively. The programmer can print <0d> or <0a> and then simply
   copy the file.


5. Responses

   Each response is a nonempty string of 8-bit bytes, encoded as a
   netstring. The first byte of the string is one of the following:

      "K" The message has been accepted for delivery to this envelope
          recipient. This is morally equivalent to the 250 response to
          DATA in SMTP; it is subject to the reliability requirements
          of RFC 1123, section 5.3.3.

      "Z" Temporary failure. The client should try again later.

      "D" Permanent failure.

   The remaining bytes are a description of what happened. It is
   expected that the description, when interpreted as UTF-2 characters,
   (1) will be human-readable, (2) will not repeat the envelope
   recipient address, and (3) will not include formatting characters
   other than <20>. However, these expectations are not requirements,
   and the client should be ready for arbitrary bytes from the server.

   Descriptions beginning with <20> are reserved for future extensions.
   In descriptions not beginning with <20>, the character "#" must not
   appear except in HCMSSC codes.

   A server must NOT accept a safe message unless it can store the
   message without corruption. More precisely: if the encoded message
   sent by the client matches the encoding of some safe message M, then
   acceptance means that the server is accepting responsibility to
   deliver M to the envelope recipient. (There is at most one
   possibility for M, since encodings are reversible on safe messages.)
   Deletion of nulls is NOT permissible; a server that deletes nulls
   must reject any message containing nulls. Folding of long lines and
   high-bit stripping are also NOT permissible.

   Servers are permitted to change unsafe messages.


6. Netstrings

   Any string of 8-bit bytes may be encoded as [len]":"[string]",".
   Here [string] is the string and [len] is a nonempty sequence of ASCII
   digits giving the length of [string] in decimal. The ASCII digits are
   <30> for 0, <31> for 1, and so on up through <39> for 9. Extra zeros
   at the front of [len] are prohibited: [len] begins with <30> exactly
   when [string] is empty.

   For example, the string "hello world!" is encoded as <31 32 3a 68
   65 6c 6c 6f 20 77 6f 72 6c 64 21 2c>, i.e., "12:hello world!,". The
   empty string is encoded as "0:,".

   [len]":"[string]"," is called a netstring. [string] is called the
   interpretation of the netstring.


7. Encapsulation

   QMTP may be used on top of TCP. A QMTP-over-TCP server listens for
   TCP connections on port 209.


8. Examples

   A client opens a connection and sends the concatenation of the
   following strings:

      "246:" <0a>
         "Received: (qmail-queue invoked by uid 0);"
         " 29 Jul 1996 09:36:40 -0000" <0a>
         "Date: 29 Jul 1996 11:35:35 -0000" <0a>
         "Message-ID: <19960729113535.375.qmail@heaven.af.mil>" <0a>
         "From: God@heaven.af.mil" <0a>
         "To: djb@silverton.berkeley.edu (D. J. Bernstein)" <0a>
         <0a>
         "This is a test." <0a> ","
      "24:" "God-DSN-37@heaven.af.mil" ","
      "30:" "26:djb@silverton.berkeley.edu," ","

      "356:" <0d>
         "From: MAILER-DAEMON@heaven.af.mil" <0d 0a>
         "To:" <0d 0a>
         "   Hate." <22> "The Quoting" <22>
	 "@SILVERTON.berkeley.edu," <0d 0a>
         "   " <22> "\\Backslashes!" <22>
	 "@silverton.BERKELEY.edu" <0d 0a>
         <0d 0a>
         "The recipient addresses here could"
	 " have been encoded in SMTP as" <0d 0a>
         "" <0d 0a>
         "   RCPT TO:<Hate.The\ Quoting@silverton.berkeley.EDU>" <0d 0a>
         "   RCPT TO:<\\Backslashes!@silverton.berkeley.edu>" <0d 0a>
         <0d 0a>
         "This ends with a partial last line, right here" ","
      "0:" ","
      "83:" "39:Hate.The Quoting@silverton.berkeley.edu,"
         "36:\Backslashes!@silverton.berkeley.EDU," ","
      
   The server sends the following response, indicating acceptance:

      "21:Kok 838640135 qp 1390,"
      "21:Kok 838640135 qp 1391,"
      "21:Kok 838640135 qp 1391,"

   The client closes the connection.
Commit	Line	Data
2117e02e MW	1	Quick Mail Transfer Protocol (QMTP)
	2	D. J. Bernstein, djb@pobox.com
	3	19970201
	4
	5
	6	1. Introduction
	7
	8	The Quick Mail Transfer Protocol (QMTP) is a replacement for the
	9	Simple Mail Transfer Protocol (SMTP). QMTP eliminates any need for
	10	end-of-line scanning between hosts with the same end-of-line
	11	convention. It features automatic pipelining and chunking, 8-bit
	12	transmission, prior declaration of the message size, and efficient
	13	batching. It is designed to be very easy to implement.
	14
	15	QMTP is supported by the qmail-qmtpd and maildir2qmtp programs in the
	16	qmail package.
	17
	18	In this document, a string of 8-bit bytes may be written in two
	19	different forms: as a series of hexadecimal numbers between angle
	20	brackets, or as a sequence of ASCII characters between double quotes.
	21	For example, <68 65 6c 6c 6f 20 77 6f 72 6c 64 21> is a string of
	22	length 12; it is the same as the string "hello world!". Note that
	23	these notations are part of this document, not part of the protocol.
	24
	25
	26	2. Protocol
	27
	28	A QMTP client connects to a QMTP server, as discussed in section 7,
	29	over a reliable stream protocol allowing transmission of 8-bit bytes.
	30
	31	Protocol outline: the client sends one or more packages; after each
	32	package, the server sends back some responses.
	33
	34	The client begins by sending a package. A package contains a mail
	35	message, an envelope sender address, and one or more envelope
	36	recipient addresses. See section 4 for the format of a package.
	37
	38	When the server sees the end of the package, it sends back a series
	39	of responses, one response for each envelope recipient address, in
	40	the same order as given by the client. The server is not permitted to
	41	change the order under any circumstances, even if two addresses are
	42	the same. See section 5 for the format of a response.
	43
	44	The server is not permitted to send any portion of its responses to a
	45	package until the client has sent the final byte of the package. The
	46	client is permitted to close the connection before sending the final
	47	byte of the package; in this case, the server must throw away the
	48	package without attempting to deliver the message. However, the
	49	server must not throw away previously accepted messages.
	50
	51	The client does NOT need to wait for a server response before sending
	52	another package. The server must NOT throw away incoming data when it
	53	sends a response. It is the client's responsibility to avoid
	54	deadlock: if it sends a package before receiving all expected server
	55	responses, it must continuously watch for those responses. The server
	56	is permitted to delay its responses if further data has already shown
	57	up from the client; while it is delaying responses, it must not pause
	58	to wait for further data for the client.
	59
	60	The server is permitted to close the connection at any time, although
	61	high-quality servers will try to avoid doing so. Any response not
	62	received by the client indicates a temporary failure.
	63
	64	A QMTP session should take at most 1 hour. Both sides are expected
65	to close the connection after this time.
66
67
68	3. Messages
69
70	In this document, an ``8-bit mail message'' means a sequence of
71	lines. Each line is a string of zero or more 8-bit bytes.
72
73	A message is called ``safe'' if none of its bytes are <0a>.
74
75	Implementation note: Here is the intended interpretation of text
76	files as messages under some current operating systems. Under DOS, a
77	message is stored on disk as
78
79	first line, <0d 0a>, second line, <0d 0a> ... <0d 0a>, last line.
80
81	Under UNIX, a message is stored on disk as
82
83	first line, <0a>, second line, <0a> ... <0a>, last line.
84
85	Notice that both of these encodings are reversible for safe messages.
86
87	In practice, it is very common for the last line to be empty. Many
88	existing utilities refer to the last line as a ``partial line'' and
89	ignore it whether or not it is empty.
90
91
92	4. Packages
93
94	A package is the concatenation of three strings:
95
96	first, an encoded 8-bit mail message;
97	second, an encoded envelope sender address;
98	third, an encoded series of encoded envelope recipient addresses.
99
100	Each envelope address is a string of 8-bit bytes. The interpretation
101	of addresses depends on the environment in which QMTP is used and is
102	outside the scope of this document. Each address is encoded as a
103	netstring, as discussed in section 6. The series of encoded recipient
104	addresses is in turn encoded as a netstring.
105
106	A message is encoded as a string of 8-bit bytes in one of two ways:
107
108	Encoding #1 is <0d>, the first line, <0d 0a>, the second line,
109	<0d 0a>, the third line, ..., <0d 0a>, the last line.
110
111	Encoding #2 is <0a>, the first line, <0a>, the second line, <0a>,
112	the third line, ..., <0a>, the last line.
113
114	This string of 8-bit bytes is in turn encoded as a netstring, as
115	discussed in section 6.
116
117	Every server must be prepared to handle encoding #1 and encoding #2.
118	A server must not reject a message merely because of its encoding.
119
120	Implementation note: The intent of encoding #1 and encoding #2 is to
121	allow very straightforward handling of text files under DOS and UNIX
122	respectively. The programmer can print <0d> or <0a> and then simply
123	copy the file.
124
125
126	5. Responses
127
128	Each response is a nonempty string of 8-bit bytes, encoded as a
129	netstring. The first byte of the string is one of the following:
130
131	"K" The message has been accepted for delivery to this envelope
132	recipient. This is morally equivalent to the 250 response to
133	DATA in SMTP; it is subject to the reliability requirements
134	of RFC 1123, section 5.3.3.
135
136	"Z" Temporary failure. The client should try again later.
137
138	"D" Permanent failure.
139
140	The remaining bytes are a description of what happened. It is
141	expected that the description, when interpreted as UTF-2 characters,
142	(1) will be human-readable, (2) will not repeat the envelope
143	recipient address, and (3) will not include formatting characters
144	other than <20>. However, these expectations are not requirements,
145	and the client should be ready for arbitrary bytes from the server.
146
147	Descriptions beginning with <20> are reserved for future extensions.
148	In descriptions not beginning with <20>, the character "#" must not
149	appear except in HCMSSC codes.
150
151	A server must NOT accept a safe message unless it can store the
152	message without corruption. More precisely: if the encoded message
153	sent by the client matches the encoding of some safe message M, then
154	acceptance means that the server is accepting responsibility to
155	deliver M to the envelope recipient. (There is at most one
156	possibility for M, since encodings are reversible on safe messages.)
157	Deletion of nulls is NOT permissible; a server that deletes nulls
158	must reject any message containing nulls. Folding of long lines and
159	high-bit stripping are also NOT permissible.
160
161	Servers are permitted to change unsafe messages.
162
163
164	6. Netstrings
165
166	Any string of 8-bit bytes may be encoded as [len]":"[string]",".
167	Here [string] is the string and [len] is a nonempty sequence of ASCII
168	digits giving the length of [string] in decimal. The ASCII digits are
169	<30> for 0, <31> for 1, and so on up through <39> for 9. Extra zeros
170	at the front of [len] are prohibited: [len] begins with <30> exactly
171	when [string] is empty.
172
173	For example, the string "hello world!" is encoded as <31 32 3a 68
174	65 6c 6c 6f 20 77 6f 72 6c 64 21 2c>, i.e., "12:hello world!,". The
175	empty string is encoded as "0:,".
176
177	[len]":"[string]"," is called a netstring. [string] is called the
178	interpretation of the netstring.
179
180
181	7. Encapsulation
182
183	QMTP may be used on top of TCP. A QMTP-over-TCP server listens for
184	TCP connections on port 209.
185
186
187	8. Examples
188
189	A client opens a connection and sends the concatenation of the
190	following strings:
191
192	"246:" <0a>
193	"Received: (qmail-queue invoked by uid 0);"
194	" 29 Jul 1996 09:36:40 -0000" <0a>
195	"Date: 29 Jul 1996 11:35:35 -0000" <0a>
196	"Message-ID: <19960729113535.375.qmail@heaven.af.mil>" <0a>
197	"From: God@heaven.af.mil" <0a>
198	"To: djb@silverton.berkeley.edu (D. J. Bernstein)" <0a>
199	<0a>
200	"This is a test." <0a> ","
201	"24:" "God-DSN-37@heaven.af.mil" ","
202	"30:" "26:djb@silverton.berkeley.edu," ","
203
204	"356:" <0d>
205	"From: MAILER-DAEMON@heaven.af.mil" <0d 0a>
206	"To:" <0d 0a>
207	" Hate." <22> "The Quoting" <22>
208	"@SILVERTON.berkeley.edu," <0d 0a>
209	" " <22> "\\Backslashes!" <22>
210	"@silverton.BERKELEY.edu" <0d 0a>
211	<0d 0a>
212	"The recipient addresses here could"
213	" have been encoded in SMTP as" <0d 0a>
214	"" <0d 0a>
215	" RCPT TO:<Hate.The\ Quoting@silverton.berkeley.EDU>" <0d 0a>
216	" RCPT TO:<\\Backslashes!@silverton.berkeley.edu>" <0d 0a>
217	<0d 0a>
218	"This ends with a partial last line, right here" ","
219	"0:" ","
220	"83:" "39:Hate.The Quoting@silverton.berkeley.edu,"
221	"36:\Backslashes!@silverton.berkeley.EDU," ","
222
223	The server sends the following response, indicating acceptance:
224
225	"21:Kok 838640135 qp 1390,"
226	"21:Kok 838640135 qp 1391,"
227	"21:Kok 838640135 qp 1391,"
228
229	The client closes the connection.