In my last two articles My first Java Application (apache commons cli) #1 and Server / Thread (threading & connection handling) #2, I’ve started to write a Policy-Daemon for Postfix. In this article I’m stumbling through postfix documentation and the RFC to create a flowchart and a state diagram for my before-queue-filter. Hence I call that „pre“-work.
The RFC to use is RFC 5321. I’ll quote a lot of the RFC here, the source used for my quotes can be found here: RFC 5321. The source for the Postfix stuff can be found here: SMTPD_PROXY_README and here: XFORWARD_README.
Starting on page 17 the RFC states that an SMTP session is initiated when a client opens a connection. The server responds with an opening message (SMTP Code 220). A server might reject a mail session initially (SMTP Code 554) and responds to any further commands with „bad sequence of commands“ (SMTP Code 503). The server still _needs_ to wait for the QUIT command by the client (which is obviously not answered with SMTP Code 503). Since lyraPolicy is communicating directly with Postfix and no commands have been sent yet, it does not make much sense to sent an initial 554 – If we would get our connections directly from a client, one might implement a client-ip blacklist and hence send the initial 554. Since this is not possible here the above is useless for my policy daemon.
It is stated in the RFC that older SMTP systems might initiate their session using HELO instead of EHLO and that the server should handle this accordingly. Because we talk to Postfix and Postfix is sending EHLO we don’t care.
The interesting part, the transaction, starts now. The RFC says:
„There are three steps to SMTP mail transactions.“, „The transaction starts with a MAIL command that gives the sender identification. (In general, the MAIL command may be sent only when no mail transaction is in progress; see Section 4.1.4.) A series of one or more RCPT commands follows, giving the receiver information. Then, a DATA command initiates transfer of the mail data and is terminated by the „end of mail“ data indicator, which also confirms the transaction.“
My interpretation of this is as follows: A mail transaction always starts with the MAIL command and ends with the END OF MAIL data indicator „\r\n.\r\n“. Within a transaction (when the MAIL command was given) no further MAIL command might occur. DATA command might happen only once within a transaction and RCPT might happen more than once. Also MAIL is first, RCPT is second and DATA is the third stage (let me call that stages).
In Mail-Stage a 250 OK is given if the Mail From is valid, and an error code which indicates whether it is a permanent or temporary error if not.
In Recipient-Stage a 250 OK is given if the Rcpt To is valid, and an error code which indicates whether it is a permanent or temporary error if not. Furthermore if a RCPT command appears without a previous MAIL, a 503 error code has to be given (bad sequence of commands)
„If accepted, the SMTP server returns a 354 Intermediate reply and considers all succeeding lines up to but not including the end of mail data indicator to be the message text. When the end of text is successfully received and stored, the SMTP-receiver sends a „250 OK“ reply.“
the RFC goes on with:
„If there was no MAIL, or no RCPT, command, or all such commands were rejected, the server MAY return a „command out of sequence“ (503) or „no valid recipients“ (554) reply in response to the DATA command. If one of those replies (or any other 5yz reply) is received, the client MUST NOT send the message data; more generally, message data MUST NOT be sent unless a 354 reply is received.“
There’s an interesting aspect in the RFC which might be used against mail-harvesters, by the way:
„Using a „550 mailbox not found“ (or equivalent) reply code after the data are accepted makes it difficult or impossible for the client to determine which recipients failed.“
Hence it might make sense, to give that command after data stage instead of at recipient stage 🙂 Let’s take a short look at the postfix documentation in regards to before-queue-filters. The documentation says that:
„The before-filter Postfix SMTP server connects to the content filter, delivers one message, and disconnects. While sending mail into the content filter, Postfix speaks ESMTP but uses no command pipelining. Postfix generates its own EHLO, XFORWARD (for logging the remote client IP address instead of localhost[127.0.0.1]), DATA and QUIT commands, and forwards unmodified copies of all the MAIL FROM and RCPT TO commands that the before-filter Postfix SMTP server didn’t reject itself. Postfix sends no other SMTP commands.“
That means, the only commands we need to implement are: EHLO, XFORWARD, DATA, QUIT, MAIL FROM and RCPT TO. Because postfix sends no other SMTP commands. Let’s continue on Page 29 of the RFC:
„An SMTP server MUST NOT intentionally close the connection under normal operational circumstances (see Section 7.8) except: After receiving a QUIT command and responding with a 221 reply, After detecting the need to shut down the SMTP service and returning a 421 response code. [..] After a timeout, as specified in Section 126.96.36.199, occurs waiting for the client to send a command or data. An SMTP server that is forcibly shut down via external means SHOULD attempt to send a line containing a 421 response code to the SMTP client before exiting.“
Here are some further snippets:
- Page 32 „In any event, a client MUST issue HELO or EHLO before starting a mail transaction.“
- Page 33 „In general, the MAIL command may be sent only when no mail transaction is in progress“
- Page 34 RCPT „is used to identify an individual recipient of the mail data; multiple recipients are specified by multiple uses of this command.“
- Page 39 „The receiver MUST NOT intentionally close the transmission channel until it receives and replies to a QUIT command (even if there was an error).“
Starting on Page 43 the „order“ of the SMTP commands is explained once again. Instead of writing down and quoting what’s written in there I’ve created a diagram (using yed – in my opinion one of the best tools available to create diagrams) which shows the flow:
I think that shows the basic smtp flow pretty good. Not included are special errors due to filtering and contrary to the rfc XFORWARD is mandatory here (because without XFORWARD we won’t know the client and since Postfix is always sending XFORWARD this is mandatory for the policy daemon). Since Postfix won’t send them, RSET, NOOP, etc aren’t implemented. Also don’t worry that it is not continuing after an error happened; in fact there is nothing like abort, the only two reasons an abort might be caused in, for example postfix (apart from special reasons) is a reached timeout or too many errors.