[Top] [All Lists]

Re: We need an IETF BCP for GREY LISTING

2011-10-17 11:40:09

Hector wrote:

I think someone should take up the effort to begin/draft an GreyList BCP for systems to follow.



It seems there is a majority consensus that a IETF document for Greylisting would be a worthy effort.

I do not have the time, so I am hoping some other interested party can write it. But if not, I may consider writing this document. I can certainly help co-author it. If someone wishes to assist, please write me off list.

I sent a note to Evan Harris, AFAIK, the original author of Greylisting asking for his interest in authoring or contributing to the development official IETF GreyListing document. I seem to recall long ago, a comment he made that he had avoided the IETF knowing there would be major resistance with some an "odd idea" of using an initial rejection as a form to filter a certain form of "single shot" random spammers. I agreed 100% the IETF resistance would of been there, but that was back then in 2003. Not today. Attitudes have changed about GL and now you see GL implementation widely adopted, even among the early IETF critics.

Maybe to start such a document, we can gather the following regarding the basic GL framework I think many use or part of the following:

  - Recorded Information
  - Class C support
  - Blocking Time
  - White Listed Time
  - Rejection Point
  - Rejection response

For example, for our GL implementation:

  - Recorded Information

      Connecting IP, MAIL FROM, RCPT TO

  - Class C support

    We record the IP A.B.C.0 parts only so that additional
    attempts from different A.B.C.0 are also blocked.

  - Blocking Time

     55 seconds

    This was learned. First, I wanted the shorted possible to
    not hurt timely deliveries with the initial rejection, but
    many spammers do try almost immediately and 55 was a good
    delay I found to capture most of these abusers.

  - Sender White Listed Time

     96 hours

    For us, if set at 0, then the accepted sender after
    being greylisted is permanently greylisted.  Otherwise,
    the sender is just white listed for 4 days.

  - Rejection Point

    We reject after the DATA is captured and issue an
    EOD 451 response when the transaction is greylisted.

  - Rejection Response

    Current, we have stock response of:

      451 Please try again later (TempFail)

I wish to explore adding a retry=hint concept for the response, so maybe that is something we can work on to come up with a formal format/hint value.

Additional features or BCP discussions items:

It will be good to gather GL implementation field experiences out there among the different SMTP developers and operators. I have a few points here:

- Auto White Listing

I would suggest that "auto-whitelist" concepts be part of the BCP recommendations.

For our system, we use an auto white list concept where if we send mail to an email address and the address is added to an grey-list-autowhite list file that the greylister component will check when a new anonymous sender comes in. This was a major god-send to not only accept the GL concept, but to deal with customer support issues when we need them to send some logs or data to us. So during a support call, we will send them an email called a "Permission List" that will auto-whitelist the customer and have them reply back to the email to send the support data. Otherwise, we have to wait. Although we have a short 55 seconds, we don't know what is the reschedule time for customer's or their company's MTA setup and that can be in the minutes or hour.

- Rejection point

We use DATA rejection point only because originally the "confident level" was still low with the then new "odd" concept of initial rejections with an expectation of retrying before accepting the transaction. I recall operators setting at the greylist monitors where they will see a rejection entry from a legit sender with non-spam email they can view and they sat there wondering when it will retry again. When the sender's MTA has retries that not short, that made some people paranoid so capturing the DATA alleviated the concern that the mail was lost. They can push the email if they were impatience and of course, they got a dupe eventually.

But many systems reject at RCPT or MAIL or even at the Connection level with a 421 response. These are of course, lower overhead rejections than a DATA capture and rejection. So today, I would recommend a RCPT rejection since the GL proof of concept is valid and the lost of legit mail is low or zero. I don't recall a lost of email, delays in retries, but no lost among SMTP compliant MTAs.

- Recorded Information

We use IP:FROM:TO but I believe is the common practice among GL systems. But I have read where some GL systems use the 5322.Message-ID header.

Of course, if the GL system is rejecting at the connection level, then the only recording it has is the IP and if the GL system is rejected at MAIL FROM, then it is recording IP and MAIL FROM only.

The only thing I can note here is the blur with using GL at the connection level where a 421 is used.

Some retry strategies may try the next different MX/IP immediately when they see a 421 assuming it is a Loading restriction.

However, for us, because of the increasing GL usage at the connection level and this 421 blur, like YAHOO and others, a 421 response is rescheduled rather than try the next different MX/IP, if any, for current outbound attempt.