[Top] [All Lists]

Re: Processing after the end of DATA

2010-08-10 00:32:15

Tony Hansen wrote:

A big timeout doesn't mean that one can takes lots of time, but it might explain where someone got the idea that they *could* take lots of time.

    Tony Hansen

And there is nothing wrong with that, after all, we encourage people to explore the unknowns, the least untested, learn and share experiences.

Remember the KEEP-ALIVE discussions?

Well, there was experiences here. Much exploration was done and you helped decide what I was proposing in regards to using continuation line with reply codes different from the final line as an idea keep clients alive for a world that would soon doing more DATA level processing. Once we found there are legacy software which will not work here, we codified RFC 5321 to explicitly state all reply codes in continuation line MUST be the same as the final reply code.

But that did not stop DATA processing efforts. In the end, I can safely say a 3-5 minute DATA processing time seems to be the maximum. Of course, it should be as fast as possible, but that would be an operator setup situation. Its may not be one script but a series of organized scripts.

The best SMTP can do here is add SHIM management logic with watch dog logic, hot port drop logic, etc. We did add a GLOBAL TIMEOUT for all scripts as well to keep it within some level of SMTP expectations.

# v2.0
# Maximum Scanning Time (secs) allowed. This
# time is to help prevent SMTP timeouts in the DATA
# state.  10 mins is official wait time for DATA
# termination timeout. However, 5 mins is pretty
# common (unfortunately) for many SMTP clients. The
# default value is alittle less than 5 minutes
# (4 mins, 30 secs).  What this means is that
# all the WCX filters in the [hooks] sections MUST
# complete within this time.  To disable, set it
# too zero, however, keep in mind, SMTP clients will
# timeout and disconnect if the filters are taking
# too long. This can cause dupes and unnecessary
# retries by the client.

;GlobalTimeout = 270    # use this for 5 mins timeouts (default)
;GlobalTimeout = 570    # use this for 10 mins timeouts
;GlobalTimeout = 60     # TESTING ONLY

# v2.0
# GlobalCarrierCheck allows for the Wildcat! WCX processor
# to globally check for connection drops for all
# wcx run by the loader.  When FALSE, then each WCX
# is responsible to perform direct Carrier() or
# sfCheckCarrier() calls if it wants to gracefully
# check for drops and exit gracefully.
# GlobalCarrierCheck is FALSE by default to allow each
# WCX to take control.  It is possible that even when
# a connection a drop, a WCX made want to get its hand
# on the email file to do something with it.  In these
# cases, you should have these WCX scripts in the
# [Event:Drop] section.

;GlobalCarrierCheck = false

So when you guys catch up, those are some of the things to consider in SMTP DATA shims. It is workable concept. We should not be afraid to do DATA shims by recommending we should use 1988 RFC 1074 guidelines molded in an era where hardware, OS and software data processing efficiency was very low compared to today. In the past, it was not very efficient to do SQL database queries or move large files around networks during a DATA state. This is pretty much a non-issue today.


Hector Santos