So, the most recent discussion about IMAP started me thinking ... how
bad would a connection for every command really be?
I wrote a command called "imaptest", and I put it into the nmh distribution.
It will be build by default, but not installed. It needs a LOT of cleaning
up, but it is sufficient to gather some data if you are conversant with
the IMAP specification.
I started poking around a bit; the Cyrus-SASL archives are available via
anonymous IMAP, so this lets you access them:
imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
However it should work with any IMAP server that supports the SASL mechanisms
we support (it also works with TLS, both STARTTLS and "initial" tls). If
you wish to use it, you probably want the -snoop flag so you can see
what is going on (we it does no verification of the IMAP syntax, just tries
to match up responses).
Just to give you an idea, here's what we're talking about in terms of
response time. This is basically doing a 'scan last:200' on the
Cyrus-SASL archives:
% imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
-timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE
BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)'
Connect time: 0.018900 sec
Command execution time:: 0.018105 sec
Authentication time: 0.037288 sec
Command execution time:: 0.068890 sec
Command execution time:: 0.056778 sec
Total command execution time: 0.125706 sec
Command execution time:: 0.018582 sec
Total elapsed time: 0.200631 sec
There is some variance here. It seems that the DNS TTL for
'cyrus.andrew.cmu.edu' is set to 1 (!) second, so if it has to do a DNS query
it takes noticably longer:
% imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
-timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE
BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)'
Connect time: 0.433292 sec
Command execution time:: 0.017532 sec
Authentication time: 0.035130 sec
Command execution time:: 0.061797 sec
Command execution time:: 0.056097 sec
Total command execution time: 0.117972 sec
Command execution time:: 0.018482 sec
Total elapsed time: 0.605052 sec
If you add encryption with -tls, it's actually not so bad (cached DNS this
time):
% imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
-timestamp +archive.cyrus-sasl 'FETCH 10226:10426 (FLAGS RFC822.SIZE
BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' -tls
Connect time: 0.019291 sec
Command execution time:: 0.018414 sec
Command execution time:: 0.017851 sec
Authentication time: 0.113366 sec
Command execution time:: 0.054519 sec
Command execution time:: 0.057359 sec
Total command execution time: 0.111911 sec
Command execution time:: 0.018694 sec
Total elapsed time: 0.263568 sec
And if you decide you want the WHOLE mailbox, it's not really that bad either:
% imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
-timestamp +archive.cyrus-sasl 'FETCH 1:10426 (FLAGS RFC822.SIZE
BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)'
Connect time: 0.019226 sec
Command execution time:: 0.017619 sec
Authentication time: 0.035386 sec
Command execution time:: 0.053753 sec
Command execution time:: 0.456180 sec
Total command execution time: 0.510001 sec
Command execution time:: 0.018752 sec
Total elapsed time: 0.583572 sec
Even with TLS, not that bad:
% imaptest -host cyrus.andrew.cmu.edu -sasl -saslmech anonymous -user nobody
-timestamp +archive.cyrus-sasl 'FETCH 1:10426 (FLAGS RFC822.SIZE
BODY.PEEK[HEADER.FIELDS (FROM TO SUBJECT DATE)] BODY.PEEK[TEXT]<0.80>)' -tls
Connect time: 0.019436 sec
Command execution time:: 0.017595 sec
Command execution time:: 0.017777 sec
Authentication time: 0.111909 sec
Command execution time:: 0.068417 sec
Command execution time:: 0.437721 sec
Total command execution time: 0.506181 sec
Command execution time:: 0.018700 sec
Total elapsed time: 0.656586 sec
Some things jump out at me after playing with this a bit. A FETCH on
a single message is around 18/20ms. If you just do a simple client,
with a single FETCH and response for every message, that would be over
200 seconds to fetch the necessary information to do a scan(1) on all
10426 messages in that mailbox. But done as one command it's a half
second! That suggests to me that really, a new connection for every
mailbox actually might be fine.
Now, we'd have to TOTALLY rethink the internal API to make this work;
the key there is you need to figure out which messages you were interested
in before you started talking to the IMAP server, to minimize the number
of network calls. But I think that's achievable. And of course there's
the mapping between MH message numbers and IMAP messages, but I think that's
not that hard; obviously you'd use UIDs, and I think making it work is
relatively straightforward
If anyone else wants to play around with imaptest, please try it out!
I am sure there are bugs (and features to add). The code is kind of
a mess, sadly; improvements welcome in that area as well.
--Ken
_______________________________________________
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers