one problem we run into is that reading a large mailbox just takes a
while even if the directory is cached in memory, because of the small
buffer size used by the readdir() call.
I've been wondering if a long-lived MH server would be helpful. `scan
+inbox last:42' would connect to it, learn the message numbers in inbox,
and ask for the fields it wants for the format string for the last 42.
The server would likely have this information to hand, and if not, go to
work, readdir(), opening files, parsing, decoding, etc.
If the platform supports, non-POSIX, inotify(7) or similar then it
learns of the ground shifting under its feet and invalidates,
pessimistically, its `cache'. That's assuming ~/mail, like now, but it
seems it could be a veneer in front of IMAP too.
Lazy readdir()ing is an easier, incremental, step though.