procmail
[Top] [All Lists]

procmail log file corruption

2004-07-14 21:21:52
Hello,

My .procmailrc handles over 10,000 messages a day for my domain.
The email processing at my ISP is distributed among at least 2
different machines (Red Hat 7.2, procmail 3.21) and the home
and mail directories are NFS mounted.

I log information so I can gather statistics about spam and how
my spam filters are working.  With a decent amount of email, the
log entries would get interleaved and overwritten if I just used
LOG to write the entries.  Instead I followed some advice from
the archives of this mailing list and collected all the info and
wrote to the log with a single echo using TRAP.

This worked fairly well in the past, but the interleaving and
overwriting got worse as the volume of email increased.  I finally
resorted to locking the log file.  Here is my TRAP:

TRAP='abstract=`formail -l $LASTFOLDER`; date=`date`; lockfile -1 -r 60 -l 1024
$LOGFILE$LOCKEXT; echo "$abstract
  Filter: $filter$test
  Delivered: $date"; rm -f $LOGFILE$LOCKEXT'

(Yes, I use bare newlines here.)

Here is a typical log entry:

From egxmt47rpqfk(_at_)radix(_dot_)net  Tue Jul 13 00:05:27 2004
 Subject: NORT0N 2004 $15. XP PR0 $5O; AD0BE PH0TOSH0P $8O; 0FFICE XP $100; 0FF
  Folder: /dev/null                                                        4940
  Filter: bad-address
  Delivered: Tue Jul 13 00:05:28 PDT 2004

I've been using this for about a month and the interleaving has
gone away, but some overwriting still occurs.  Shouldn't lockfile
prevent log entries from overwriting each other?

Here is an example where the beginning got chopped off somehow:

iz.net  Tue Jul 13 12:44:14 2004
 Subject: re:emplace,JVGI - Huge news and st0ck will explode on Tuesday
  Folder: /dev/null                                                        6286
  Filter: bad-address
  Delivered: Tue Jul 13 12:44:15 PDT 2004

Here is another example where almost all of an entry is gone:

From rochel_wahl(_at_)surfer(_dot_)at  Mon Jul 12 06:20:47 2004
  Folder: /dev/null                                                        4536
  Filter: bad-address
  Delivered: Mon Jul 12 06:20:47 PDT 2004
livered: Mon Jul 12 06:20:47 PDT 2004
From njedkhrcgyol(_at_)langoo(_dot_)com  Mon Jul 12 06:20:49 2004
 Subject: dont ignore your girlfriend!
  Folder: /dev/null                                                        3518
  Filter: bad-address
  Delivered: Mon Jul 12 06:20:49 PDT 2004

Can anyone please explain what's going on?  If there was a problem
with lockfile, I would expect to see mailboxes getting corrupted too
but the mailboxes are fine.

Any help is greatly appreciated,
Lloyd Lee-Lim


____________________________________________________________
procmail mailing list   Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail

<Prev in Thread] Current Thread [Next in Thread>
  • procmail log file corruption, Lloyd Lee-Lim <=