From: Dallman Ross
Sent: Saturday, February 14, 2004 4:30 AM
[...]
The reason the failure was transparent to me all this time is as you
also implied: everything thrown out -- which is almost always spam, in
this case (it's either spam or a[nother] spamc "error") -- gets tagged
by my next recipe as SpamAssassin-identified spam. Like I said, yeesh!
I have to rewrite the whole thing. Thanks, Bart, for pulling the wool
from my eyes! Okay, w/W does not function at all under the premises I
had presumed it did for error-status results.
I ran a couple of experiments, but didn't have time to go through the test
and make sure it was done in careful way. What I think that I found out,
is that:
:0
PIPE=|cat && /bin/false
(cat copies the input, above, and /bin/false guarrantees a non-zero exit
code)
Will set PIPE to the contents of the message, in spite of the non-zero
return code.
but if either 'w' or 'W' are in place, below it will set PIPE to the *empty
string*.
So, neither 'w' nor 'W' are safe in this context. I'd appreciate it if
someone
else verified this;
# works fine (subject to LINEBUF limitation)
:0
PIPE=|cat && /bin/false
# does NOT work (sets PIPE to null)
:0 w
PIPE=|cat && /bin/false
# does NOT work (sets PIPE to null)
:0 W
PIPE=|cat && /bin/false
Note: It seems that both using a pipe and backquotes are subject to the
limitation that the amount of data being copied into the variable must
be less than LINEBUF in length. I'm uncertain as to whether the actual
restriction is (LINEBUF-1) characters (to allow for an internal null
character, perhaps, and so on). I'm also uncertain how procmail behaves
if the data being read contains control characters like a null byte
(\000), and/or control-z on Windows platforms.
The LINEBUF restriction is serious, and surprising, and might be the
cause of problems that we've had with the virus scanning example.
For example,
:0
VIRUS=|virus_scan 2>&1
let's say virus_scan above operates as a filter, so it copies the
defanged message to stdout. Further, we've redirected stderr to
stdout. Thus VIRUS will be set to (1) a copy of the defanged message,
followed by (2) a line/two of error output. Also note that according
to "man procmailrc" a trailing newline will be removed. This sort of
thing (a lot of output followed by a few lines of error message)
might be what we're seeing in the clamav example. If this were true,
then we need to realize that the output will be truncated to $LINEBUF
characters (+/- a few characters).
Again, I haven't carefully verified this; it just seemed to be what was
going on. It can be a little complicated printing the contents of variables
with text that nearly exceeds LINEBUF, because the LINEBUF limitation will
kick in for things like:
LOG="PIPE = '$PIPE'$NL"
Thus, LINEBUF should be set to something large after the experiment is run,
and before log statements are executed. Further it, can be difficult piping
the contents of these variables into shell scripts, for example:
LOG="length(PIPE) = "`echo '\$PIPE' | wc -c`"$NL"
was my first attempt, but I think this has problems due to shell quoting
requirements.
It'd be great if someone could come up with a comprehensive set of
careful tests in this area, or possibly just make some informed
conclusions based upon reading the source code.
ObAddendum, a year or so ago I had tried this type of syntax:
:0
* HB ?? ! ? spamassassin -options
And that works fine as far as procmail goes. There was a very
odd, hard-to-discern bug in SA back then, though, that caused
a buffer overrun when some messages were fed to it via the pipe that
way. The bug baffled us who were discussing it locally for a
couple of weeks until Mike Peeler looked at the source code and
figured it out.
It'd be good if Mike could share a description of the bug, a recommended
source code patch, etc. Keep in mind that procmail in many settings is
running as 'root', or some privileged user, when executing /etc/procmailrc.
A buffer overrun could lead to a serious security compromise, DoS, etc.
So we couldn't use that syntax with that version
of SA. (I think the current version is different enough for this
problem to have vanished.) But anyway, I wanted to capture the
value of spamc's -c option in the variable. So I went with what
I posted. Never caught that it didn't do what I thought!
The above syntax will work for clamscan. I use it (for testing only;
I still rely on Virus Snaggers in production). Its only drawback
that I can see is that it does not permit the variable capture.
The following recipe to the above will, indeed, read the exit
status correctly.
But anyway, now we come to a big caveat about we have been mostly
silent, but which is pretty important: Procmail's var-capture code
(VAR=| program) is flawed.
See above.
We already know that it fails on some
systems under some compiles. What is also known to me is that it
destroys memory space willy-nilly due to a bug in the code.
Again, I think it is important to subject this to a wider
review, to make sure that the security implications are known.
In general,
for the current release of procmail, I find it a dangerous thing to use.
It seems to work (for those systems where it does not fail outright);
but it is zapping memory locations. This is what I understand from
what a programmer who studied the source code said, it's not what I can
see myself. I'm not trained well enough in reading source code to know
first-hand about what I'm saying. (And it might be that I'm restating
this all slightly wrongly, I don't know.) But in my recipes that I
craft for the general use of others, I try always to avoid the syntax.
I'll use "VAR = `program`" instead.
In the mythical procmail of the future, I think that all fixed LINEBUF
restrictions should be removed. Procmail will still be reasonably efficient,
without interjecting all these surprises due to line length limitations.
Personally, I do use a private compile of procmail that has this and
several other bugs fixed, and that offers some other extended
features, the biggest being left-match capturing.
Left capturing is really needed. I'd settle for delimited capturing:
* VIRUS ?? ' : \/\<.*\>() FOUND$
where () is a null pattern match that serves to delimit the match
beginning at \/.
It's very cleverly
conceived by Mike Peeler. I do not know if Mike wishes to make
his code public or not.
It's both bad and good that procmail is no longer actively developed.
Bug fixes for memory corruption certainly should be made. Fixes for
local compilation problems should be made. Extensions are nice, but
there would need to be the usual open source steering committee that
would ensure compatibility, performance, and so on for the extensions
to have a good chance of being accepted by the user community.
_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail