procmail
[Top] [All Lists]

Re: How to let procmail use mre memory?

2010-01-31 16:52:28
Hello,

Am 2010-01-31 12:51:35, schrieb JW Simpson:
Following up on that, I get this:
        john(_at_)system76-pc:~/kon$ SIZE=127;export BIGVAR=`seq -ws '' 1 
1000000 | dd count=$SIZE bs=1K`;test -z "${BIGVAR}"; echo /bin/sh -c 'test -z 
"${BIGVAR}"; echo $?' | xargs --show-limits
        127+0 records in
        127+0 records out
        130048 bytes (130 kB) copied, 0.0055172 s, 23.6 MB/s
        Your environment variables take up 132526 bytes
        POSIX upper limit on argument length (this system): 1962578
        POSIX smallest allowable upper limit on argument length (all 
systems): 4096
        Maximum length of command we could actually use: 1830052
        Size of command buffer we are actually using: 131072

[XXXXXXXXXXXXXXXX(_at_)mail:~] xargs --show-limits
Your environment variables take up 1613 bytes
POSIX lower and upper limits on argument length: 2048, 129024
Maximum length of command we could actually use: 127411
Size of command buffer we are actually using: 127411

Execution of xargs will continue now, and it will try to read its input
and run commands; if this is not what you wanted to happen, please type
 the end-of-file keystroke.

It seems that I hit the limit using TMPVAR3 (the fetched HTML-File)  and
then with TMPVAR5 (get the HTML stuff from a marker to the bottom).

TMPVAR is 87 kByte and TMPVAR over 60 kByte and this hit the limit.

I will try to change the code from

----8<------------------------------------------------------------------
    # Download the AppNote
    TMPVAR3=`wget --user-agent="tdtools-procmail v${TDTP_VERSION}" --quiet -O - 
${TMPVAR1} ; :`
    :0
    * ? test -n "${TMPVAR3}"
    {
      # Get the link to the PDF
      TMPVAR4=`echo -e "${TMPVAR3}" |grep 'pdfserv' |head -n1 |sed 
's|.*http://pdfserv|http://pdfserv

      # Eleminate the unused HEADER and attach a new HEAD
      # if it fails, use the original and continue
      TMPVAR5=`echo -e "${TMPVAR3}" |sed -n '/APPLICATION NOTE&nbsp;/,$p' |sed 
's|.*APPLICATION NOTE&
      :0
      * ? test -z "${TMPVAR5}"
      {
        TMPVAR5=`echo -e "${TMPVAR3}" |sed -n '/REFERENCE DESIGN&nbsp;/,$p' 
|sed 's|.*REFERENCE DESIG
      }
      :0
      * ? test -z "${TMPVAR5}"
      { TMPVAR5="${TMPVAR3}" }
----8<------------------------------------------------------------------

to

----8<------------------------------------------------------------------
    # Download the AppNote
    TMPVAR3=`wget --user-agent="tdtools-procmail v${TDTP_VERSION}" --quiet -O - 
${TMPVAR1} ; :`
    :0
    * ? test -n "${TMPVAR3}"
    {
      # Get the link to the PDF
      TMPVAR4=`echo -e "${TMPVAR3}" |grep 'pdfserv' |head -n1 |sed 
's|.*http://pdfserv|http://pdfserv

      # Eleminate the unused HEADER and attach a new HEAD
      # if it fails, use the original and continue
      :0
      * ? echo -e '${TMPVAR3}' |grep 'APPLICATION NOTE&nbsp;'
      {
        TMPVAR3=`echo -e "${TMPVAR3}" |sed -n '/APPLICATION NOTE&nbsp;/,$p' 
|sed 's|.*APPLICATION NOTE&
      }
      :0E
      * ? echo -e '${TMPVAR3}' |grep 'REFERENCE DESIGN&nbsp;'
      {
        TMPVAR3=`echo -e "${TMPVAR3}" |sed -n '/REFERENCE DESIGN&nbsp;/,$p' 
|sed 's|.*REFERENCE DESIG
      }
----8<------------------------------------------------------------------

The problem is, if something goes  wrong  with  it,  the  ORIGINAL  HTML
content is trashed.

OK, I could try to put the HTML-Stuff into the Mail and use a filter  to
modify it

----8<------------------------------------------------------------------
    # Eliminate any encodings and get the URL
    TMPVAR1=`mimedecode |grep --max-count=1 "^URL: " |sed 's|^URL: ||' ; :`

    # Get the number of the AppNote
    TMPVAR2=`echo "${TMPVAR1}" |sed 's|.*an_pk/||' |tr -d ' ' ; :`

    # Download the AppNote
    TMPVAR3=`wget --user-agent="tdtools-procmail v${TDTP_VERSION}" --quiet -O - 
${TMPVAR1} ; :`
    :0
    * ? test -n "${TMPVAR3}"
    {
      # Get the link to the PDF
      TMPVAR4=`echo -e "${TMPVAR3}" |grep 'pdfserv' |head -n1 |sed 
's|.*http://pdfserv|http://pdfserv

      :0fw
      | ( formail -i "Subject: [${TMPVAR2}] ${MSG_SUBJECT}" ; \
          echo 
"========================================================================${NL}" 
; \
          echo "PDF URL: ${TMPVAR4}${NL}" ; \
          echo 
"========================================================================${NL}" 
; \
          echo "####_BEGIN_HTML_2_####" ; \
          echo "####_BEGIN_HTML_1_####" ; \
          echo "${TMPVAR3}" ;  0
          echo "####_END_HTML_1_####" ; \
          echo "####_END_HTML_2_####" )

      TMPVAR3=""

      :0fw
      * ?? B APPLICATION NOTE&nbsp;
      |sed '/####_BEGIN_HTML_1_####/,/APPLICATION NOTE&nbsp/d' |sed 
's|.*APPLICATION NOTE&nbsp;|<html><head></head><body><table><tr><td>APPLICATION 
NOTE |'

      :0fw
      * ?? B REFERENCE DESIGN&nbsp;
      |sed '/####_BEGIN_HTML_1_####/,/REFERENCE DESIGN&nbsp/d' |sed 
's|.*REFERENCE DESIGN&nbsp;|<html><head></head><body><table><tr><td>APPLICATION 
NOTE |'

      # Eleminate the unused FOOTER
      :0fw
      * ?? B <!\-\-\ BEGIN:\ EE-MAIL\ \-\->
      | sed '/<!-- BEGIN: EE-MAIL -->/,/####_END_HTML_1_####/d'

      # Attach a footer if there is no one
      :0fw
      * ?? B ! </html>
      | sed  
"s|####_END_HTML_2_####|${NL}</table></body></html>${NL}####_END_HTML_2_####|"

      TMPVAR9=`sed '/####_BEGIN_HTML_2_####/,/####_END_HTML_2_####/$p' |grep -v 
'HTML_2_####' |html2text -width 72 -nobs -style pretty ; :`

      :0fw
      * ? test -n "${TMPVAR9}"
      | ( sed "/####_BEGIN_HTML_2_####/,/####_END_HTML_2_####/d" ; \
          echo "${TMPVAR9}"
    }
    :0
    .Business.USA.Maxim.App_Notes/
----8<------------------------------------------------------------------




Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Electronic Engineer
    Tamay Dogan Network
    Debian GNU/Linux Consultant

-- 
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   Apt. 917
<http://www.flexray4linux.org/>               50, rue de Soultz
Jabber linux4michelle(_at_)jabber(_dot_)ccc(_dot_)de           67100 
Strabourg/France
IRC    #Debian (irc.icq.com)                  Tel. DE: +49 177 9351947
ICQ    #328449886                             Tel. FR: +33  6  61925193
____________________________________________________________
procmail mailing list   Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)de
http://mailman.rwth-aachen.de/mailman/listinfo/procmail