mharc-users

Re: make rebuild - just one list's archives?

2002-10-26 11:22:45
Earl Hood <earl(_at_)earlhood(_dot_)com> writes:
'make rebuild' is still going, twelve hours later. This is on a
reasonably powerful netbsd system.

You must have alot of mail.  Doing a complete rebuild of the
mhonarc.org archives does not take nearly that long, and it is
running on some older hardware.

These are the archives of the bigger list. The smaller list is about
14% its size. 

-rw-r--r--  1 mharc  wheel    418526 Oct 24 17:21 1996-07
-rw-r--r--  1 mharc  wheel   1019117 Oct 24 17:21 1996-08
-rw-r--r--  1 mharc  wheel   1055008 Oct 24 17:21 1996-09
-rw-r--r--  1 mharc  wheel   1837535 Oct 24 17:21 1996-10
-rw-r--r--  1 mharc  wheel   1981520 Oct 24 17:21 1996-11
-rw-r--r--  1 mharc  wheel   1693020 Oct 24 17:22 1996-12
-rw-r--r--  1 mharc  wheel   2342448 Oct 24 17:22 1997-01
-rw-r--r--  1 mharc  wheel   1877405 Oct 24 17:22 1997-02
-rw-r--r--  1 mharc  wheel   2222398 Oct 24 17:22 1997-03
-rw-r--r--  1 mharc  wheel   3086224 Oct 24 17:22 1997-04
-rw-r--r--  1 mharc  wheel   2811872 Oct 24 17:22 1997-05
-rw-r--r--  1 mharc  wheel   2925452 Oct 24 17:22 1997-06
-rw-r--r--  1 mharc  wheel   2973698 Oct 24 17:22 1997-07
-rw-r--r--  1 mharc  wheel   2059037 Oct 24 17:22 1997-08
-rw-r--r--  1 mharc  wheel   2632679 Oct 24 17:22 1997-09
-rw-r--r--  1 mharc  wheel   2943649 Oct 24 17:22 1997-10
-rw-r--r--  1 mharc  wheel   2894231 Oct 24 17:22 1997-11
-rw-r--r--  1 mharc  wheel   2871421 Oct 24 17:22 1997-12
-rw-r--r--  1 mharc  wheel   3552532 Oct 24 17:22 1998-01
-rw-r--r--  1 mharc  wheel   2888604 Oct 24 17:22 1998-02
-rw-r--r--  1 mharc  wheel   2916816 Oct 24 17:22 1998-03
-rw-r--r--  1 mharc  wheel   4001156 Oct 24 17:22 1998-04
-rw-r--r--  1 mharc  wheel   4016140 Oct 24 17:22 1998-05
-rw-r--r--  1 mharc  wheel   2838774 Oct 24 17:22 1998-06
-rw-r--r--  1 mharc  wheel   2983469 Oct 24 17:22 1998-07
-rw-r--r--  1 mharc  wheel   4009710 Oct 24 17:22 1998-08
-rw-r--r--  1 mharc  wheel   3817540 Oct 24 17:22 1998-09
-rw-r--r--  1 mharc  wheel   5371269 Oct 24 17:22 1998-10
-rw-r--r--  1 mharc  wheel   5150090 Oct 24 17:22 1998-11
-rw-r--r--  1 mharc  wheel   4996670 Oct 24 17:22 1998-12
-rw-r--r--  1 mharc  wheel   6428615 Oct 24 17:22 1999-01
-rw-r--r--  1 mharc  wheel   6726588 Oct 24 17:23 1999-02
-rw-rw-r--  1 mharc  wheel   7001129 Oct 24 17:23 1999-03
-rw-r--r--  1 mharc  wheel   6621326 Oct 24 17:23 1999-04
-rw-r--r--  1 mharc  wheel   6486829 Oct 24 17:23 1999-05
-rw-r--r--  1 mharc  wheel   7366366 Oct 24 17:23 1999-06
-rw-rw-r--  1 mharc  wheel   7487250 Oct 24 17:23 1999-07
-rw-r--r--  1 mharc  wheel   6929670 Oct 24 17:23 1999-08
-rw-r--r--  1 mharc  wheel   6900745 Oct 24 17:23 1999-09
-rw-r--r--  1 mharc  wheel   7139678 Oct 24 17:23 1999-10
-rw-r--r--  1 mharc  wheel   7300526 Oct 24 17:24 1999-11
-rw-r--r--  1 mharc  wheel   6253570 Oct 24 17:24 1999-12
-rw-r--r--  1 mharc  wheel   9826252 Oct 24 17:24 2000-01
-rw-r--r--  1 mharc  wheel   8902635 Oct 24 17:24 2000-02
-rw-r--r--  1 mharc  wheel   8194280 Oct 24 17:24 2000-03
-rw-r--r--  1 mharc  wheel   7465003 Oct 24 17:24 2000-04
-rw-r--r--  1 mharc  wheel   4790273 Oct 24 17:24 2000-05
-rw-r--r--  1 mharc  wheel   6572079 Oct 24 17:24 2000-06
-rw-r--r--  1 mharc  wheel   4819116 Oct 24 17:24 2000-07
-rw-r--r--  1 mharc  wheel   6122246 Oct 24 17:24 2000-08
-rw-r--r--  1 mharc  wheel   7209013 Oct 24 17:24 2000-09
-rw-r--r--  1 mharc  wheel   8851658 Oct 24 17:25 2000-10
-rw-r--r--  1 mharc  wheel   7911982 Oct 24 17:25 2000-11
-rw-r--r--  1 mharc  wheel   6870020 Oct 24 17:25 2000-12
-rw-r--r--  1 mharc  wheel  10674858 Oct 24 17:25 2001-01
-rw-r--r--  1 mharc  wheel   8887084 Oct 24 17:25 2001-02
-rw-r--r--  1 mharc  wheel   9495560 Oct 24 17:25 2001-03
-rw-r--r--  1 mharc  wheel   8766086 Oct 24 17:25 2001-04
-rw-r--r--  1 mharc  wheel   7186051 Oct 24 17:26 2001-05
-rw-r--r--  1 mharc  wheel   9811347 Oct 24 17:26 2001-06
-rw-r--r--  1 mharc  wheel   4879772 Oct 24 17:26 2001-07
-rw-r--r--  1 mharc  wheel   9886600 Oct 24 17:26 2001-08
-rw-r--r--  1 mharc  wheel   9438079 Oct 24 17:26 2001-09
-rw-r--r--  1 mharc  wheel   9633048 Oct 24 17:26 2001-10
-rw-r--r--  1 mharc  wheel   8140176 Oct 24 17:26 2001-11
-rw-r--r--  1 mharc  wheel   7754983 Oct 24 17:26 2001-12
-rw-r--r--  1 mharc  wheel   8757192 Oct 24 17:26 2002-01
-rw-rw-r--  1 mharc  wheel   7893845 Oct 24 17:15 2002-02
-rw-rw-r--  1 mharc  wheel   9129078 Oct 24 17:09 2002-03
-rw-rw-r--  1 mharc  wheel   9157211 Oct 24 17:09 2002-04
-rw-rw-r--  1 mharc  wheel   8982185 Oct 24 17:09 2002-05
-rw-rw-r--  1 mharc  wheel   8029780 Oct 24 17:09 2002-06
-rw-rw-r--  1 mharc  wheel   8253443 Oct 24 17:09 2002-07
-rw-rw-r--  1 mharc  wheel   9293162 Oct 24 17:09 2002-08
-rw-rw-r--  1 mharc  wheel   7497775 Oct 24 17:09 2002-09
-rw-rw-r--  1 mharc  wheel   7881816 Oct 24 17:10 2002-10

The mhonarc portion of the rebuild is quite fast, but the mknmz step
is painfully slow, with very long pauses in the output of 'make
rebuild'. CPU usage generally remains around 98%, when no other
high-priority processes are running.

  PID USERNAME PRI NICE   SIZE   RES STATE     TIME   WCPU    CPU COMMAND
20620 mharc     67    4    51M   50M run      23:21 97.80% 97.80% perl
11567 mharc     10    4    52M   50M sleep   103:06  0.00%  0.00% perl

Some info about the machine:

cpu0: Intel Pentium III (Katmai) (686-class), 551.28 MHz
total memory = 511 MB
avail memory = 469 MB

I ran out of disk space on the first try, so I'm doing another 'make
rebuild' now. (Started after I read your last email.) It's been
running for about twelve hours now, going backward from 2002-10 it is
at 1999-12.

Just providing some data, I guess. Is there a way to optimize the
mknmz portion of the job?

Morgan
 

---------------------------------------------------------------------
To sign-off this list, send email to majordomo(_at_)mhonarc(_dot_)org with the
message text UNSUBSCRIBE MHARC-USERS

<Prev in Thread] Current Thread [Next in Thread>