On Wednesday, 5 בMarch 2008 23:56, Bart Schaefer wrote:
Hello,
We're using postfix+procmail with NFS on netapp for
many years, on what I believe to be a largish setup
for a NFS environment, about 60,000 users. We never
saw mail corruption except on one case, that is, when
one of the mounts was using UDP and the filer hit
100% cpu for more than a few minutes.
I never used the `sync' mount option, due to it's
enormous hit on performance. Also, I believe that the examples
and data quoted about "data corruption when using async"
was related to exporting filesystems over NFS from a Linux
NFS server(!) with the sync/async option enabled in the
export.
Are you using NFS over UDP ? Are you using soft mounts ?
Our setup:
2 postfix mail servers
2 imap servers (used to be UW, now using dovecot)
2 pop servers (using UW ipop3d).
All 6 servers mount the same /var/spool/mail, and
the same home directory trees, all via NFS from the
same filer (FAS3050, not clustered). All servers are
active (load balanced via an external load balancer),
so a mailbox may be open from some least 3 locations
(2 mails delivered to the mailbox at the same time
by mailserver1 or 2, and also an active pop or imap
session for the user).
We use NFSv3 only, and on the netapp we're not allowing
mounts with rsize/wsize greater than 8k (due to switches
having buffering problems with larger packets).
Our mount options:
rw,rsize=8192,wsize=8192,lock,tcp,nfsvers=3,hard,intr,bg,nosuid,nodev
On the netapp, our options for NFS (and other relevant stuff) are:
ip.tcp.newreno.enable on
ip.tcp.sack.enable on
nfs.acache.persistence.enabled on
nfs.assist.queue.limit 40
nfs.export.allow_provisional_access on
nfs.export.auto-update on
nfs.export.exportfs_comment_on_delete on
nfs.export.harvest.timeout 1800
nfs.export.neg.timeout 60
nfs.export.pos.timeout 3600
nfs.export.resolve.timeout 6
nfs.hide_snapshot off
nfs.ifc.xmt.high 16
nfs.ifc.xmt.low 8
nfs.kerberos.enable off
nfs.kerberos.file_keytab.enable off
nfs.locking.check_domain on
nfs.mount_rootonly on
nfs.mountd.trace off
nfs.netgroup.strict off
nfs.notify.carryover on
nfs.ntacl_display_permissive_perms off
nfs.per_client_stats.enable on
nfs.require_valid_mapped_uid off
nfs.response.trace off
nfs.response.trigger 60
nfs.rpcsec.ctx.high 0
nfs.rpcsec.ctx.idle 360
nfs.tcp.enable on
nfs.tcp.xfersize 8192
nfs.udp.enable on
nfs.udp.xfersize 8192
nfs.v2.df_2gb_lim off
nfs.v3.enable on
nfs.v4.acl.enable off
nfs.v4.enable off
nfs.v4.id.domain taucc.tau.ac.il
nfs.v4.read_delegation off
nfs.v4.write_delegation off
Some optimization of /etc/sysctl.conf for the imap/pop servers:
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 30
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 600
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn on the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
# Increase max files in the system
fs.file-max = 65536
kernel.threads-max = 65536
#
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_wmem = 4096 87380 8388608
net.ipv4.ipfrag_low_thresh=262144
net.ipv4.ipfrag_high_thresh=39321
#
# cd into /proc/sys/net/core
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576
net.core.rmem_default = 65536
net.core.wmem_default = 65536
#
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=1
YMMV
--Ariel
--
Ariel Biener, CISO
Tel-Aviv University CIT div.
e-mail: ariel(_at_)aristo(_dot_)tau(_dot_)ac(_dot_)il phone: 03-6406086
PGP key: http://www.tau.ac.il/~ariel/pgp.html
____________________________________________________________
procmail mailing list Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail