I apologize for bothering you with a series of questions that I already know
the theoretical answer to.  I shall try to keep this as short as possible.

As an overview, the company I work for is an ISP, and as such does all of
the usual things of web pages, e-mail, etc.  In this case, we are
redesigning the mail servers.  Currently there are 4 mail servers running
with all storage handled through NFS to a RAID5 server.  We are completely
overloading NFS based on the amount of data we are sending and receiving,
which can often hit 6MB/s.  I've seen it spike to just below 10MB/s while
getting mail bombed.  NFS is not supposed to be able to handle this level of
traffic, and we can only relate it to the excellent implementation that
FreeBSD did with the NFS code, and pure luck.

We will be purchasing a brand new RAID and 6 servers to go with it, all with
fibre cards to create a SAN.  However, as we are completely redesigning it,
we want to implement functionality of the sort coda offers, replication,
redundancy, backup, etc.  The scenario would be the 6 servers mounting, in
some fashion, the data from the RAID, and all the data on the RAID being
replicated onto a second RAID, also through the SAN.  Automatic failover
would be an extremely nice benefit, but I have not studied enough to be
certain that is possible.

We will be handling in the area of 40k users currently, with total data in
the range of 400gigs, average file size of about 5k, and thousands of files
per folder.  In one case I've even seen over 200k files in one folder, which
was insane.  (Of course that was myself when I forgot about a user I had
created and had subscribed to a few mailing lists.)  Anyway, from my
understanding of the messages I've read, coda will not be able to provide
the level of performance we need.  However, I would be very glad to be
wrong.

Finally, and I am sorry to mention this here, but if coda does not suffice,
are there any good places you'd recommend exploring?  We refuse to use any
operating system than FreeBSD, so sun is pretty much out.  Outside of that,
are there any recommendations of what to research?

Thank you very much,

Mark Dickey
mark_at_bestweb.net
(cyon_at_bestweb.net)