Coda File System

Re: big coda / bsd vs. linux / intermezz vs. coda vs. lustre

From: Greg Troxel <gdt_at_ir.bbn.com>
Date: 14 Jan 2004 16:05:57 -0500
shivers_at_cc.gatech.edu writes:

> - Would coda have trouble with large filesystems, in the range 600Gb - 1.5Tb?
>   (E.g., a raid)

There is a limit on RVM size, which holds metadata.  So for moderate
numbers of huge  files it will be ok, but not huge numbers of moderate
files (this is all per server).  See the  spiffy new 'rvmsizer'
program in coda CVS.

> rvmsizer /usr/ANONCVS/netbsd-1-6 
9219 directories, 68400 files, 9622 directory pages
total file size        476470597 bytes (454.40MB)
average file size      6965 bytes
total directory size   5288448 bytes (5.04MB)
average directory size 573 bytes
estimated RVM usage based on object counts, 37938188 bytes (36.18MB)
estimated RVM usage based on 4% rule,       19270361 bytes (18.38MB)




> - If I were to set up a coda server, and I was just as willing to run
>   Free/Net/OpenBsd or linux, is there a best candidate?

I would run NetBSD.  As a coda server, the real issue is 'anonymous
rvm mapping', and I think that works better on BSD.  But others can
speak up and that may not be an issue any more.  Aside from that, I
think it's easier to deal with, more reliable, etc. but  that doesn't
have anything to do with coda.  There is also the issue of the
underlying fs for the coda files, and stability.  I don't know about
reiserfs, but I recall that with ext2fs the norm is to run async which
means no ordered metadata updates.  This is vulnerable to real trouble
on power fail/crash, and I have heard stories that indicate this
really happens.  With BSD, you get synchronous metadata by default,
and ordered metadata (softupdates) if you want.  As long as your hw
doesn't reorder writes (beware IDE, or check out write caching), big
corruption should not happen.  Plus, get a UPS for your server of course. 

There is a small minus that 1.6.2 doesn't have the 'realms' stuff in
the kernel.  -current does, and it's easy to patch the 1.6 seriies
kernels.



>   + Were one to choose between lustre, coda & intermezzo, can anyone
>     give me a rough picture of the tradeoffs? My criteria are:
>     + private use, can't pay $$$
>     + disconnected use over WAN
>     + replicated servers good
>     + clients for Win, BSD, linux & Mac good

Coda works mostly ok, except that when you are write-disconncted over
a thin pipe (28.8k) you will lose often with repair fake conflicts
that require a full client reinit.  The server is quite stable, but
has limitations on sizes that are annoying but not crazy.


>   Perhaps this coda/intermezzo/lustre question is a politically charged
>   question of the sort that makes regular members of this mailing list roll
>   their eyes in a here-we-go-again sort of way, and if so, please excuse my
>   ignorance.

I don't think this is the case.  Basically no one here has played with
it, I suspect.  The problem with systems like this is that they
require a significant investment to understand them and set them up.
I've been running coda since 1997 or so, and would be inclined to play
with intermezzo, but haven't had spare time or motivation.  I also
have the impression that it is mored tied to Linux.  Being a BSD
weenie, the lack of BSD support is a big drawback.

-- 
        Greg Troxel <gdt_at_ir.bbn.com>
Received on 2004-01-14 16:08:24