Coda File System

Re: Coda for syncronizing NFS servers?

From: Scott Serr <serrs_at_theserrs.net>
Date: Tue, 13 Sep 2005 09:34:02 -0600
Stephen J. Turnbull wrote:

>    Scott> Currently we have offices in several countries that have
>    Scott> large NFS servers - I think most are NetApps (Network
>    Scott> Appliance) NAS.  These are currently being syncronized by
>    Scott> using rsync cron jobs at night.  The problem is this
>    Scott> doesn't scale well and the network is having trouble
>    Scott> keeping up.
>
>Have you considered the BitTorrent protocol instead of rsyncing to a
>central server?  Of course you'd still have to find a way to identify
>new and changed files (BitTorrent as such assumes that's already been
>done), and if the changes are small relative to the files you might
>want to use rsync for transport anyway.  Those might be bigger
>problems than daisy-chaining by hand if it's a fairly small number of
>servers.
>
>Besides the comments Ivan made, you should also be aware that Coda
>doesn't do a lot to optimize file transport IIRC: if one bit changes,
>you have to transport the whole file before you can access any of it
>again.
>  
>
<a little off topic but maybe it will help the next guy that asks>
We have talked about BitTorrent a bit.  The trick would be to syncronize 
the small changes and not the whole structure, like you said so there 
would have to write our own client.  (I had not thought about writing 
our own software ontop of bittorrent -- thanks)  We probably have 180GB 
in the tree and say 2GB change nightly.

Something even more exotic would be Directory Tree Mirroring by 
multicasting.  There are a couple of projects, one on Sourceforge but it 
requires kernel modules and I believe it's own partition type.  A 
different project's draft paper here looks cool:  
http://member.*wide*.ad.jp/draft/*wide*-draft-soiasia-mtm-00.pdf  
(Interesting Stuff)

Thanks again!
-Scott
Received on 2005-09-13 11:33:50