(Illustration by Gaich Muramatsu)
Stephen J. Turnbull wrote: > Scott> Currently we have offices in several countries that have > Scott> large NFS servers - I think most are NetApps (Network > Scott> Appliance) NAS. These are currently being syncronized by > Scott> using rsync cron jobs at night. The problem is this > Scott> doesn't scale well and the network is having trouble > Scott> keeping up. > >Have you considered the BitTorrent protocol instead of rsyncing to a >central server? Of course you'd still have to find a way to identify >new and changed files (BitTorrent as such assumes that's already been >done), and if the changes are small relative to the files you might >want to use rsync for transport anyway. Those might be bigger >problems than daisy-chaining by hand if it's a fairly small number of >servers. > >Besides the comments Ivan made, you should also be aware that Coda >doesn't do a lot to optimize file transport IIRC: if one bit changes, >you have to transport the whole file before you can access any of it >again. > > <a little off topic but maybe it will help the next guy that asks> We have talked about BitTorrent a bit. The trick would be to syncronize the small changes and not the whole structure, like you said so there would have to write our own client. (I had not thought about writing our own software ontop of bittorrent -- thanks) We probably have 180GB in the tree and say 2GB change nightly. Something even more exotic would be Directory Tree Mirroring by multicasting. There are a couple of projects, one on Sourceforge but it requires kernel modules and I believe it's own partition type. A different project's draft paper here looks cool: http://member.*wide*.ad.jp/draft/*wide*-draft-soiasia-mtm-00.pdf (Interesting Stuff) Thanks again! -ScottReceived on 2005-09-13 11:33:50