Coda File System

Re: slow write performance on linux

From: Ivan Popov <pin_at_math.chalmers.se>
Date: Fri, 28 Mar 2003 17:28:32 +0100 (MET)
Hello Steve,

pleasant to hear that you are taking the Coda evaluation orderly.

> the problem i'm having is that write operations are painfully slow.

As Jan has described, it is the synchronous nature of the operations on
metadata that makes updates inevitably slow, in strongly connected mode.
Data transfer is rather fast. My "ancient-hardware" clients and servers
fill up a 10 Mbit connection at store operations, giving about 900
Kb/sec.

> i noticed that this issue has come up again and again in various
> forms on this list, i'm curious to know if anyone has any insight,
> or can suggest something new i can try out.

If you want to have guaranteed consistency on a read-write replicated
filesystem, you have to live with these delays.
It can be improved with time, by smarter kernel on the server and
efficient scheduling, but synchronous operations always will be a lot
slower than asynchronous ones.

> i've considered using my volumes in write-disconnected mode and
> trickle in the changes, but in my brief testing, i've observed various
> problems with reconnecting. so i'd prefer not to use this mode.

There are certainly still some bugs that make write-disconnected operation
prone to unnecessary conflicts or similar problems.

Anyway, the recent cvs coda version I am running behaves reasonably well,
except that write-disconnected *massive* updates are not to recommend.

"Massive" means "more information than the least of rvm log sizes
on your client and the servers". The asynchronous nature of the updates,
that gives you performance, stresses the logs and you risk overflowing
them.

For example rm -r /file-tree-with-thousands-of-files
is a dangerous operation, as it generates a lot of metadata updates really
fast (my recent experience :)

For a fair comparison, on a traditional *local* filesystem
rm a-flat-dir-with-thousands-of-files/*
can be a very slow operation - then you have to either live with it,
or rename the directory, let it be cleaned in the background
and create a new empty one - in some way like you have to choose between
slow/always-ok or fast/with-limitations on Coda.
(my recent experience on a commercial *nix system with its standard
local filesystem, fast hardware, on a huge dir - a couple of unlink() per
second...)

Best regards,
--
Ivan
Received on 2003-03-28 11:30:12