Coda File System

Re: Files Bigger Than Cache Size

From: <coda_at_bobich.net>
Date: Thu, 31 Jan 2008 09:43:57 +0000 (GMT)
On Thu, 31 Jan 2008, u+codalist-p4pg_at_chalmers.se wrote:

>> Right. So to handle 2GB files, the RVM log would also need to be 2GB? I
>
> No. There are "cache size", "RVM size" and "rvm LOG size". The last one
> is the least to be concerned about.
>
> Asking Google for "rvm log size", the second entry looks as follows:
> --------------
> Optimizing Coda 6.x - Codawiki
> A log size of 20M should work for any setup; Note that the size of the 
> RVM is dependent on the number of files and directories across all 
> volumes (not per ...
> coda.wikidev.net/Optimizing_Coda_6.x - 19k
> --------------

Right, OK. The reason why I wasn't sure that the documentation was correct 
was because it says that the nature of file replication is copy-on-close, 
so since Coda handles the whole file in one transaction, I wrongly 
concluded that the transaction log needs to be the size of the largest 
file I am likely to need to handle.

> The "Client cache" section on that page needs improvement though. There
> are contradicting statements, each of those would be misleading if taken
> literally.

Just to make triple-sure - except on the Windows client, the cache needs 
to be big enough to fit the biggest file we intend to handle (with the 
largest handlable file size being 2GB). Is that right?

>> noticed that 8GB cache works fine, but at 16GB (on 32-bit x86) venus seems
>> to fail to initialize.
>
> Max usable RVM size is about 1GB. This is mentioned on the Wiki.
> Jan points out that bigger sizes are possible but in reality they are
> not to be recommended, in any case not for a casual user or novice admin.

I presume you are referring to the metadata size when you say RVM size.

> (The above concerns 32-bit systems, may be considerably different
> on 64-bits systems, then it is you who provides the answers by experimenting :)

Will do as soon as I get my 64-bit system up and running. At the moment I 
only have 32-bit ones. :)

> When you ask Venus for a certain cache size, it guesses how many files
> and modification entries you might need (unless you specify explicitly)
> and allocates RVM accordingly.
> This may lead to allocation of a potentially huge RVM - which possibly fails.

OK, now I'm confused - I thought the RVM things were for the server side, 
not the client side. The server setup asks for RVM log and data sizes, 
venus-setup doesn't. Or are you saying that venus does the same thing, 
only the setup doesn't ask for details in order to make the client setup 
easier?

> A huge RVM, when it is populated, makes Venus memory footprint very large,
> affecting the performance of the client host.

I can see venus using 430MB of virtual memory, but I suspect this is 
mmapped to a file, because the total memory (RAM+swap) consumption of the 
system is about half of what venus claims to use.

> So, if you will be accessing primarily big files, always specify
> explicit (low) file / cml numbers when setting up a client.

Right, OK. The problem is that I'm hoping to use Coda for home directories 
which contain anything from thousands of small files (Maildirs) to dozens 
of large files (e.g. tared backups, Linux ISOs, etc.).

Gordan
Received on 2008-01-31 04:46:16