(Illustration by Gaich Muramatsu)
So many questions, On Tue, Nov 16, 2004 at 03:28:32PM -0600, Jerry Amundson wrote: > I have several GB of maildirs in a compressed tar file. When I try to > untar (to venus on the SCM), it gets a ways before giving me "File too > large" errors. But the SCM has plenty of space. Either there is a single file > 100MB, or a directory has become > 256KB (although that second case still crashes the client far too often). > [root_at_uhura home]# cfs lv /coda/pbs.com I guess you only have a single volume in which you are trying to untar the tarfile. In that case there will be other limits that will affect you. There probably is some arbitrary ceiling wrt the total number of files in a single volume and I know there is a (configurable) limit for the number of directories in a volume. The known directories per volume limit is related to the size of the resolution log, even if there is nothing to resolve, every directory has a ResolveNULL placeholder entry. And since the administrative limit is somewhere around 4K or 8K resolution log entries this means we run out once we create the same number of directories. This shouldn't affect volumes with only a single replica. To increase the administrative limits we'd need to run for every replica of the replicated volume, something like, 'volutil setlogparms <replica-id> reson 4 logsize 16384' > Can I bypass venus, or manipulate cache size somehow? You cannot bypass venus, but the venus cache size can be increased (/etc/coda/venus.conf:cacheblocks=500000). You will need to reinit the client before it picks up the new value though. > Also, how to I make sure the data is replicated to my second codasrv? > Ie. venus on the second codasrv could really just be looking at the > first. Ehh, that should be automatic as long as the server is reachable and the volume actually was created as a doubly replicated volume on both servers. A simple check on which servers a volume is replicated, 'cfs whereis /coda/pbs.com' On Tue, Nov 16, 2004 at 11:52:04PM -0600, Jerry Amundson wrote: > On Tue November 16 2004 8:57 pm, Stephen J. Turnbull wrote: > > this limit is given in bytes (256K, IIRC), not dentries, I suppose > > that a somewhat imprecise "file too large" error might be issued when > > you run out of space for more entries in a directory. > > Could be, I didn't think that would rear it's ugly head so soon, though. > > [root_at_uhura vicepa]# find 0 | wc -l > 2717 If the filenames are shorter than about 32 characters I would expect the limit to hit around 4000 entries in a single directory (approximately 32 for the name + probably another 32 for other data per entry ). My maildir filenames are already 25 characters long but only seem to use only the hostname and not the fully qualified domain name. If it were to use the fqdn most files exceed 32 bytes and end up getting padded to the next multiple of 32, limiting the total size to 256K/96 = 2730. All of these numbers are approximate, there is some additional padding going on and I think there is some reserved space for an allocation bitmap and such. The 'rvmsizer' tool contains the full formula and will display a warning whenever it encounters a directory that won't fit in the existing Coda directory structure. Considering how close your measured 2717 and my estimated 2730 entries are I guess that your filenames contain the fully qualified domain name and are typically longer than 32 bytes. > > Jerry> Also, how to I make sure the data is replicated to my > > Jerry> second codasrv? Ie. venus on the second codasrv could > > Jerry> really just be looking at the first. > > > > Shut down the first codasrv, and take a look via either venus, is the > > obvious brute-force approach. I don't think looking in the server's That is definitely a brute-force solution. First thing would probably be to check if the volume is actually replicated by using 'cfs whereis'. > Looks like createvol_rep has to specify both server - the second server > cannot be added to an existing volume...? > http://www.coda.cs.cmu.edu/maillists/codalist/codalist-2004/6180.html Correct, growing and shrinking existing replicated volumes is definitely something I want, but right now it requires a series of magic volutil incantations and various editing/regenerating of VRList/VRDB and VLDB files (and some required crossing of fingers). Definitely not recommended if you're just getting started with Coda. On Wed, Nov 17, 2004 at 09:05:53AM -0600, Jerry Amundson wrote: > [root_at_uhura mail]# cfs lv /coda/pbs.com > Status of volume 0x7f000000 (2130706432) named "/" > Volume type is ReadWrite > Connection State is WriteDisconnected > Minimum quota is 0, maximum quota is unlimited > Current blocks used are 214647 > The partition has 6392816 blocks available out of 11294220 > *** There are pending conflicts in this volume *** > There are 13528 CML entries pending for reintegration > > [root_at_monamie mail]# cfs lv /coda/pbs.com > Status of volume 0x7f000000 (2130706432) named "/" > Volume type is ReadWrite > Connection State is WriteDisconnected > Minimum quota is 0, maximum quota is unlimited > Current blocks used are 214647 > The partition has 6392816 blocks available out of 11294220 > *** There are pending conflicts in this volume *** > There are 9421 CML entries pending for reintegration Ehh, are these from different startups, or right after each other? It looks like the client is rapidly reintegrating as over 4000 operations got pushed to the server between the two calls. But if there really is a conflict it should have frozen the volume. > Q1: How do I get back to "Connected"? "cfs wr" didn't seem to change > anything. Really only 2 ways out, either find and repair the conflict or throw away all local (but uncommitted) changes. To dump the local CML the 'friendly way', cfs purgeml /path/to/volume Alternatively to completely wipe any bad state from the Coda client, kill the client, unmount /coda, and restart venus with a -init flag or create a file named INIT in the venus.cache directory. > Q2: How do I repair the conflict? beginrepair prompts "Pathname of > object in conflict?", which I do not know. Typically it is the first entry in /usr/coda/spool/<uid>/<volname>.cml (/var/lib/coda/spool/?) which is the first modification log entry that failed to reintegrate. Alternatively, one that I tend to use a lot as it can be used to find server-server conflicts as well, find /coda/path/to/volume -lname '@*' > Q3: Do I need to use "cfs fl" in some controlled fashion to copy the > maildir's to this box? And throttle my tar extract, somehow? Ehh, no. cfs fl simply dumps things from the cache. If your client is (write-)disconnected it will automatically start throttling when the CML fills up. Ofcourse disconnected operation does tend to be more sensitive to conflicts, although most conflicts occur on the boundary where we switch between connected and disconnected because a connected operation typically times out and is retried during reintegration. Some operations have no problem with this, but others complain that we're trying to modify an already modified file, or create an already existing object, etc. You could use 'cfs strong ; cfs cs ; cfs wr', cfs strong tells venus to not switch to write-disconnected operation as a result of sluggish server behaviour. cfs cs then makes sure that all servers are reachable. And finally 'cfs wr' tells the client to try to move all volumes to 'write-reconnected' or fully connected state. JanReceived on 2004-11-17 16:25:55