(Illustration by Gaich Muramatsu)
Jan- Thanks for the reply! Impressive, my caches tend to be up to 500Mb or so. I think you may possibly be the first one to ever use a 10Gb client cache. 10Gb is small, if you use the proper unit of measurement: it's $4 worth of storage. I have a little over a terabyte of disk on my personal office computer, so dedicating 10Gb to my cache is trivial. My home directory is about 20Gb -- that is, about $7 of storage -- so 10Gb is only half of what it takes to hold my homedir. My intended use mode for coda was to have a filesys shared between my laptop and my office machine. I want my entire volume resident on both systems; I want complete access to the volume when I am disconnected on my laptop. So the mode of use is to arrange for my coda cache to be larger than the entire data set, and run with *everything* hoarded. I am trying this out to see if it will work. > - hoard it all with > hoard add /coda/lambda.csail.mit.edu d+ Do you use a single volume on the server, or are there multiple volumes? Single volume. I am not sure why it would want to revalidate the attributes of that CVS directory. Something like that would indicate that we either don't have cached access rights for the user who is trying to access it, or that the object is considered 'demoted' (i.e. we got a callback or returned from disconnection). Since there are no callbacks (incoming packets are dropped) I assume it is trying to check if the current user has access. > and the find tree walk hangs. After a minute or two, codacon says > unreachable lambda.csail.mit.edu ( 11:58:10 ) > and the find walk resumes with the following output: > > find: ./research/mrlc/mrlc/spim/CVS: Permission denied ... > find: ./research/mrlc/mrlc/confpaper: Permission denied Interesting, when we are disconnected the permission check is extremely permissive. Even if the cached rights would otherwise be considered stale we still allow access, and if they are missing we fall back on the System:AnyUser rights. I guess your volume is protected by an ACL, so that anonymous users cannot read those files. Yes, which is important. I wouldn't want to lose access to my protected files just because I disconnected. So to test that out, I removed all rights from System:AnyUser on the top-level dir. And we don't actually remove rights, but only mark them as stale (removal of cached rights only happens when you explicitly drop tokens with cunlog). So this would indicate that we never actually cached rights for those object, which is strange. Yes, that does seem strange. Both the earlier connected treewalk didn't actually walk the complete tree and hoard didn't cache rights. The hoard not caching rights may very well be possible, I may only try to make sure the file attributes and data are 'fresh'. I know GNU find has some optimizations that may make it skip subdirectories when the directory linkcount is off. You can disable those by using 'find -noleaf'. Maybe the workaround we try to use isn't working. It seems very weird that the hoard wouldn't cache rights. I have also had these timeout / no-permission errors when doing a big "cp -pr" copy into the coda filesys. cfs listacl and setacl only work while we are connected. This is because the client doesn't really know anything about users and groups so it is not useful to locally cache ACL data. Ahh. Would be nice if there was something about this in the cfs man page; I would never have guessed that on my own. What the client caches is what set of rights the user that fetched the object had. I would also never have guessed this, either! > - Then I reconnect with > cfs reconnect > Now I can see the problem directories with no trouble. I redo the find > tree-walk a third time and it completes with no problems. > + codacon shows *no* server->client file motion. Where there one or more 'validateattr' or 'validatevols' calls after reconnection? I don't understand what you are asking here. I also am having problems with hoard authentication. The man page says the hoard user must be either - root - the console user - the "primary" user from /etc/venus.conf but I have a hard time making this work. I'm logged into my computer, with X running on the console (vt7, which is standard). But hoard loses: % hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt canonicalize: chdir(shivers) failed (Permission denied) Then I su and try again: # hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt canonicalize: chdir(shivers) failed (Permission denied) Then I note that I have no tokens while su'd as root, so I clog and try again: # ctokens Tokens held by the Cache Manager for root: @lambda.csail.mit.edu Not Authenticated # clog shivers_at_lambda.csail.mit.edu username: shivers_at_lambda.csail.mit.edu Password: # hoard add /coda/lambda.csail.mit.edu/user/shivers/text/3min.txt pioctl:Add(7f000000_at_lambda.csail.mit.edu, ./user/shivers/text/3min.txt, 10, 0, 0): Permission denied # By the way, I made /usr/bin/hoard suid-root as suggested by the man page before trying any of this; no good. I also switched to the console's virtual tty vt1 and logged in there before trying any of this. No good. Then, I switched away from X and back to virtual tty 1 and tried the hoard. No good. Then, I logged out from vt1, logged back in as root, and was *finally* able to get my hoard to complete successfully. What's the deal with hoard? Two other problems with hoard: - the "hoard walk" command is not documented anywhere in the man page. I just stumbled over it in a scenario described in the admin & user's manual. - The "hoard clear", "hoard list" and "hoard modify" commands appear in the man page, but it never defines what they do. -OlinReceived on 2007-05-01 23:35:19