(Illustration by Gaich Muramatsu)
On 5 Dec 1998, Greg Troxel wrote: > Just for grins, I built the 4.7.4 source under FreeBSD 2.2.6. The > only hitch was that coda.h didn't end up including <sys/time.h>, and > therefore timespec wasn't defined. Other than that, it built. > Because of my later lossage, I didn't try to figure out the twisty > maze of ifdefs and determine how to fix this without breaking anything > else. I then tried the 4.7.4 venus on a system with a 4.6.5 > (patch-4.4.?) kernel. It gave me some message about can't mount /coda > or can't open cfs0. > > Before I try any further, is there any reason to expect this to work? > Or do I need the 4.7.4 kernel version? Because of the plethora of versions of FreeBSD, we are currently concentrating on -stable and -current, and will probably drop support at some point for -stable. I have successfully built and used Coda on many versions of 2.2-, but because 2.2- is a moving target (as is Coda) I believe we only test on 2.2.7 and 2.2-stable, and with the release of 2.2.8 presumably 2.2.7 will fall off the list. I believe a 4.7.4 userland should be able to talk to a 4.6.5 kernel, but because a number of bugfixes have been added to the BSD kernel code recently, I recommend running the newest Coda kernel code you can find. Under 3.0-CURRENT of FreeBSD, the Coda kernel code is in the base distribution (and can be retrieved using cvsup as part of the kernel code) so that makes life a lot easier. > On another note: > I had another client kernel crash. It was in cfs_fbsd.c:cvref, and > hit the panic with usecount <=0. I think this had to do with killing > venus while the network communication was wedged, but I can't > reproduce it at will. Should I be using 4.7.4 kernel/user code on the > clients? It seems like it has changed a bit and perhaps debugging the > 4.6.5 stuff is not useful. 4.7.x code included many bugfixes and race conditions concerning the death of venus. However, whether it is useful to debug 4.6.5 is probably a question for Bob Baron, as he is the active kernel developer for FreeBSD. > I tried to run coda across a firewall not administered by me (server > out, client in). I found that the venus to codasrv messages went out > and the replies came back, but that the codasrv-se to venus-se > messages were blocked. While I know I need to fix the firewall to > treat these as part of the same 'connection', I found that this wedged > the client particularly badly. I expected venus to give me 'operation > timed out' after a while, but it seemed not to (simple ls of /coda > which has two dirs project and home - nothing fancy). tcpdump showed > that the basic rpc request/response were getting through, but not the > -se stuff. So it might be that this isn't interpreted as failure, but > causes continuous retries. > > I don't know if it is possible to make the client send a udp datagram > from venus-se to codasrv-se when expecting side effect packets; this > would feel kludgy but probably cause coda to work across more > firewalls, as not everyone is fortunate enough to be able to cause > theirs to be changed. I haven't looked into this. Coda currently does not behave well when clients are behind a firewall. This is especially the case when outgoing connections from the firewall all appear to come from the same host (that is, it is not IP transparent). This is because the Coda server builds a 'callback connection' to each client, and attempts to maintain one per client IP address that connects to it. This connection is used to indicate changes to files on the server that the client has in its cache, and needless to say, incorrect functioning of this (the effective limit of one client per firewall :) is not good. On the other hand, having the firewall in the other direction (hiding a single server) probably would work ok. As you point out, NATs and firewalls often require a priming of their UDP packet forwarding, rather than letting unexpected UDP packets in (especially if it is not clear where the packet would go :). Indeed, changes could be made to RPC2/Coda to make sure a packet or two is always sent to or from the server first, but it depends on what your firewall arrangement is as to which would be needed. My feeling is that in the short term, firewall interoperability in Coda will be pretty poor based on a number of the underlying design assumptions (client IP == one cache manager) and the need to use an RPC package based on UDP. The use of encryption also prohibits firewall involvement, as it makes it difficult for the firewall to modify packets as they pass through the firewall so as to hide information (for example, the client IP address is included in the bind packet for the RPC connection, and communication to an RPC client is made using that IP address, not the address where it actually came from). There used to be some strange failure conditions when the main RPC connection succeeded and the side effects consistently failed -- have you observed any? I tried running a client from behind a NAT for a while, and while this often worked, the side effect behavior sometimes resulted in the client crashing due to a failed assertion. There are a few feasible long term solutions to the firewall problem. Most of them involve modifying the Coda servers to support the idea of multiple cache managers on a single client IP. This, in combination with not using fixed port numbers on the client should provide most of what we need, in combination with connection priming. Another interesting thing to try might be to have a Coda application proxy on the firewall that speaks the Coda protocol on one side (the outside) and looks like a single client, but speaks a lightly modified version of the RPC protocol on the inside to work around the encryption concerns, and proxies the data requests to the correct clients as needed. This would not be so easy to write, I suspect. Having the server behind a firewall is likely to be an unfortunate experience, and may not be easily solvable. For now, it is a fair assumption that one IP address per server (at minimum) is required. > The good news veiled in this message is that coda is already working > well enough for me that I'm expanding the user community to about 6 > from 1.2 people. This is great news! Robert N Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: 03 01 DD 8E 15 67 48 73 25 6D 10 FC EC 68 C1 1C Carnegie Mellon University http://www.cmu.edu/ TIS Labs at Network Associates, Inc. http://www.tis.com/ SafePort Network Services http://www.safeport.com/Received on 1998-12-05 13:41:45