The Coda HOWTO <author> Peter Braam, <tt/braam@cs.cmu.edu/, Andres Kruse <tt/Andres.Kruse@cern.ch/, Jan Harkes <tt/jaharkes@cs.cmu.edu/ </author> <date>v0.8, Nov 02 1998 <abstract> This HOWTO should give you help in running Coda. We try to cover enough detail for you to install, build and run Coda, and also explain how to troubleshoot and prepare useful bug reports. </abstract> <toc> <sect>Coda's ingredients <p> <sect> Getting Coda <p> All code can be obtained from: <URL URL="ftp://ftp.coda.cs.cmu.edu/pub/coda" NAME=here> . <sect1> Linux <p> <sect2> Getting packages <P> We have RPMS for a RedHat Linux at <URL URL="ftp://ftp.coda.cs.cmu.edu/pub/coda/linux/" NAME=here> . <sect2> Getting sources <P> You need to get a kernel module and user level code for the cache manager (Venus), the server (Vice) and the utilities. <itemize> <item> the linux-coda-?.?.?.tgz file contains the kernel code. <item> the coda-?.?.?.tgz file contains the user level code. </itemize> <sect> Installing Coda <p> <bf/WARNING:/ CODA IS BARELY READY FOR PRODUCTION USE. THIS RELEASE IS JUST FOR THOSE INTERESTED IN EXPLORING THOSE FEATURES WHICH WORK. IT CONTAINS KERNEL CODE, AND SERVERS RUNNING WITH ROOT PRIVILEGES, AND COULD LEAD TO DATA LOSS. <sect1> Linux and BSD systems <p> These are partial instructions on how to setup and configure the Coda filesystem. For further information, refer to the manual in /usr/local/share/doc/coda-doc-4.6.5 <sect2>Venus -- the client<p> 1. Initializing before running Venus: The venus-setup script does all the hard work, it will setup the coda control files, create /dev/cfs0 to communicate with the kernel, ... It also initializes a directory for cache files. The cache size should be at least 10Meg, typically 60-100Meg is used (ie. 60000 or 100000 in venus-setup [below]). DO NOT GO ABOVE 200-300Meg. All the files created will be placed under /usr/coda. You should make sure that there is enough space in the file system that /usr/coda resides on to hold a fully populated cache. <verb> venus-setup <comma_separated_host_list> <cache_size_in_kb> </verb> venus-setup and venus (below) are in sbin/. Make sure that sbin is in your path or that you use fully qualified pathnames. We strongly recommend that you try testserver.coda.cs.cmu.edu as the <tt/comma_separated_host_list/ first. NOTE: venus-setup will edit /etc/services to add some additional services.) NOTE: YOU MUST HAVE A KERNEL ENABLED WITH VENUS SUPPORT TO PROCEED. The INSTALL document and the BUILD document explain how to get a VENUS capable kernel. 2. Running Venus: The following assumes you are running X-Window. However, you could run these commands from virtual consoles as well, but omitting the <tt/xterm -e/ in front of the commands below. Start Venus with: venus & An -init flag can be given when venus is started; it flushes the local cache contents. It should be used whenever venus is connected to a different server. (venus-setup essentially forces an init to happen when venus is first started.) Observe the venus log with: xterm -e tail -f /usr/coda/etc/console It will tell you when venus has started and give status. Type: xterm -e codacon & to see the communications between the Venus and Vice. It is possible to see the upcalls from the kernel to Venus by turning up logging in Venus, but they are not very interesting. (To turn on minimal debugging, type: vutil -d 1 and then tail -f /usr/coda/coda.cache/venus.log.) NOTE: Remember, make sure your have enough free space on the filesystem in which /usr/coda/venus.cache resides for the cachesize indicated. That is, if you specify 20000 for for twenty-thousand kilobytes, this means you must have at least 20MB of free on the partition containing venus.cache. To halt venus, type: vutil shutdown (Or you can kill -9 venus, if you must.) IV. Codasrv -- the server 1. Initializing before running Vice: NOTE: that running a server seriously dips into your virtual memory. Running a server _and_ a client and X11 needs slightly over 64M of available VM. To set up an SCM server, you need to have the following resources: 1) an empty directory (viz /vicepa) where the fileserver will put files. There must be as much free space on this filesystem as the data you wish to store in Coda. 2) a raw partition for RVM metadata (you can use a file but it will be quite slow on a decent size server). This partition must be around 4% of the total size of the files you wish to store under /vicepa (e.g. on a 2GB server we use around 80M of rvm data). Consider 10M to be the minimum. 3) a LOG partition, preferably on a disk by itself. This needs not be large. 4) two secret tokens of _exactly_ 8 characters (eg elephant). The RVM files are used for the journalling/transactional aspects of Coda. Then run: vice-setup and answer its questions. 2. Running Vice: Start the rpc2portmap server, update server, client and the auth server, as well as the fileserver by typing: etc/rc.vice start Now observe the log: xterm -e tail -f /vice/srv/SrvLog & Determine with ps that codasrv, rpc2portmap, updatesrv and updateclnt are running. The SrvLog should show <em/File Server started/. If not, you have a problem. 3: Making Volumes Make a root volume (coda:root [below] should be the name you gave to vice-setup for the root volume name): createvol_rep coda:root E0000100 /vicepa NOTE: E0000100 is the Volume Storage Group set up for you by vice-setup. With more servers you can define other groups in /vice/db/VSGDB -- see the Coda User Manual. Now you are ready to point a Venus (client) at this server. The vice-setup program installed an administrative user on the server, named admin. It has a uid you chose and has been assigned password changeme. You may clog into Coda with this uid. Read section 7.7 in the Coda Manual to find out how to set up another user and password database (such users must be installed on clients too, but need not be in /etc/passwd on the servers). From the coda client side, the root volume will appear under /coda. <sect1> Installing and Configuring Coda for Windows 95 <P> <sect2> Clients <P> <bf/We are workign on this dream at present.../ Fetch ftp://ftp.coda.cs.cmu.edu/pub/coda/win95/coda-client-4.5.1.exe ftp://ftp.coda.cs.cmu.edu/pub/coda/win95/dpmi-win95ext-2.0.1.exe Run the installer: it installs the following (all of which are needed): From the DPMI extender package: vxd's in C:\dpmi95 mmap.vxd sock.vxd All further files are installed by the coda-client package: a) vxd's: in C:\coda-4.5.1\bin mc.vxd mcstub.vxd b) DOS-type user level code: - mount utilities in C:\coda-4.5.1\bin mount.exe relay.exe unmount.exe - the cache manager in C:\coda-4.5.1\bin venus.exe ==================== Manual Configuration ==================== a) Configuration work for the VxDs Edit C:\windows\system.ini and add device=c:\VxDs\mcstub.vxd device=c:\VxDs\mmap.vxd in the [386Enh] section. b) Coda binaries: you need at the minimum: venus.exe, relay.exe, mount.exe, umount.exe They are all built by gcc and reside under djgpp or coda-src/venus. c) Make sure you have a directory tree: c:\usr\coda\{etc,venus.cache,spool} You will need to edit the files: c:\usr\coda\venus.cache\myhost and put the ip of your host in it. Also edit: c:\usr\coda\etc\vstab and edit the server if you do not want to contact the testserver ============= Starting Coda ============= 1. start relay c:\coda-4.5.1\bin\relay This will load the socket and mc VxDs. 2. initialize Venus c:\coda-4.5.1\bin\venus -child 3. start venus (first time -- very few cache files just to play) c:\coda-4.5.1\bin\venus -init -cf 500 Wait a little bit. 4. mount N: Coda is mounted on the N: drive and should but doesn't work. To restart venus, do not give any flags. When you unmount coda with <tt/unmount.exe n:/ then you have to start relay.exe again. <sect1> Configuring Coda on Windows NT 4.0 <P> <enum> <item> Install into an area for NT binaries on your Linux box still (drink coffee, ignore or better even fix the compiler warnings) Put the binaries in a suitable place using: make BINDIR=TARGET/bin SBINDIR=TARGET/bin server-install <item> get samba running on that RH machine and export the build area for Coda. <item> get the Cygwin kit from http://www.cygnus.com/misc/gnu-win32/ <itemize> <item> install this on the NT machine, for example in C:\Cygnus <item> get a patch to <em/select/ and replace C:\Cygnus\B19\H-i386-cygwin32\bin\cygwinb19.dll with the one from: ftp://ftp.coda.cs.cmu.edu/pub/coda/support/nt-patches <item> get bash going (its in the Cygwin package). <item> organise your mountpoints for Cywin. Find a large directory say C:\coda that can serve as your Unix root on NT and type mount -b //C/coda / The -b, for binary mounting is vital. mkdir /bin /vice /rvm /vicepa We need /bin/sh in several places. cp //C/Cygnus/B19/H-i386-cygwin32/bin/bash /bin/sh export PATH=/bin:$PATH </itemize> <item> install your Coda stuff. Suppose the coda-4.5.1 area is a mapped drive on the NT machine with drive letter H: In bash: cd //H/ <item> Copy all the .exe files you built above to /bin </enum> Configuring ----------- If all is well you just need to run /bin/vice-setup Choose /rvm/log and /rvm/data as the log and data file. I recommend starting small with 2MB log and 20MB data. When this script completes startserver should bring up the server. However, it may very well not do this. Now create a volume: You are ready to attach a client: On a Linux box for example install Coda client software: venus-setup nt-server.domain.org 10000 venus & And play around. <sect> Building Coda <p> As a file system, Coda has several major components that need to be built: <itemize> <item> Kernel code on the client <item> The cache manager Venus on the client <item> The fileserver <item> Utilities for client and server administration. </itemize> <sect1> Building on Linux <p> <sect2> Building the kernel module <p> We now have a reaonably flexible method to build kernel modules. You can build the module for a kernel which you are not running at the time of the build. First of all get the linux-coda-?.?.?.tgz archive from <tt>ftp://ftp.coda.cs.cmu.edu/pub/coda/src/</tt>. You can unpack this anywhere on your target system. <bf/Prepare the kernel tree/ You do need a kernel tree handy, to give the module header information. To get ready: <tscreen> <verb> make oldconfig make dep </verb> </tscreen> In the top directory of linux-coda build the coda.o module: <tscreen> <verb> make config --- answer the questions make coda.o su make install depmod -a </verb> </tscreen> <bf/Notes/ <itemize> <item> If you build for a running kernel, you must still have a source tree for that Linux release, otherwise the headers cannot be found. However, that source tree doesn't need modversions.h etc. <item> We don't actively maintain 2.0 code anymore. The 2.1 code is preferred, but not yet perfect. </itemize> <sect2> Building the userlevel code <p> Coda builds most easily on <em/glibc/ systems. We do not actively maintain the <tt/libc/ variant anymore. You will need several libraries to link the Coda binaries, and include files from some of the lib???-devel packages. Readline and termcap are probably the most important ones. To build: <itemize> <item>Unpack <tt>coda-?.?.?.tgz</tt>. <item>./configure --prefix=/usr <item> make (make coda for versions 4.6 and older) <item> make client-install <item> make server-install </itemize> <sect1> Building on NetBSD and FreeBSD <sect1> Building a Client on Windows 95 <P> <bf>WARNING:</bf> The software provided to run Coda on Windows 95 is an early pre-alpha snapshot, made available to those interested. Coda runs kernel level code and priviliged processes which can cause damage to your system. Backup all data before playing with Coda. <bf>Short rationale</bf> Why DOS applications?? It would seem more straightforward to implement the Coda client cache manager, a user level program named Venus, as a Win32 application. Sadly on Windows 95 we ran into the following (fairly well known) problem. When a user application calls a Win32 file system call, the application may acquire a mutex in a win16 system dll. The request should reach the kernel, and make its way up to Venus. Venus is then unable to service the request because it cannot grab the mutex. Deadlock results. Implementing all of Venus as a kernel level cache manager seemed an invitation for disaster. Instead, by running the cache manager in a virtual DOS machine, as a DOS Protected Mode Interface application, one can bypass these problems, since such applications do not share the Windows dynamic libraries which gain the mutex. The price of following this path is high. There was no freely available socket support for such DPMI applications and no memory mapping support. These are provided by Michaels VXD's and standard library calls are now incorporated in the DJGPP toolchain. 1) Building the user level Code What do you need: a) for the user level code: a Linux machine with a few 100M of free space. download the tool chains made available under: ftp://ftp.coda.cs.cmu.edu/pub/tools/{win95,nt,djgpp,cygwin} You need the rpms for: - djgpp package - djgpp-win95ext - cygwin - gdbm-nt - gdb-djgpp a <tt/remote debugging/ environment for DPMI applications. It is highly recommended to set up Samba to share Linux generated binaries with the Windows box. b) for the VXD's: Visual C++ 5.0 (Internet Explorer), MS assembler 6.11, the Win95 DDK (comes with professional MSDN). Also you need to download a collection of header files vxdtdi.zip for networking from ftp.microsoft.com://developr/drg/WinSock/MS-Extensions/VXDTDI.ZIP. Note that the 95DDK needs the VC2.0 linker, which on the 95DDK in the directory MSVC20 (also get the files Link.err and DBI.DLL, that will be needed by the linker). You'll also need the latest MS assembler 6.11 which is on the 95DDK in the directory MASM611C. The VxD's you will want to build are in: vc95\95vxds\{mmap,sock,mcstub,mc} To build sock.vxd you'll need the header files provided by microsoft and can be found in VXDTDI.ZIP. You'll have to learn how to build VxDs, but it can be done from Developer Studio. -Change the path of the Linker in all .mak files. -In the MS-Studio environment (options) add: -executable path of the assembler 6.11. -include path of the 95DDK (Inc32). -lib path of the 95DDK (lib). -include path where VXDTDI Headers are. -include path of the 95DDK (Block/Inc). Place the mcstub and mmap VxD's in c:\VxDs (or elsewhere). The sock and mc vxd's need to go to the place where you install relay. (If you fancy finishing off Michael's nice codastart project, make sure to let us know! It is not essential for now.) c) Coda specific source code. Get the latest tarball of Coda sources. example: ftp://ftp.coda.cs.cmu.edu/pub/coda/src/coda-4.6.4.tgz Also get: ftp://ftp.coda.cs.cmu.edu/pub/coda/src/vc95.zip You can rebuild Venus with ./configure --host=windows95 ; make coda and the client utilities and server with ./configure --host=nt (the Venus built here is unusable on Win95). c) Win32 console applications C:\coda-4.5.1\bin The install script adds the vxd to your c:\windows\win.ini file and installs a coda configuration directory: c:\usr\coda with a default layout. <sect1> Building for Windows NT <p> Note: At present there is no client kernel code available yet. We will release some very experimental binaries soon. When are a little further with it, we will release the sources, of course. You can however build a Venus (which you can't use without kernel code) and you can build a server. Coda has alpha support for Coda servers running on NT and Windows 95. The Win32 binaries are constructed using the Cygnus Win32 kit, which effectively translates Unix system calls to Win32 calls. We build the Win32 binaries on Linux workstations using a cross compiler, just for our convenience. At the moment this is not for the faint of heart, since quite a few things have to be done by hand. Of course we encourage playing and will try to help and fix bugs. Building -------- <enum> <item> Get the cross compiling kit: <tt>ftp://ftp.coda.cs.cmu.edu/pub/tools/win32/cygwin-b19_glibc-2.i386.rpm </tt> You also need: <tt>ftp://ftp.coda.cs.cmu.edu/pub/tools/win32/libgdbm-nt-1.7.3-2.i386.rpm</tt> (These are rpms for RedHat 5.0. Sources are available of course.) <item> Get the coda-?.?.? tar ball <tt>ftp://ftp.coda.cs.cmu.edu/pub/coda/src</tt> <item> Unpack & build <verb> ./configure --host=nt make coda </verb> <item> drink coffee, ignore or better even fix the compiler warnings. </enum> Native building should be possible; you'll have to make a few changes to configs/coda.m4 and create a new configs/Makeconfig.ntnative file; then run autoconf and get cracking. But don't you dare send us patches with ^M characters made by Windoze machines! <sect> Administering Coda <p> V. Common Tasks 1. Normal server startup: startserver & it comes up at boot time from rc.vice, unless you said <bf/No/ to automatic startup, OR if the there is a file /vice/srv/CRASH exists, which indicates the fileserver crashed. Howver you need the update server and client for system administration and the authserver for authentication -- see /usr/pkg/etc/rc.vice. 2. Re-run venus-setup and point your server to your own server. * make sure the server is up _AND_ you have made root volumes* On your client do setup and start the client as follows: <tscreen> <verb> venus-setup your-server 40000 venus -init & xterm -e tail -f /usr/coda/venus.cache/venus.log xterm -e tail -f /usr/coda/etc/console xterm -e codacon </verb> </tscreen> When the server is up cd to /coda and start playing. You won't run -init the next time you start venus. The flag is necessary in this case since we are switching the server venus is talking to. (Currently, venus does not support Cells.) 3. Restarting a server: volutil shutdown shuts down the server. Restart as above. 4. Restarting a client. The official procedure is to: vutil shutdown kill -9 venus works equally well. Then restart as stated above. <sect> Troubleshooting Coda <p> <sect1> Troubleshooting the Server <p> If the server crashes before you have used it through /coda, do rm -rf /vice and start all over. At the minimum you must do before you restart. Remember: first clean, then startserver, then make rootvolumes, but ... If your server doesn't come up, ask for help on linux-coda@coda.cs.cmu.edu. It is Likely cleaning will reproduce the same problem. <sect1> Troubleshooting the Client <p> If venus comes up but hangs or crashes, check for zombie venus processes. Also read the etc/console file and see if venus has an assertion failure. If these happen, rebooting may help. If you are having repeatable difficulties starting venus, try starting venus with: venus -init Occasionally, bad state is left in the cache and this will clear it out. <sect1> Debugging under Unix<P> <sect1> Debugging under Windows 95 <P> Michael also built a remote debugging target for gdb. From the comfort of your Linux box you can step through the Coda code. Get the gdb-djgpp rpm from <tt> ftp://ftp.coda.cs.cmu.edu/pub/tools/{win95,nt,djgpp,cygwin} </tt> First run gdbserver :4711 your-prog your-arg on in the DOS command windows on Windows 95. Then fire up dos-gdb and attach to the server with target remote dos.host:4711 (4711 is a TCP port number). You can set break points, step and examine (try prepending _ if necessary). You cannot stop programs. It works for C and C++ compiled with the GNU C compiler gcc. </article>