(Illustration by Gaich Muramatsu)
(Illustration by Gaich Muramatsu)
A Coda cell is a set of servers which all believe one member of
the set to be the master, or SCM server. All modifications to
important Coda databases should be made on the SCM, otherwise the
SCM plays the role of an ordinary Coda server. The
updateclt
/
updatesrv
daemons will then
propagate changes from the SCM to the other servers.
The first server setup must be the SCM. This chapter is divided
into three sections: installing the binaries, configuring the
system using the configuration script
vice-setup
(for
both SCM and non-SCM servers) and finally, a detailed look at what
needs to be done to setup a server manually. Before beginning, we
have provided some overview information to assist in setting up the
server.
To help ensure that data is not lost or left in inconsistent state between server restarts, Coda uses Recoverable Virtual Memory (RVM) which is a transaction system that maintains the system state of the Coda server meta data. RVM is a data logging transaction system which writes modifications into the RVM log and upon truncation or restart incorporates such modifications into the RVM data file. Both log and data file for RVM are ideally held on raw partitions.
NOTE: this should not be confused with Virtual Memory.
Upon startup, the Coda servers use RVM to restore the Coda system state. To efficiently use this feature, you should have dedicated disk partitions for optimal performance the log partition ideally on its own disk. However, a disk sharing the log partition with other disk partitions or the use of a log file may be used at corresponding loss of optimal performance. See section XXX for more details on RVM.
Coda servers require a minimum of 2 disk partitions for optimal
performance (one raw partition for the RVM Log, one raw partition
for RVM Data and one regular UNIX filesystem to hold file data
store in Coda), data security and protection from accidental
deletion. For additional performance gains the RVM Log partition
should be on its own disk to avoid head movement latencies which
reduces disk seek times on log operations. Optionally,
/vice
can be a separate partition for the same reasons
it is advantages to have
/var
as a separate
partition.
However, other configuration may be used such as having the RVM
Data and Log information stored as regular UNIX files at a loss in
performance and data security. Also, if more than one Storage Area
Data is needed on a Coda Server (the default directory is called
/vicepa
), the additional storage areas must be
separate partitions (different partition from
/vicepa
the default, initial storage area for data under Coda) and mounted
as
/vicepb
, for example.
The table below shows a possible partitioning of disks on Coda servers with their respective purpose, mount points, typical sizes and consistency check program. Please note that the sizes of these partitions were taken from one of the Coda servers at CMU-SCS: the actual sizes may vary at other installations.
|
Partition |
Storage Purpose | Mounted | Typical Size | Whether fscked |
| hda2 | RootandUser File System | / | 650MB | Yes |
| hda5 | Var file system | /var | 100MB | Yes |
| hda3 | Vice File System | /vice | 300MB | Yes |
| hdc1 | RVM Log | Not | 12MB | No |
| sda1 | RVM Data | Not | 130MB | No |
| sda2 | Coda FS Data0 | /vicepa | 1.6GB | Yes |
| sda3 | Coda FS Data1 | /vicepb | 1.6GB | Yes |
| sda5 | Coda FS Data2 | /vicepc | 1.6GB | Yes |
Currently, the binary and source packages are available via ftp from
( ftp://ftp.coda.cs.cmu.edu/pub/coda / < platform > )
as well as the Coda Project's local Coda servers
(
/coda/project/release/objs
/
<
platform
>
) for CMU
users. The platform layout for finding the Coda binaries relative
to the base directory indicated above is:
linux/ < coda-release > /{i386,sparc,coda-fs-module,SRPMS} # For Linux freebsd/ < coda-release > / < OS_RELEASE_VERSION > /i386 # For FreeBSD netbsd/ < coda-release > / < OS_RELEASE_VERSION > /i386 # For NetBSD
Where
<
coda_release
>
refers to the Coda version and
<
OS_RELEASE_VERSION
>
is the version of the Operating System.
For example, if you want Coda release 4.3.13 for i386 FreeBSD
2.2.5, the correct platform directory is:
fbsd/4.3.13/2.2.5/i386/
. For i386 Linux, it is:
linux/4.3.13/i386/
, and for i386 NetBSD 1.3, it is:
netbsd/4.3.13/1.3/i386/
. For more information on the
layout, please check the document
LAYOUT
available in
the top-level of the locations listed above for obtaining Coda.
However, the following files are needed by the various
platforms:
coda-debug-server- < release > .i386.rpm coda-doc- < release > .i386.rpm coda-debug-backup- < release > .i386.rpm
Note that the Coda Project supports the glibc (GNU C Library) version of Red Hat (5.X and higher). Support for the older Linux standard Lib C is no longer provided although Coda is known to run on many other Linux systems.
coda-server- < release > .tgz coda-doc- < release > .tgz
Currently, Linux, NetBSD and FreeBSD are supported (CMU-MACH is no longer supported). For Linux, the Red Hat Package Management tool is supported. The package system of FreeBSD 2.X and higher as well as NetBSD 1.3+ is now supported.
TIP : Always check the INSTALL. < platform > on ( ftp://ftp.coda.cs.cmu.edu/pub/coda / < platform > ) for last minute changes and updates that have not yet found their way into the manual.
Currently, server excutables for the supported platforms are installed into the following directories:
installs files/binaries in
/etc
,
/usr/bin
and
/usr/sbin
.
installs files/binaries in
/usr/local/etc
,
/usr/local/bin
and
/usr/local/sbin
installs files/binaries in
/usr/pkg/etc
,
/usr/pkg/bin
and
/usr/pkg/sbin
NOTE:
Please make sure your PATH environment variable is
set properly for your platform. That is, ensure FreeBSD has
/usr/local/{bin,sbin}
and NetBSD has
/usr/pkg/{bin,sbin}
in its default search paths. Linux
installs Coda into the default system paths, so Linux shouldn't
need its default seach path modified.
There is one directory used by all platfroms:
/vice
which is used to store the supporting Coda configuration and Coda
databases. It does not need to be in the PATH.
Specific steps to install the binaries and documentation are:
rpm -Uvh coda-debug-server- < release > .i386.rpm rpm -Uvh coda-doc- < release > .i386.rpm
The installation procedure is identical for both platforms (we assume you are using GNU tar which is the default tar for both platforms):
Please note that NetBSD uses a different location for NetBSD packages (/usr/pkg instead of /usr/local). Please make sure /usr/pkg/{bin,sbin} are in your paths under NetBSD.pkg_add coda-server- < OS_RELEASE_VERSION > - < coda_release > .tgz tar zxfv coda-doc- < release > .tgz -C /usr/share
Server setup is similar for all platforms.
The
/vice
directory
will be created by the setup script automatically if it does not
already exist. However, if your root partition does not have enough
additional space (e.g. the SrvLog file can become huge +10MB under
certain circumstances), you will need to create a partition to be
mount as
/vice
, format the partition as normal UNIX
file system, create the mount point
/vice
, add the
partition and mount information to
/etc/fstab
and
mount it by hand
BEFORE
running the setup script.
Alternatively,
/vice
may be a symbolic link to a
directory on an existing partition with enough space but it also
must be created before running the setup script,
vice-setup
. Sub-directories needed under
/vice
will be created by
vice-setup
.
Also note that the setup of an SCM differs in some important
ways from a non-SCM setup. So, when
vice-setup
asks:
Will this machine will be the SCM?
it is very important
this question is answered correctly.
Only one SCM should be setup per Coda Cell and Coda does not support multiple-homing cells at this time.
While
vice-setup
sets up things that are common for
both the SCM and non-SCM servers, the setup of scripts called by
answering yes to the
Is this the SCM?
are:
vice-setup
vice-setup-scm vice-setup-user vice-setup-rvm vice-setup-srvdir
vice-setup
in the order listed above.vice-setup-rvm vice-setup-srvdir
If this is the first (or only) server being setup for a Coda
cell, it must be setup as the ``SCM'' by answering yes to the
question when asked by
vice-setup
. The SCM coordinates
the sync'ing of Coda database and global configuration files needed
to keep track of data stored in Coda. The SCM also coordinates the
authentication of a Coda user.
If you are adding a non-SCM server which keeps copies of databases, but does not distribute them to the rest of the machines comprising a Coda Cell answer no to the question.
Other than the distinctions indicated above, the SCM plays no special role in a Coda cell and can actually be down for a short time without denying service in a multiple-machine cell site. However, is extremely important to only have one SCM
NOTE:
that running a server seriously dips into system
virtual memory. Running a Coda server, a Coda client
and
X11, the Coda Group as observed that a slightly over 64MB of
available VM is needed. The command
top
gives
information on memory, cpu and process usage. Therefore, we
recommend not running a Coda client on a Coda server and only run X
when performing server maintenance if X is convenient to use.
Whether setting up an SCM Coda server or adding a server to an existing Coda Cell, make sure the following items are taken care before running the setup script:
/vicepb
,
additional space must be a separate partition from
/vicepa
.
vice-setup-rvm
script).
/vice
, having alternative measures setup as outlined
in
Before Begining
.
Once you have the above items available, run:
vice-setup
This provides several useful ``canned'' configuration for
setting up a Coda server. It does a number of things ``behind the
scenes'' as well (such as setting up the directory
/vice
if it doesn't exist and populating it with the
sub-directories
backup db srv vol bin
) while asking
the following questions:
This must be _exactly_ eight characters in length. This requirement is due to a bug that hasn't been completely fix yet.
This must be _exactly_ eight characters in length and different that the auth2 token. This requirement is due to a bug that has not been completely fixed yet.
Answering yes to this question creates the file
/vice/srv/STARTFROMBOOT
which must be present for
either
rc.vice
or
vice.init
to start the
server at boot time. Removal of this file will prevent the Coda
server from starting automatically at boot time.
For the first Coda machine being setup in a Coda cell, the answer must be "yes". If you are adding a server to an existing Coda cell, the answer must be "no".
If you have answered ``no'' to the above question, you will be asked for the HOSTNAME of an existing SCM:
If you have answered ``yes'' to the above question, you will see the message:
. and pick up at the question Are you ready to setup up RVM?An initial administrative user "admin" with password "changeme" now exists
If you have answered ``no'' to the above question, you will be asked for the HOSTNAME of an existing SCM:
This question is only asked if the machine is not the SCM. The hostname entered must be that of the SCM which distributes the Coda cells'database and global configuration files.
Answer yes to this question or the setup will abort.
RVM is the Recoverable Virtual Memory used to keep a copy of
Coda Virtual memory on local disk. While this can be a file, it is
strongly recommended that it be a small partition on a dedicated
disk. If you have set aside, for example, /dev/sdc1 under Linux as
the log partition, enter the partitions' device name at the prompt.
If you are setting up a *BSD system, you must enter the raw
partition name. For example, if you are going to use
/dev/sd2e
you must enter the raw device name
/dev/rsd2e
. The maximum template size we provide for
and have tested is 30MB.
The log partition keeps transaction records for Coda volume
updates that may not have been writen to the Coda data partition
yet. We have not extensively tested values above 30MB and don't
recommend more than this. 12M is a good value for a small to medium
sized server. The value may be entered as,
12M
or
12288
for 12 mega-bytes.
Please see section
The Data Partition
for a more
detailed explanation
This allows you to specify a partition or file. We strongly
recommend a partition which can reside with other system
partitions. Remember, if you use a partition, you are using it
"raw" . it does not contain a file system of any kind. Again, if
you are using Linux, you may enter a block device name such as
/dev/sdc1
. If you are using a *BSD system, you must
enter a raw device such as
/dev/rsd2e
.
This specifies the the size of the RVM data partition. It must
be typed exactly as
22M
,
44M
,
90M
, or
130M
. These are associated with
default parameters that must be feed to
codasrv
when
it is started. The following table provides the log size to total
data storage conversion:
22M
is good for up to 500MB
44M
is good for up to 1.0GIG
90M
is good for up to 2.2GIG
130M
is good for up to 3.3GIG
if you have entered erronous entries for RVM log and RVM data partitions, you could damage your system. Be sure you entered correct information!
y
to commit or
N
to
abort.
Older development versions of coda require
/vicepa
to be present. This version does not. You give a name other than
the default we suggest. However, we strongly advise not calling it
/code
as that can be confused with venus.
Once you successfully complete
vice-setup
:
you are ready to start the update server, the update client and the auth server, as well as the fileserver by typing:
/etc/rc.d/init.d/update.init start /etc/rc.d/init.d/auth2.init start /etc/rc.d/init.d/codasrv.init start
/usr/local/sbin/auth2 & /usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv & /usr/local/sbin/updatesrv -p /vice/db &
/usr/pkg/sbin/auth2 & /usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv & /usr/local/sbin/updatesrv -p /vice/db &
One final step is needed to make the SCM server function. Use the root volume you specified in vice-setup in place of your-root-volume. This step creates the volume, which will be mountable by the client software.
createvol_rep your-root-volume E0000100 /vicepa
Once you have done this, check that the server has started by viewing the log:
xterm -e tail -f /vice/srv/SrvLog &
You should start auth2, updatesrv and updateclnt first by typing:
/etc/rc.d/init.d/update.init start /etc/rc.d/init.d/auth2.init start
/usr/local/sbin/auth2 -chk & /usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv & /usr/local/sbin/updatesrv -p /vice/db &
/usr/pkg/sbin/auth2 -chk & /usr/pkg/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv & /usr/pkg/sbin/updatesrv -p /vice/db &
This will cause the new server to first obtain the current
server files such as
/vice/db/ROOTVOLUME
and
/vice/db/VRList
as well as others needed for this
server to participate in the Coda cell. Once these files have been
retrieved, you may start the server manually for the first type by
typing:
/usr/sbin/startserver &
/usr/local/sbin/startserver &
/usr/pkg/sbin/startserver &
Once you have done this, check that the server has started by viewing the log:
xterm -e tail -f /vice/srv/SrvLog &
If the
codasrv
process started correctly, you
should see ``File Server started'', if not you need look at the
messages in the SrvLog to determine what went wrong. If the
messages are not helpful, you should kill the server
(
NOTE:
The server is designed to continue to stay up so gdb
can be attached for debugging):
and then you can restart the server with:kill -9 `cat /vice/srv/pid`
startserver -d 10 &
However, assuming that the server came up, also make sure
updatesrv and updateclnt are running by using either
top
or
ps
.
you need the package bc (binary calculator on the server). you can either obtain from our ftp site or a mirror site containing your operating system and contrib software.
E0000100 is the Volume Storage Group set up for you by vice-setup when setting up an SCM. With more servers you can define other groups in /vice/db/VSGDB.
Once this is complete, you should setup a client machine and run
Venus and point
venus
point to this server. Setting up
a client machine with
venus
is covered in the next
chapter. Once you have tested and verified the Coda server works
from a client, you are ready to adduser more volumes and users!
NOTE:
A current limitation of the current vice-setup-user
script is that the initial admin account needed is assigned a Coda
uid of 500. In order for a client to authenticate a user, that user
must exist with the same corresponding user-id in the UNIX password
file. If you need to change the admin account uid to suite local
uid space, please edit the script
vice-setup-user
,
change the default Coda uid to the needed value and re-run the
script. This only applies if you are setting up the SCM.
Otherwise, if this is a non-SCM server, you might consider setting up a singlely replicated volume on the new server to test it.
The sections contained here describe what
vice-setup
does for you. This information is useful
for those who either wish to customize our default vice-setup
script or wish to have a custom server setup outside the scope of
Coda's setup script..
RVM initialization requires the selection of several parameters,
each of which involve tradeoffs. Although the RVM log and data can
be kept as regular UFS files,
this is not recommended
: Raw
partitions have much stronger data consistency guarantees as well
performance advantages. It is probably best to plan out the RVM
partitions on paper first, taking into consideration both the
effect on RVM performance as well as overall disk usage if you
choose not to use the values provided in the
vice-setup
script.
The size of the log device is based on available space and
issues involving truncation. Larger logs provide a longer
accessible history of operations, are truncated less frequently,
but each truncation will take a longer period of time. Shorter logs
truncate more often, but each truncation takes less time. Log size
is also strongly related to server startup time as well. We use a
90M log size, on a storage size roughly 3.2GIG spread between two
UNIX partitions. (We suggest leaving a little space at the end of
the RVM log partition for safety, as RVM automatically adds about
one extra page to the amount you specify). Our
vice-setup
script provides for several default values,
the largest of which is 130MB. If you decide to follow the
recommendation and use a dedicated partition, creating a 130MB
partition will leave plenty of room to expand even if your initial
use only suggests a size of 22M, for example..
The log is initialized with rvmutl . At rvmutl s prompt, use the command i , and then specify the size of the log segment. In specifying the size, you can use M for megabyte and K for kilobyte. For example, to initialize a log on partition 0g to eight megabytes:
# @B(rvmutl) * i Enter name of log file or device: /dev/sdc1 Enter length of log data area: 22M * q
The data segment contains the meta-data of the system such as volume headers, Coda directories, vmond lists, resolution logs, etc. The size of the data segment depends on the amount of disk space for file data, i.e. the size of the /vicep? partitions. As a rule of thumb, you will need about 3-5% of the total file data space for recoverable storage. We currently use 4% as a good value under most circumstances. In our systems the data segment is 90Meg for approximately 3.2 gigabytes of disk space. By making it smaller, you can reduce server startup time. However, if you run out of space on the RVM Data partition, you will be forced to reinitialize the system, a costly penalty. So, plan accordingly.
In initializing (or reinitializing), you need to pick several
parameters. The first is the starting address of the recoverable
segment in your address space. On our servers we start the RVM
segment at 0x20000000 for both Linux and BSD based systems on Intel
architecture. The second is the amount of space to give the
recoverable heap. The heap will obviously grow over use, so plan
accordingly. Our heap space is 0x1000000. We suggest that for the
last parameters you use 1Meg (0x100000) for the static area, use 80
free lists (or nlists), and a chunksize of 32. These numbers will
work well with the internal structure of the fileserver and are
provided as the default vales in the setup script
vice-setup
.
To perform the data initialization, run the program rdsinit . rdsinit takes two parameters, the names of the RVM log and data devices. For example, to initialize one of our Intel based servers:
# rdsinit /dev/hdc1 /dev/sdb1 Enter the length of the device /dev/hdc1:119070700 Going to initialize data file to zero, could take awhile. done. rvm_initialize succeeded. starting address of rvm: 0x20000000 heap len: 0x1000000 static len: 0x100000 nlists: 80 chunksize: 32 rds_zap_heap completed successfully. rvm_terminate succeeded.
NOTE: the use of the decimal value for the length of the device and the use of hex values for the address and lengths of the next three values.
The update monitor is used to propagate changes to the Coda
server databases to all servers from the SCM. Client update
processes run on all the Coda servers and connect to a server
udpate process running on the SCM. The server process uses the file
/vice/db/files
to determine which files should be kept
consistent on all the servers. See the
updateclnt
(8) and
updatesrv
(8) man pages for more details.
Create the file
/vice/db/files
on the SCM.
Currently our
/vice/db/files
looks like this:
VLDB auth2.pw auth2.tk pro.db servers hosts vice.pdb vice.pcf volutil.tk VRDB files VSGDB dumplist scm ROOTVOLUME
Coda uses an authentication database that is separate from the UNIX password file. This database is maintained by the SCM. When someone authenticates to Coda, their password is checked against this database and that person is issued a token if they successfully authenticate. This section describes how to initialize the authentication database.
If the server you are installing is the SCM, you must set up the
initial authentication database. Make sure you add at least one
user to the
System:Administrators
group. You need to
do the following:
user.coda
can be
found in the directory
/vice/db
as well as in Appendix
XXX
.
user.coda
file to the group
System:Administrators
. An example of
group.coda
can be found in the directory
/vice/db
as well as in Appendix
XXX
.
/usr/sbin/pwd2pdb -u user.coda -g group.coda
>
vice.pdb
/vice/bin/pcfgen vice.pdb
which will create
vice.pcf.
uid
<
TAB
>
actual cleartext
password
<
TAB
>
any desired info
. The information field
could be the full name of the user. Name this file
/vice/db/passwd.coda
. Then run the command
initpw -k "drseuss "
<
/vice/db/passwd.coda
>
/vice/db/auth2.pw
This command produces the file
auth2.pw
in the format:
uid
<
TAB
>
encrypted
password
<
TAB
>
Desired info
. Note: "drseuss " is an
encryption key to prevent accidental disclosure of the passwords to
the system administrators. In future versions of coda we will store
a sites specific encryption key in a configuration file. Note that
all future changes of the password must be made through the au(1)
program, and not by repeating this procedure since that would
overwrite any passwords changed by users after initialization. New
passwords must be
exactly
8 characters long. To add
additional users, please see Section
XXX
.
/vice/db/auth2.tk
.
/vice/db/volutil.tk
, used
for secure communication among the servers. This file should
contain a single word which will be used to encrypt messages
between the server and server applications. The file should be
present on all servers. The UpdateMon presently takes care of
propagating them from the SCM.
On servers other than the SCM, the
Update
monitor will
ensure that the proper database files are propogated from the SCM.
However, you must copy the file
/vice/db/volutil.tk
to
the new server for the
Update
monitor to work.
Every combination of servers setup expects to use as a VSG which
must be given a multicast address in
/vice/db/VSGDB
.
This file must be created by hand; please refer to
VSGDB (5)
in Appendix
XXX
for more
details.
We have provided a script called
startserver
which
reads in the file
/vice/srv.conf
which contains the
startup information needed by
codasrv
. The syntax of
/vice/srv.conf
is:
The-rvm < log_partition > < data_partition > < size_of_log_partition >
vice-setup-rvm
contains the canned default value
for the corresponding value entered through
vice-setup
. Should you deviate from one of these
canned default values, you will need to either create or modify the
/vice/srv.conf
file to indicate the correct data size
of the data partition. For example, one of CMU's Coda servers
contains:
-rvm /dev/hdc1 /dev/sda1 119070700
You should be careful to specify the correct partitions for the RVM log, data segment, and size of the data segment.
Once this file has been correctly edited, you can start the server with startserver & .
If you are installing any server other than the SCM, please use the instructions in section XXX to create new volumes on this server.
The root of the Coda file system must reside on the /vicepa partition for correct functioning. Since the SCM probably also acts as a server, place the root of the Coda file system in the /vicepa partition on the SCM. Create the root volume by using either one of the following commands
% createvol_rep codaroot
<
VSG_ENTRY
>
/vicepa
% createvol codaroot
<
hostname
>
/vicepa
the former is for replicated root volume, where
<
VSG_ENTRY
>
is the entry in the
VSGDB
for the
volume storage group for this volume; the later is for
non-replicated root volume, where
<
hostname
>
is the name of
the host which hosts the root volume. Since it is likely that the
root will be cloned and read-only replicated, you might wish to
create this volume as singly replicated for simplicity.
These commands will create the
/vice/db/VLDB
,
/vice/db/VRDB
,
/vice/vol/VolumeList
,
/vice/vol/BigVolumeList
, and the
/vice/vol/AllVolume
files.
Create or modify the file
/vice/db/ROOTVOLUME
to
contain the volume name (E.g.
codaroot
) corresponding
to the root volume as listed in the file
/vice/vol/VRList
.
You may wish for the root to be readonly. To do this, first
create a root volume as above. Start a venus which can talk to this
server. Setup the directory structure you want in your root volume.
Once you are satisfied with the structure, clone the volume and
dump it to a disk file with the the
volutil
program.
Now restore the dump file to the servers in the desired VSG for the
readonly volume. Be sure to specify the volumeId and the volume
Name for the new volume:
% volutil restore
<
filename
>
/vicepa
<
newrootname
>
<
volid
>
Then modify the
/vice/db/ROOTVOLUME
file to contain
the new rootvolume name,
<
newrootname
>
.