On Mon, Sep 26, 2005 at 12:33:30PM +1200, Jeremy Bowen wrote:
> Has the VSGDB file been obsoleted ? The man pages and the docs still refer to 
> this file but the script vice-setup-scm which is supposed to create it, does 
> not mention it at all.

It has been 'mostly' unused for a while, only the createvol_rep script
was using it. Release 6.0.7 removed the last traces of the VSGDB file.

> What is the preferred method of creating a multiple server
> installation if the VSGDB file is not used ? The docs for
> createvol_rep must be incorrect.

When you run multiple Coda servers on a single machine, they all have to
be associated with a separate ip-address, which would be set up as
aliases on the same interface. The idea is that as far as clients are
concerned, it looks like you have multiple machines connected to the
network (although everything is hosted on a single physical piece of
hardware).

But each unique ip-address/Coda server would have a unique name in the
DNS, and you would use those unique names when creating the volumes.

> The only way I can make it do anything is to invoke it something like:
> $ createvol_rep /shareme mrblue.localdomain/vicepa mrorange.localdomain/vicepa
>
> Is this correct ? 
> (It has an ID of 7f000001)

If mrblue and mrorange (where is mrpink?) resolve to the correct
ip-addresses it should work.

> Also, how do I relate "Realms" to "Servers". I have 
> a /usr/local/etc/coda/realms file on the client which contains: 
> myrealm	mrblue.localdomain mrorange.localdomain

This should allow the client to use,

    /coda/myrealm/

And it should send any volume location lookups to both mrblue and
mrorange. Those servers should respond with things like '/' can be found
on the mrblue server, '/shareme' is replicated on both mrblue and
mrorange, etc.

> Do I refere to it by server-name or realm ? Do I need to make a cfs
> mountpoint for it ? 

You would have to create a cfs mountpoint for shareme. Once you get the
/coda/myrealm/ thing working you simply run 'cfs mkm /coda/myrealm/shareme'.

However, since both of your servers are actually on the same physical
hardware, I'm not entirely sure how useful it is to have replicated
volumes. I guess it would still protect you if a software bug only took
out one server. But with replication, any bug like that would hit both
replicas at the same time anyways. And if they are running on the same
machine, the replication isn't protecting you from hardware or network
failures either. So the only thing it does is add load to the machine
and increases the chances of getting conflicts.

Anyways, let's continue with the current scenario,

> "cfs lv /coda/myrealm" fails with: No such file or directory
> "cfs mkm /coda/myrealm" fails with the error "/coda: File exists"

My guess is that the client is sending queries to mrorange, and it
doesn't have a current copy of the VRDB or VLDB files, or wasn't
correctly notified of updates to these files. As a result it has no
clue what the client is talking about when it asks for the location of
a volume named '/' and returns ENOENT (No such file or directory).

> "cfs lv /coda/mrblue.localdomain" returns:
>   Status of volume 7f000000 (2130706432) named "/"
>   Volume type is ReadWrite
>   Connection State is Connected
>   Reintegration age: 4294967295 sec, hogtime 4294967.295 sec
>   Minimum quota is 0, maximum quota is unlimited
>   Current blocks used are 3
>   The partition has 69703404 blocks available out of 107514268
> 
> "cfs lv /coda/mrblue.localdomain/shareme" results in:
>   Status of volume 7f000001 (2130706433) named "/shareme"
>   Volume type is ReadWrite
>   Connection State is Connected
>   Reintegration age: 4294967295 sec, hogtime 4294967.295 sec
>   Minimum quota is 0, maximum quota is unlimited
>   Current blocks used are 3
>   The partition has 105652880 blocks available out of 107514268

I guess you already created the mountpoint. Coda servers don't really
know anything about realms or replication. So you can actually access
the same 'realm' through different paths,

    /coda/myrealm/
    /coda/mrblue.localdomain/
    /coda/mrorange.localdomain/
    /coda/<ip-address of mrblue or mrorange>/

And all of those should actually show you the identical root volume. The
difference is really to which server(s) we send our volume location
queries. In the myrealm case we send to either mrblue or mrorange,
whichever is available, in all other cases we would only send queries to
one server, and fail when the server happens to be unreachable.

Now one problem in the current implementation is that the client has no
idea that these different 'realms' are in fact provided by the same set
of servers. So it creates multiple local objects to represent each of
these realms. On the other hand, the server has no idea that the client
has multiple views, so it will only send back a single invalidation
message. As a result when a new file is created in the realm, only one
of the realms will correctly receive an invalidation message and see the
new file, all other views on the realm will still show the stale copy.

> "cfs whereis /coda/mrblue.localdomain/shareme" returns:
>   mrblue.localdomain  mrorange.localdomain

So that volume actually is correctly replicated, according to the VLDB
and VRDB databases that mrblue is using.

It could be that you only have to signal the mrorange server that a new
version of the volume location databases is available, Normally the
updateclient process sends this message to the local codaserver, but
that might not work right on a multi-server installation.

    'volutil -h mrorange.localdomain updatedb'

Jan