Coda File System

Coda releases 6.0.16 and 6.1.0

From: Jan Harkes <jaharkes_at_cs.cmu.edu>
Date: Mon, 28 Aug 2006 14:33:19 -0400
Coda-6.0.16 and Coda-6.1.0 are available for download

There are also build related updates for lwp, rpc2 and rvm.

Sources, Debian packages, FedoraCore5 rpms and an updated Windows client
can be found at
    ftp.coda.cs.cmu.edu:/pub/coda/

A lot of 64-bit fixes were merged in 6.0.16, and some of these broke RVM
compatibility for the clients (even on 32-bit systems). Make sure your
client has reintegrated any changes before upgrading. Venus will have to
be started with -init after the upgrade, alternatively you can create an
empty file named INIT in the venus cache directory, depending on
configuration this is likely at either /usr/coda/venus.cache/INIT or
/var/lib/coda/cache/INIT.

It is not required to upgrade lwp, rpc2, or rvm. These only have build
fixes for 64-bit systems. At this point the Coda client and server will
build successfully and run natively on an amd64 or x86-64 machine.
However there are still some issues remaining related to alignment of
character arrays. I did successfully use an x86-64 client to push all
the 64-bit related patches to a repository in Coda, but it died later
during a recursive 'ls' over a large volume. So caution is definitely
still required and the 64-bit clients and server should not really be
considered in any form of production environment.

Coda-6.1.0 is the start of a new and hopefully exiting series. I'll try
to be more agressive (test a bit less), but release new versions more
frequently. At the moment 6.1.0 only adds the rebuilt ASR framework
(application specific resolvers) from Adam Wolbach. He has been
documenting various parts of it on the Coda wiki. Debian users who are
tracking our binary releases with apt, the 6.1 series should be
available for people tracking unstable while stable/testing will stay
with 6.0 for a while.

Jan

ps. I'm still in the process of moving the various 6.1.0 tarballs and
rpms to ftp.coda, they should be available soon.


---------------------------------------------------------------------
The following note is from Phil Nelson about the new Windows release.

This release also includes the beta-2 distribution of the Windows
2000/XP Coda client.  This release is intended to be a short term
release with the beta-3 planned for the end of September 2006.   This
release includes:
    1) bug fixes:  symbolic links should work (but repairs still may not)
    2) new feature:  "big file" support (see below)
    3) newest release of OSR's fsdk that includes some bug fixes.

The major change is for "big file" support.  First, the only
implementation of "big file support" is in the Windows beta-2 release.
This doesn't mean that it won't migrate to the Linux/BSD releases
some day.  The Windows implementation was kind of a quick check out the
concept implementation.  As such, we would like feedback on how it works
for you.

Next, what is "big file support"?  Well, it is assumed that most
venus caches have a relatively small cache, say in the range 500Mb or
smaller.  (The default value for the Windows beta-2 is 100Mb.)
So, how would one be able to put their favorite large media file
in Coda and use it?  Many media files (DVDs) are much larger than a
standard venus cache.  This brings up two problems in most cases:
    1)  the file won't fit in cache
    2)  even if it fit, it takes a long time to move the file to cache.

How does big file support solve both of these?   There is a tool,
mkcodabf, that takes a large file from some place (your media file)
and produces a tree of small files, each hunk is currently a multiple
of 1Mb in size.  The tree could be up to two directories deep.  The
last thing written in the top level directory is a file named
_Coda_BigFile_ that contains the meta data for the directory tree:
    total number of bytes, number of directory levels, hunk size
    maximum hunks (files) per directory, ...

When the Windows kernel module sees the magic file (_Coda_BigFile_),
it returns information that makes the top level directory look like
a big regular file.  The rest of the system and userland just see
one big file.  It is marked read only so that we preserve write
semantics.

Now, a media player can now read and play this big file.  And to
support the media stream, the semantics of the vget upcall to
venus changed just a bit so it now fetches the data into the
cache, but returns immediately after scheduling the fetch.  So,
when the file is opened, the first hunk is fetched and the
process is blocked until the first hunk is in the cache.  After
the first one arrives, the kernel module vgets the next sequential
hunk.  At this point, the process can read from the first hunk.
Once it moves into the next hunk, the following hunk is fetched
via vget.   So, while the media player is playing one hunk, the
next sequential hunk is being fetched by venus so that when the
media player needs the next hunk, it is in the cache ready to be
read and there is no interruption in the data flow.  Random access
is still supported, except the read to a random location may have
to wait for the hunk to be fetched.

Venus doesn't see the directory tree as a single file, just the
users of the Coda file system.

Coda-6.0.16 includes the mkcodabf program as well as a manual page
explaining its use.

---------------------------------------------------------------------
Changes:

coda-6.0.16
    - Fix client crash during repair introduced in 6.0.15
    - Merged 64-bit fixes from Brett Lymn
    - Fix many 64-bit issues in the Coda client, 64-bit seems to be
      somewhat working now.
    - Changes to have the vget upcall fetch the file into cache yet return
    - Try to finalize server-server part of a repair.
    - Try to not cache attributes from individual replicas.
    - Fix libreadline test on FC2 and link libreadline5-dev on Debian.
    - Remove prototypes for unused/no longer existing functions. (Rune)
    - One small fix for coda.spec to fix the init.d file locations on
      OpenSuse. (Jerry Amundson)
    - Increase minimum RVM allocation size. (Jerry Amundson)
    - Compilation fix for coda with MIT krb5. (Maurice van der Pot)
    - Make the test for libreadline find it in /usr/pkg tree if it is
      there. (Phil Nelson)
    - Add the mkcodabf utility. (Phil Nelson)

coda-6.1.0
    - Rewrite of the application specific resolver (ASR) framework.
      (Adam Wolbach)
    - Double stack size for Callback threads to avoid stack overflows
      on cygwin.

lwp-2.3
    - LWP builds and passes simple tests on amd64/em64t.
    - Avoid crash when running lwp_basher.
    - Work around broken makecontext on amd64
    - Fix warning in rw.c
    - Fix some autoconf/automake warnings during bootstrap.
    - On bootstrap, get rid of the autom4te.cache. (Phil Nelson)
    - struct ucontext changed to struct lwp_ucontext. (Phil Nelson)

rpc2-2.2
    - Fix RPC2 binding on amd64.
    - Fix layout of struct SFTP_Parms for 64-bit.
    - Fix MultiRPC packing and unpacking on 64-bit.
    - Convert a couple other longs to int32_t in sftp.
    - Remove the autom4te.cache when bootstrapping. (Phil Nelson)
    - Cygwin does not have IPV6 yet. (Phil Nelson)

rvm-1.13
    - Fix rdsinit interactive mode on amd64.
    - Additional rdsinit fix.
    - On bootstrap, remove the autom4te.cache. (Phil Nelson)
Received on 2006-08-28 14:36:53