Coda File System

Re: Windows Coda Client, Beta-3 release announcement

From: Eric Toombs <eric.toombs_at_gmail.com>
Date: Sat, 02 Sep 2006 14:34:50 -0400
Phil Nelson wrote:
> There is a new Windows Coda Client now available, the Beta-3
> version.  You may find this this new windows client at
>
>   ftp://coda.cs.cmu.edu:/pub/coda/winnt/ntcoda-b3.exe
>
> This release is a quick kernel bug fix for the beta-2 distribution.
> There are no changes to the other parts of coda.  Userland is
> version 6.0.16.
>
> I would appreciate people trying this release and sending me a report.
> I have found that the VLC media player (http://www.videolan.org/vlc)
> works well with the "big file" support provided by the beta-3 release.
>
> The following is the Windows part of the 6.0.16 release announcement.
> In case you didn't read that far in the 6.0.16 announcement, here
> it is again. (Slightly modified for the beta-3 release.)
>
> --------------
>
> This Windows client release includes:
>     1) bug fixes:  symbolic links should work (but repairs do not work.)
>     2) new feature:  "big file" support (see below)
>     3) newest release of OSR's fsdk that includes some bug fixes.
>
> The major change is for "big file" support.  First, the only
> implementation of "big file support" is in the Windows beta-2 release.
> This doesn't mean that it won't migrate to the Linux/BSD releases
> some day.  The Windows implementation was kind of a quick check out the
> concept implementation.  As such, we would like feedback on how it works
> for you.
>
> Next, what is "big file support"?  Well, it is assumed that most
> venus caches have a relatively small cache, say in the range 500Mb or
> smaller.  (The default value for the Windows beta-2 is 100Mb.)
> So, how would one be able to put their favorite large media file
> in Coda and use it?  Many media files (DVDs) are much larger than a
> standard venus cache.  This brings up two problems in most cases:
>     1)  the file won't fit in cache
>     2)  even if it fit, it takes a long time to move the file to cache.
>
> How does big file support solve both of these?   There is a tool,
> mkcodabf, that takes a large file from some place (your media file)
> and produces a tree of small files, each hunk is currently a multiple
> of 1Mb in size.  The tree could be up to two directories deep.  The
> last thing written in the top level directory is a file named
> _Coda_BigFile_ that contains the meta data for the directory tree:
>     total number of bytes, number of directory levels, hunk size
>     maximum hunks (files) per directory, ...
>
> When the Windows kernel module sees the magic file (_Coda_BigFile_),
> it returns information that makes the top level directory look like
> a big regular file.  The rest of the system and userland just see
> one big file.  It is marked read only so that we preserve write
> semantics.
>
> Now, a media player can now read and play this big file.  And to
> support the media stream, the semantics of the vget upcall to
> venus changed just a bit so it now fetches the data into the
> cache, but returns immediately after scheduling the fetch.  So,
> when the file is opened, the first hunk is fetched and the
> process is blocked until the first hunk is in the cache.  After
> the first one arrives, the kernel module vgets the next sequential
> hunk.  At this point, the process can read from the first hunk.
> Once it moves into the next hunk, the following hunk is fetched
> via vget.   So, while the media player is playing one hunk, the
> next sequential hunk is being fetched by venus so that when the
> media player needs the next hunk, it is in the cache ready to be
> read and there is no interruption in the data flow.  Random access
> is still supported, except the read to a random location may have
> to wait for the hunk to be fetched.
>
> Venus doesn't see the directory tree as a single file, just the
> users of the Coda file system.
>
> Coda-6.0.16 includes the mkcodabf program as well as a manual page
> explaining its use.
>
> --------------  Additional information --------------
>
> When one "deletes" a big file, it only deletes the big file attribute.
> To do that, the coda kernel module deletes the _Coda_BigFile_ file in
> the "big file's directory" and then the full file tree becomes visible
> to the client.
>
>
>   
so will big files be created automatically if a user creates a file
that's bigger than the cache?
Received on 2006-09-02 16:40:24