(Illustration by Gaich Muramatsu)
Hi everyone, It has been nearly a year and a half since the last message on this mailing list (December 2018, for Coda 7.0.1). Since then we have continued to work on Coda. Here is a summary of the main changes. Apologies for this long email. There is a lot to cover. The biggest change is in security and network communication. Coda now runs over TCP, and uses GNUTLS to provide encrypted tunnels. There is still a fallback mode that operates over UDP as before, and there are some Coda utilities like "clog" that still need to be ported to GNUTLS/TCP. The implementation uses a proxy process at each end called "codatunnel" that encapsulates all knowledge of TCP and GNUTLS. codatunnel connects via Unix domain pipes to Venus and CodaSrv. The latter still think they are running over UDP, and still retain their legacy coroutine-based LWP threading model. The codatunnel process uses pthreads, without any legacy knowledge of LWP/RPC2 etc. Eventually, our plan is to remove UDP support altogether. At that point quite a bit of Coda code can be simplified or deleted. During this transition period, we have preserved full upward compatibility with older UDP-based Coda clients and servers. The homegrown AES based RPC2 encryption layer is active, but of course is redundant inside a TLS-on-TCP tunnel. The UDP fallback can be disabled with the 'tcponly' flag in venus.conf or on the venus command line in which case a client will only connect to servers through a TCP connection after a successfull TLS handshake. In using TLS, we have aimed at minimalism. Because we are not the web, we have intentionally kept the authentication chain infrastructure separate from the existing X509 infrastructure. The Coda hierarchy starts at the top with a long lived CodaCA root certificate, which is used to sign realm-specific authorities. The realm authorities in turn sign short-lived server certificates, where short lived here is in the 1 to 7 day range. We're trying to get the system setup so that realm-specific certificate authorizes are constrained to only sign certificates for specific subdomains and ip-address ranges. IP address ranges are currently still necessary because codatunnel has currently no idea which domain name we are trying to connect to. Another area of progress has been in support for very large files such as videos. We have extended Coda with a concept called "VASTRO" ("Vast Streamed Read-Only Object"). This is a client-side-only concept. The server is not aware that a client is caching a file as a VASTRO. Venus makes the decision based on file size and current bandwidth. As the name implies, a VASTRO is read-only. You can't modify it, but Venus is able to fetch it in pieces to do things like playback of videos. There have also been small incremental changes in areas such as the embedded Plan9 file server in the Coda client, and the mapping of 64-bit timestamps from the Linux kernel module to 32-bit timestamps for Venus. Below the signature line is a more complete list of changes in each Coda release since 7.0.1. Jan can provide much more detail on these changes, as well as the above architectural changes. We would love your usage feedback, bug reports, etc. Our long-term goal is to rip out support for server replication in Coda. That is the oldest part of Coda (originally implemented circa 1987-1989) and was conceived well before disconnected operation was implemented around 1990-1991. It was also long before all our modern knowledge of distributed systems on topics such as Paxos-based failover. We have long believed that the combination of disconnected operation on the client and Paxos-based failover on servers would be a much better architecture. This should lead to a much simpler, more robust, and maintainable Coda code base. Cheers -- Satya ===== SUMMARY OF CHANGES SINCE coda-7.0.1 ===== coda-7.0.2 - A lot of code cleanups - standardized code formatting with clang-format-6.0 - remove unnecessary 'register' annotations in various places - add missing prototypes for RVM functions coda-7.0.3 - Several large object (VASTRO) related fixes - Silence unnecessary warning on file-based pioctls coda-7.0.4 - file-based pioctl improvements - More large object (VASTRO) related fixes coda-7.0.5 - More large object (VASTRO) related fixes - Fix 9pfs server crash - Add a new non-replicated volume type that unlike previous non-replicated types will support reintegration and disconnected operation. coda-7.0.6 - Continued improvement of large object support. - Added support of large partial objects to 9pfs server. - Add lazy allocation for large object bitmaps. - Continued work on the new non-replicated volume type, it now supports file and volume level callbacks and reintegration. - Add support for 64-bit time_t in venus <-> kernel communication path. - Fix the ability to remove large partial files with 'rm'. - Update gcodacon for python3. coda-7.1.0 - 7.0.6 accidentally introduced an RVM layout change and here we bumped the minor release number to signify this. - There also was an incompatible kernel API change, introducing 64-bit timestamps. This is in the mainline kernels and backported to various long term support kernel versions and is detected using a version flag when we initialize the kernel interface. On older kernels it may be possible to build the newer kernel module with DKMS from the linux-coda repository, or the client can be recompiled to revert to using 32-bit timestamps. coda-7.1.1 - Fixes reference count bugs in pioctl files and access intents. coda-8.0.0 - Use GnuTLS to secure codatunnel's TCP connections. coda-8.0.1 - Add helper script to create realm certificate and regenerate short-lived server certificates. - Disable UDP fallback when we fail to validate a server certificate. - Correctly cleanup data structures related to TLS peers. - Add support for older versions of the gnutls library. - Replace Coda root CA certificate with a 2048-bit key that is now stored offline. coda-8.0.2 - Automatically reload certificates when on-disk server certificate file is modified. - Resend packet on EAGAIN errors. - Improve packet sequencing and EOF handling on TLS connections. ====================================Received on 2020-07-05 12:59:42