There is a frozen state for already sent CML entries, although I believe
  they are `defrosted' in some cases. There is also the issue that the
  shadow copies of files associated with Store CML's are not part of the
  cml entry, but part of the FSO. This means we can currently only have a
  single Store record in our log.

This is unfortunate, since it seems like being flakily connected has
the potential to cause conflicts unnecessarily.  Allow me to ramble
about fixing this:

The client has said

based on VV#1, Store[0]
      causes VV#2
based on VV#1, Store[400]

The problem is that the client doesn't know that the first store
produced VV#2, and thus doesn't list VV#2.

1) A fix might be for the responses from the server with the new VV to
be sent reliably (retained at the server for retransmission the next
time the client wakes up).

2)  Another fix might be for
the server to know that VV1->VV2 was done by client A, and that
therefore client A asking for a store based on vv1 is ok.
This trick doesn't work if clients change IP addresses before
reintegrating.

3) If the Store[0] has a client-generated transaction id, that can be
stored and used as a proxy for the VV, so that the new store can say
Store[400] based on VV#1 as modified by transaction id list {....}

4) Not coalescing the store operations might work if one observes that
if a client does 3 stores, only the last one's _contents_ matter.
However, the first store[0] when retransmitted has to look like a
retransmission of the original (back to transaction ids, but avoiding
having to send a list with the subsequent stores).


  Well, ip-firewalls come to mind as more common introducers of similar
  one-way connectivity.

I suppose, but typically they simply inhibit operation so you give
up.  They do age out state and thus time out connections, but this
problem requires an outgoing packet and then a repeatedly lost
incoming packet for the next 10-20 seconds.

Another observation is that many programs provoke more operations than
seem necessary - I have started running even well-connected machines
with 'cfs wd -age 10' to get some coalescing to avoid this.