CS244 ’13: Mosh

Kanthi Nagaraj and Emily McMilin

Original Problem

SSH is a very popular application for maintaining a secure log in at a remote server.  However, SSH is implemented on TCP, which has some frustrating implications including high latency for often very small (single character) payload traffic and no roaming between IP subnets.

Mosh was introduced to solve these problems.  Mosh is a remote terminal application that supports intermittent and low latency connectivity by using State Synchronization Protocol (SSP) and by speculatively echoing user keystrokes. SSP, which is also introduced in the Mosh paper, is a lightweight datagram protocol that synchronizes the state of objects between a local node, which controls the object, and a remote host that may only be intermittently connected.

We describe SSP here since we implement SSP in this project (and not the speculative echo feature). SSP is organized as two layers:

(a) Datagram layer: this layer accepts a payload from the transport layer, prepends a sequence number, encrypts the data, and sends the data in a UDP datagram. Among other things, it is responsible for estimating the RTT of the link.

(b) Transport layer: this layer sends packets at “frame rate” to the receiver depending on the RTT calculated by the Datagram layer. This ensures that the buffers do not fill up and increase latency.

The transport layer is responsible for updating the receiver to the current state of the sender. It does this by sending an “instruction”, which lists the local node and remote host states, and a “diff” between these states. The contents of the “diff” is used to synchronize the state maintained at the local node with the state maintained at the remote host.

Motivation: Why is the problem important/interesting?

One of the most desirable characteristics of a remote terminal application (such as SSH) is low latency.  This is because the payload will often be a single character, and the state at the local node will hang until this payload is received.  SSP introduces TCP-like features a top UDP, to take advantage of UDP’s more flexible and lightweight transport protocol.  In addition to introducing SSP, the Mosh paper also introduces other interesting features such as roaming and keystroke prediction (to further reduce latency) that we do not explore in this experiment.

Almost all of us have experienced frustrating episodes where high latency has rendered SSH almost useless, however without a better alternative to SSH, we must adapt to a “type… pause… backspace… pause…” behavior.  Any proposed solution to SSH latency is very exciting!

Results: What did the original authors find?

The authors have compared the performance of Mosh vs. SSH, both with and without Mosh’s predictive echo feature.

With predictive echo, on a real Sprint EV-DO link, the authors played back 9,986 keystrokes recorded from different users. The results can be summarized as follows: over 70% of the time, Mosh response times were instant (locally predicted) and the rest of the time, Mosh response time distributions are similar to that to SSH.

The authors also evaluate Mosh response times, without keystroke prediction, as we have done here.

Subset Goal: What subset of results did you choose to reproduce?

We choose to emulate the results from the Mosh paper that tested SSP without the keystroke prediction.  Specifically, the table below from the paper evaluates the delays of Mosh vs SSH on a link with packet losses.  The delay is defined as the response time that occurs between a user keystrike event and the terminal key-display (or key-associated action). The authors emulate a link using netem with 29% probability of i.i.d packet loss and an artificial RTT of 100ms to obtain the following results:

	Median	Mean	StDev
SSH	0.416s	16.8s	52.2s
Mosh	0.222s	0.329s	1.63s

This table demonstrates that Mosh has a lower median and mean response time than SSH and also the response time doesn’t vary wildly with Mosh, as it does with SSH.

Subset Motivation: Why that particular choice?

We chose to recreate the table demonstrating the resilience of Mosh to packet losses, without the use of the application layer predictive echo technique. We find this more exciting because SSP incorporates lower layers in it’s protocol suite that employs application layer framing and delay based rate control, which are generic techniques. If these technique holds promise, then they can be used in other protocols as well. We wanted to understand SSP better.

 Subset Results: How well do the results match up? Explanation if you see significant differences.

We conducted our experiment in Mininet with set of parameters that are similar to what is published. We collected our own keystroke logs using the code provided by the authors to do the same. We emulated a link with 50ms delay and 29% loss rate and played back these keystrokes over the link.

The results from our emulation of published response time are:

	Median	Mean	StDev
SSH	0.159s	5.9s	23.8s
Mosh	0.160s	0.188s	0.174s

Our results follow the general trend of the results reported in the paper. However we note that both Mosh and SSH have median performance of about 33% and 50% better, respectively, in our Mininet testbed.  Furthermore, the standard deviation and mean values for both Mosh and SSH are considerably lower in Mininet, suggesting that a long tail distribution of outlier RTTs may not be captured in our emulation.  One possible explanation for this is that our relatively small dataset of  2966 keystrokes allowed for fewer opportunities of a very rare outlier event, as compared to the 9,986 keystrokes in the Mosh paper.

Challenges: What challenges did you face in implementation?

Mosh is open sourced and the code, along with the test scripts, are available online.  The Mosh authors were very responsive to our questions and initial feedback.

One of the main challenges was to measure the keystroke latency. Upon our inquiry, the authors provided us links to their test scripts for recording and replaying keystrike events in the terminal. The part that took most time was figuring out the test scripts, which involved several stages, including synchronous use at both the client and remote server. The test scripts weren’t documented and we had to read through the code to figure out what was going on.

Critique: Critique on the main thesis that you decided to explore – does the thesis hold well, has it outdated, does it depend on specific assumptions not elucidated in the orignal paper?

The main thesis of the paper holds well.  We note that there may be some outlier events of very high RTT times that we were not able to capture, however the trends in our Mininet emulation strongly support the trends found in the Mosh paper.  We also found that in cases of low loss, Mosh has slightly higher (though imperceptible) median response time than SSH.  This slight difference from the publication’s results could be due to any “tuning” of the SSP algorithm that occurred after the publication of the paper.

Extensions: Any exploration outside the results in the paper?

To better understand how link characteristics effect Mosh performance, and whether there was a cross-over point at which SSH performed better, we explored the behavior of SSP under different characteristics of the link. To do this, we emulated different delay and loss values for the link between server and client, and measured the response times for Mosh and SSH over these links.

These four plots below show the mean and the standard deviation of the response times.

In these two plots above we held the RTT constant at 100ms, while we increased the link loss rate (one way) from 0 to 29% (29% one-way link loss rate corresponds to 50% packet loss on TCP). The top plot shows that the mean response of time of Mosh doesn’t change significantly at all while the SSH mean response times reach perceivable levels (1 second) at around 20% link loss rate.

The cause of the high mean response time seen for SSH can be inferred from the high standard deviation also seen for SSH, in the bottom plot. This high standard deviation in combination with the low median response times seen in the table, suggestions that although TCP generally performs well, when multiple loss events occur TCP may recover very poorly and have very large outlier response times.  SSP is able to perform well when a loss event occurs, despite an increase in losses.

In these two plots above we held the link loss rate (one way) constant at 10% while we increased the RTTs from 10ms to 200ms. As expected, we see that under increasing RTTs, both TCP and SSP have increasing response times.  The top plot shows that the mean delays with Mosh are close to 2X better than SSH.  The bottom plot shows that the standard deviation for both Mosh and SSH increases roughly linearly with increasing RTT, however SSH increases at a greater rate.

Platform: Choice of your platform, why that particular choice and how reproducible is the entire setup, what parameters of the setup you think will affect the reproducibility the most?

We used Mininet to recreate these results. One large benefit of Mininet is ease of use. We felt that we didn’t require an elaborate setup to recreate the results from the paper. A simple point-to-point link, in which we can control the characteristics of the link, would do. In Mininet, it is easy to create and control parameters of a link. Also, since Mininet provides isolation between hosts, we used multiple simultaneous links to run tests in parallel.  This was very helpful because the replay of about 3000 keystroking over a lossy and high latency link can be very time consuming.

README: Instructions on setting up and running the experiment.

Note that due to the very high RTT and thus program completion time on links with characteristics of one-way loss at 29% (two way loss at 50%), we are have set the default parameters to be a one-way loss at 20%.

# Log into a new instance of using the CS244-Win13-Mininet AMI 

# Please use our class’s established default settings of
# c1.medium and security setting of quicklaunch-1.
#
# Note that for more rapid program completion time
# this test is run on a sub-sample of our keylog file.
# (49 of the of total 2966 keystrokes logged)

# Use -Y for x11 if CLI is desired
ssh -Y ubuntu@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com

# Now you are in the EC2 main terminal
# Clone mosh_with_mininet
git clone https://emcmilin@bitbucket.org/emcmilin/mosh_with_mininet.git

# Run set-up
sudo ./mosh_with_mininet/scripts/run_setup.sh

# Run the code
sudo ./mosh_with_mininet/scripts/run.sh

# Please note that we set the default loss rate to 20% in order to
# greatly reduce overall completion time. However we held the
# default delay to 50ms.
#
# Any other loss rate can be selected by opening file:
#
# mosh_with_mininet/scripts/run.sh
#
# and editing line 2, to pass your selected one way loss rate (x)
# as shown below:
#
# sudo python ~/mosh_with_mininet/scripts/mosh_emulate.py
#                     –delay 50 –loss x
#
# With delay set to 50ms, results for link loss rate (one way) varied
# from 0 to 29% can be compared to our top two charts.