CS244 ’13: Evaluation of Mosh “mobile shell” performance results

Ahmed Aljunied and aatreya

Introduction

Mosh (“mobile shell”) is a new terminal application designed to be robust to intermittent connectivity, changes in IP addresses, packet loss, and long round-trip times.  For frequent users of terminal sessions, network issues such as these can lead to a very poor user experience via dropped sessions, long delays, and lost data.  Mosh works through the use of a new protocol called the State Synchronization Protocol (SSP).  Rather than sending raw octets from one machine to another that ultimately represent some more complex pattern of information (such as a terminal display), Mosh works to synchronize the client’s model of the terminal display with the server’s model by synchronizing keystrokes and screen updates.  This allows the system to significantly reduce latency and data transfer requirements.  Since the system uses a unique UDP port for each connection, and connection state is tracked based on filtering for authenticated packets rather than checking source IP address, the system also allows roaming.

The goal of our work was to examine the performance and behavior of Mosh under challenging network conditions.  One of the recurring pain points in shell usage is the poor latency and connectivity experience caused by the wide variety of changing network conditions in today’s computing environments.  In particular, we examined Mosh to determine whether it achieves its stated goal of improving keystroke response time over delayed and lossy networks.  We selected this piece to examine because it is one of the more difficult components of Mosh to understand, yet provides great value to the user and is thus a core feature. The Mosh performance results for keystroke response times are presented in Figure 1, below.

Figure 1: Cumulative distribution of keystroke response times from Mosh paper

These results show a clear benefit for Mosh due to the predictive model built into the system. Mosh is able to predict the vast majority of keystrokes, whereas SSH must take a full round-trip for each keystroke to be echoed.

Approach

Since the purpose of Mosh is to enable a more robust and responsive terminal, we use Mininet to model a network with varying network conditions to explore how Mosh handles the challenges.  In particular, we modeled 3G and WiFi connectivity using a host-switch-host topology in Mininet.  We chose Mininet because of the simplicity and configurability that it allows in specifying a range of network parameters, and because the configurable topology allows us to directly compare SSH with Mosh on identical networks.

3G/WiFi modeling

Through a variety of experiments with Verizon 3G EVDO Rev. A service and a typical WiFi connection, we carefully determined a set of representative modeling parameters for Mininet based on observed RTT, bandwidth, and packet loss. We then used these parameters together to create a simulated 3G+WiFi network over which to conduct experiments.

WiFi:
RTT: 30 ms
Loss: 8%
Bandwidth: 25 Mbps

3G:
RTT: 450 ms
Loss: 1%
Bandwidth: 1 Mbps

One factor that is present in 3G networks but is difficult to model is the varying latencies caused by dormant to active radio resource allocation.  When a 3G radio is dormant, it can take significantly longer to begin transmitting data once there is data to send (750 ms typical), as it must request spectrum resources from the network before it is allowed to transmit.  However, when it already has an active channel available, this step is not necessary, resulting in significantly lower delay (145 ms typical).  This means that the delay between packet transmission (which is used to determine radio dormancy for power conservation reasons) will affect the RTT.  This effect is difficult to model effectively in Mininet, so we assigned a network latency close to the dormant latency, as we believe a typical shell usage pattern is likely to exhibit this behavior at the radio level since users will frequently have a multi-second delay between successive keystrokes as they read and parse the results from the previous command.

Replay scripts

We took some time to understand the scripts provided by Keith, the author of the original Mosh paper.  Keith provided us with a pair of scripts: one is used to record terminal sessions to create a dataset, and the other is used to play back the terminal data into an SSH or Mosh session to determine performance results.

The script that records terminal sessions creates a log of keystrokes as well as the output generated as a result of the keystroke. The replay scripts consist of two parts: the client replay script that replays keystrokes to the server and the server replay script that reads the keystrokes and checks that they match those in the log. Certain anomalies that occur during recording may cause a log to not be re-playable, such as varying output.  We had to create many logs under a varying set of conditions to create ones that were reliably re-playable.

OpenVPN

We also compared Mosh against an alternative method of using SSH that alleviates some of the issues Mosh intends to address. The method was using SSH over an OpenVPN tunnel.  This helps address two issues: network reliability and IP address mobility, as OpenVPN will work to maintain a stable tunnel in the background, and will seamlessly continue to do so despite changes in IP addresses at the endpoints, as long as packets are properly routed. We focused on the network reliability component to see how the keystroke response performance of SSH over OpenVPN compares to Mosh and traditional SSH.

Results and Discussion

We recorded a multiple terminal sessions ranging from programming in emacs, editing a file in nano, and typing python commands.  The keystroke delays we experienced were aggregated and compared against the different connection types below, in Figure 2.  As we can see, Mosh was able to predict the majority of keystrokes, as exemplified in Figure 1 from the original Mosh paper.  Note that in our tests, however, we see a large “cliff” in the SSH results, since there was little jitter in the RTT in the Mininet topology.

The different terminal sessions we recorded also represented different types of usage. General typing of text or code yielded good results while code refactoring or sessions that consisted of a lot of editing and reviewing did less well since most keys were then navigational and cut-paste keys that do not contribute to the algorithm.

Therefore we find that the mileage that a user gets with Mosh varies depending on the terminal usage patterns.

Figure 2: Cumulative distribution of keystroke response times

Examining the OpenVPN results, we see that it did not add a significant performance boost over Mosh.  Indeed, OpenVPN does provide a limited benefit in reducing the fraction of packets that are lost that cause reduced performance, and we can see this in the slight improvement in the OpenVPN performance.  However, our results show that the OpenVPN approach is likely most useful for providing the IP roaming capability, as the additional performance benefit is not significant.  Mosh indeed performs quite well in comparison.

Conclusion

We believe Mosh has performed admirably at improving the quality and user experience of terminal sessions.  In particular, we have shown that its keystroke prediction capabilities are in fact useful in practice, which lines up with the results presented in the original paper.  In addition, Mosh’s other advantages seem to make it a compelling replacement for traditional SSH, and we look forward to seeing its adoption increase in the future.

Mosh References

Mosh: the mobile shell: http://mosh.mit.edu/mosh-paper.pdf (Usenix ATC)
http://mosh.mit.edu/

Source Files

Source and README can be found at https://github.com/ahmedaljunied/CS244-Mosh