Francis Yan and Kexin Rong
The increased popularity of cellular wireless network has opened up new opportunities to networking research, since cellular networks behave quite differently from the traditional wired infrastructures as well as wireless LANs.
One particularly important and challenging type of content to serve on the LTE networks is conversational video, which requires both low delay and high throughput. To this end, IETF and W3C have made efforts to standardize conversational media over web browsers, and new transport protocols such as Sprout  have been designed to improve video conferencing services over LTE networks.
Original Goals of the Paper
Wireless access such as LTE can experience a drastic change in channel bandwidth in a very short time frame. This means that an effective congestion control algorithm should be able to quickly adapt to rate changes and operate over a wide range of bandwidths.
The original paper describes SCReAM (Self-Clocked Rate Adaptation for Multimedia), a rate adaptive, delay-based congestion control algorithm devised mainly for conversational video services under LTE environment . The goal of SCReAM is to provide low latency and packet loss, ability to compete with other traffic and stable bit rate. A C++ implementation is available on Github.
The authors first demonstrated the adjustable delay target, a feature from which SCReAM benefits when competing with other flows, e.g. typically TCP flows. They also ran SCReAM against a rate based congestion control algorithm GCC (Google Congestion Control)  under a dynamic LTE simulation. Experiments showed that SCReAM outperformed GCC because the former can better handle high load levels and react promptly enough to congestion to avoid delay spikes.
Subset Goals and Motivation
We are interested in reproducing Figure 3 from the original paper. We chose this figure because it not only demonstrates the advantage of making delay target adaptive in the algorithm, but also presents the performance, e.g. throughput and latency, when SCReAM is running alone. Specifically, in Figure 3, a competing FTP transfer starts at 16s and completes at 23s, sharing the same 1Mbps bottleneck link with SCReAM video flow. It can been that by raising the delay target, the SCReAM video flow can still get a share of the bottleneck bandwidth.
Figure 1: Figure 3 from original paper (figure to reproduce)
The original SCReAM’s implementation contains two components: one component includes the SCReAM server and client classes that provide API for any potential use, and the other one includes their experimental code along with a few support classes.
However, it is notable that their experiment is a pure simulation. We decided to make the following modifications to their source code to make it more “real” and convincing.
- Replace simulated link with real UDP sockets: In the original implementation, the sender and receiver are actually NOT talking to each other via UDP sockets – the communication (“Tx channel” in Figure 2) was simulated by a queue class. We replaced the simulation by real UDP sockets, so that the SCReAM server and client run in simulated as well as real network environments.
- Introduced real timestamp: Following the line of simulated links, the original implementation also uses simulated timestamp, which is simply an integer counter shared by the sender and receiver. To send actual UDP sockets, however, we need to have real system timestamp for the packets.
Figure 2: Overview of the self-clocked rate adaptation framework
We were able to partially reproduce the results from Figure 3. First of all, we adopted all the default parameters in their server and client API and conducted the experiment. Specifically, we ran a pair of SCReAM server and client first, and then ran a pair of iperf TCP server and client as the competing flow from 15s to 18s, which actually turned out to be most consistent with the original Figure 3. They all share the same 1Mbps bottleneck link, which is created by Mahimahi . We generated Figure 3 as below using the log printed by program.
Figure 3: Reproduced figures using the default parameters in SCReAM API
It can be seen that the delay target is not adjusted at all as the paper claims. We explored the reasons by diving into SCReAM API, and finally we found two parameters, the variance and skewness of OWD (one way delay), in API are not general enough to work in different environments or handle different types of competing flows. We noticed that the two parameters are assigned with constant values in the source code, so that they are only working in their specific simulated environment but not in our real experiment.
Therefore, we had to modify the two parameters so that the delay target can be adaptive as they claimed. We ran the experiment again, and got Figure 4 which is closer to their results.
Figure 4: Reproduced figures by modifying relevant parameters in SCReAM API
Note that our SCReAM flow matches the performance in the original paper’s experiment before the TCP flow starts (0-15s), but experiences much higher delay and smaller throughput with the presence of a competing flow. This is because although in our experiment the congestion window size (CWND) increases with the increase of delay target, the increase is still not enough for the SCReAM flow to be able to compete with the TCP flow.
Since we have already seen that the constant parameters do have important effects on the performance, we speculate the difference is still partially caused by them, or broadly speaking, by their method to determine if there is any competing flow and how to adjust congestion window based on that. Another possible reason is that the target bitrate of SCReAM is only about 1Mbps, which limits the throughput performance. We provide more discussion in the Critique section.
The main challenged for reproduction was due to the readability of SCReAM’s source code. Unfortunately the source code is not very documented, which ended up causing us a lot of debugging time. For example, for a while our SCReAM flow was stuck at a low throughput; we later found out that it was because our parameter setting on feedback interval ended up limiting the target bitrate, the logic behind which was totally not obvious.
Instructions for Reproduction
There are two options for reproducing the results:
- Amazon EC2: Search for an Amazon EC2 AMI called “CS244-2016-SCReAM” in the US West (N. California) cluster. The image contains all the setup needed for the experiment.
- Manually: We provide detailed setup instructions for Ubuntu 14.04 in the following subsection.
1. Install Mahimahi following instructions here
2. Install iperf, numpy, matplotlib
sudo apt-get install iperf python-numpy python-matplotlib
3. Get the experiment code and build from source
git clone email@example.com:yuyan0/scream.git cd scream ./autogen.sh && ./configure && make
Generating the Results
To generate the results, you would need to open two terminals. Start the SCReAM and iperf servers in the first terminal:
cd scream/scream/mahimahi-test ./run-scream-iperf-servers.sh
In the second terminal, start the SCReAM and iperf clients:
cd scream/scream/mahimahi-test ./run-scream-iperf-clients.sh
The scripts will start an iperf TCP server and SCReAM server simultaneously, and then start a SCReAM client in another shell. After 15 seconds, an iperf TCP client will start as the competing flow, sharing the same 1Mbps bottleneck link with SCReAM flow. 40 seconds later, the SCReAM program will print log to scream.log.
You can then generate the plot by running:
The plotting script takes scream.log as input and generate three figures: delay.pdf, bitrate.pdf and cwnd.pdf.
To see the performance of a single SCReAM flow, in one shell run
In another shell run
When the server and client are done, generate the results by running
which will make Mahimahi output the delay and throughput graph. You can also run
to generate the three figures from SCReAM’s log file scream.log. This is a good way to check that SCReAM’s log is consistent with Mahimahi’s results.
One major realization we had during the implementation was that the target bitrate for SCReAm is set to be around 1Mbps. The target bitrate is affected by a few factors:
- As the author mentioned in the paper, the delay target is typically set to 50-100ms. To achieve such low latency, the protocol can not shoot for too high a throughput.
- Upon receiving RTP packets, the receiver sends back RTCP packet (feedback) to sender with pacing. Specifically, pacing is determined by a parameter that limits the minimal interval time of sending such packets. It turned out that through some obscure logic, this parameter also limits the target bitrate.
- In the original implementation, target bit rate is also controlled by a number of predefined constants such as the allowed range and allowed rate of increase.
One could argue that 1Mbps is a reasonable target given the current state of LTE network. The author also hinted at the number in the experiment set up of Figure 3 – the shared bottleneck link has capacity of 1Mbps. However, he could have been more explicit about the assumptions.
 Ingemar Johansson. Self-clocked rate adaptation for conversational video in lte. In Proceedings of the 2014 ACM SIGCOMM Workshop on Capacity Sharing Workshop, CSWS ’14, pages 51-56, New York, NY, USA, 2014. ACM.
 Stefan Holmer and Harald T. Alvestrand. A Google Congestion Control Algorithm for Real-Time Communication on the World Wide Web. Internet-Draft draft-alvestrand-rtcweb-congestion-03, Internet Engineering Task Force, March 2013. Work in Progress.
 Keith Winstein, Anirudh Sivaraman, and Hari Balakrishnan. Stochastic forecasts achieve high throughput and low delay over cellular networks. In Presented as part of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), pages 459-471, Lombard, IL, 2013. USENIX.
 Netravali, Ravi, Anirudh Sivaraman, Keith Winstein, Somak Das, Ameesh Goyal, and Hari Balakrishnan. “Mahimahi: a lightweight toolkit for reproducible web measurement.” In ACM SIGCOMM Computer Communication Review, vol. 44, no. 4, pp. 129-130. ACM, 2014.