The Fifth Elephant 2013

TCP/IP is known to be a pretty robust, reliable and fair mode of data transport. But what about the actual real throughput when you are transferring GB’s of data on a 10mbps link to cloud which is maybe 15 hops away.

• Do you get that 10mbps upload rate or is it lesser?
• Why is it lesser?
• What are the options?

In this session you will learn about inherent limitations about TCP/IP stack which makes it difficult to use when trying to extract the maximum throughput from a given broadband/dedicated link. You will also learn how things can be made better by using UDP instead and what some of the companies are doing to make this work.

Outline

This is not exactly a storage related talk per se, but it does involve Big Data. Huge amount of data to be transferred to the cloud at maximum throughput. Throughput that cannot be provided using a simple HTTP/FTP upload.

There are tools like Aspera, Data Expedition, File Catalyst etc.. which leverages on UDP protocol instead to provide higher throughput.

We will walk through the limitation of TCP/IP protocol. Look at some data to understand the problem. Talk about various WAN optimizations that people are doing to increase their throughput.

Requirements

• Basic understanding of TCP/IP and UDP.
• Some experience in uploading/downloading GB’s of data from cloud.

Speaker bio

I head the cloud porting team at Amagi. We are creating a cloud based infrastructure for TV broadcaster to manage their services over cloud instead of using a satellite link. This involves transferring gigabytes of data with maximum possible throughput and avoiding any additional latencies if possible.

Links

• http://mayanks.me
• http://twitter.com/mayanks