Reducing Internet Latency: A Survey of Techniques and their Merits

Latency is increasingly becoming a performance bottleneck for networks, but historically networks have been designed with aims of maximising throughput and utilisation. A goal of this work is to be able to quantify and compare the merits of the different network latency reducing techniques, contrasting their gains in delay reduction versus the pain required to implement and deploy them. To this end we have found that classifying techniques according to the sources of delay they alleviate provided the best insight: 1) the structural arrangement of a network, such as placement of servers and suboptimal routes can contribute significantly to latency; 2) each interaction between communicating endpoints adds a Round Trip Time (RTT) to latency, especially significant for short flows; 3) in addition to base propagation delay, several sources of delay accumulate along transmission paths, today intermittently dominated by queueing delays; 4) it takes time to sense and use available capacity, with overuse inflicting latency on other flows sharing the capacity; and 5) within end systems delay sources include operating system buffering, head-of-line blocking, and hardware interaction. Many of these sources of delay are spasmodic and highly variable. Solutions addressing these sources often both reduce the overall latency and make it more predictable.