On Wed, Feb 4, 2015 at 2:22 PM, Phillip Hallam-Baker
<phill(_at_)hallambaker(_dot_)com>
wrote:
Today most Web browsers attempt to optimize download of images etc. by
opening multiple TCP/IP streams at the same time. This is actually done
for
two reasons, first to reduce load times and second to allow the browser to
optimize page layout by getting image sizes etc up front.
This approach first appeared round about 1994. I am not sure whether
anyone
actually did a study to see if multiple TCP/IP streams are faster than one
but the approach has certainly stuck.
They did study it; there was/is a point of diminishing returns... In that
era, the diminishing returns were in part due to insufficient buffering at
dialup modem banks.
But looking at the problem from the perspective of the network it is
really
hard to see why setting up five TCP/IP streams between the same endpoints
should provide any more bandwidth than one. If the narrow waist is
observed,
then the only parts of the Internet that are taking note of the TCP part
of
the packet are the end points. So having five streams should not provide
any
more bandwidth than one unless the bandwidth bottleneck was at one or
other
endpoint.
Most web objects are small. In fact, given even IW4, much less IW10, most
objects will fit in the initial window of packets for that connection.
Note, this means that the increase in IW means that there is increasing
head of line blocking occurring at (most) bottleneck links.
Also note that the web is *not* a bulk data transfer system; it is very
latency sensitive.
If you increase the effective RTT, you kill performance.
On (almost all) TCP implementations today, those packets
from all of those objects
will be transmitted at *line rate* in the data center, just to arrive
effectively simultaneously at the bottleneck near the user (typically the
broadband last mile, or in the customer's home router just before the WiFi
hop), where they cause head of line blocking on those devices, as
almost all
lack any sort of flow queuing.
So you get a burst of possibly up to hundreds of packets (due to web site
sharding) transmitted at the data center arriving *splat* at the bottleneck
link(s).
I've measured transients of *up to hundreds of milliseconds* on 50Mbps
links.
*So TCP congestion control/avoidance has effectively been defeated
entirely. It just isn't in play at all!*
I call this "transient bufferbloat". It is the cause of a lot of the
VOIP/teleconferencing drop outs you hear if you surf the web.
Now there are some parts of the deployed Internet that do actually perform
statefull inspection. But I would expect increasing the number of channels
to degrade performance at a firewall or any other middle boxen.
So we have a set of behavior that seems at odd with the theory. Has anyone
done any experiments recently that would show which is right?
The reason it makes a difference is that it is becoming clear that modern
applications are not best served by an application API that is limited to
one bi-directional stream. There are two possible ways to fix this
situation. The first is to build something on top of TCP/IP the second is
to
replace single stream TCP with multi-stream.
You generally find the size information needed for page layout in the first
*packet* of data. Might be nice if those packets from different flows got
scheduled before the later packets, might it not? Might be nice if other
traffic, such as other TCP opens, or DNS lookups, or your voip packets got
scheduled for transmission before those additional packets from that bunch
of flows started when you visited that crappy web page with tons of little
objects on it (median is
of order
10 objects, btw).
Oh, wait, you say that fq_codel does that? Well, you might consider using
it.
On
any
current Linux system, that is "one line" of configuration. And you might
want to replace your home router too, and use firmware on it that
implements fq_codel.... Life gets much better, indeed.
And you might want to beat up your ISP, as the bottleneck may be your
broadband head-end and not under your control. And ask Microsoft and
Apple when their OS's will do something sensible too....
https://tools.ietf.org/html/draft-hoeiland-joergensen-aqm-fq-codel-00
- Jim