ietf
[Top] [All Lists]

TCP bandwidth usage was: Yahoo is not using ESMTP

2004-11-15 22:51:06
Fred Baker wrote:
But frankly, every sample I have seen in the past decade placed
SMTP in the "noise" category, a single digit percentage or less.

There might be a handful of exceptions (people in the middle of nowhere
with a dial-up worth of bandwidth and/or relying exclusively on pricey
satlink with a currency not worth anything, etc) but elsewhere this is
what I hear too.


Estimates I have heard from ISPs and seen on measurement sites
available to me have consistently placed HTTP/HTTPS at better
than 2/3 of total traffic in the late 1990's, and in the past
few years have either estimated HTTP/HTTPS as neck-and-neck
with file sharing traffic or have put file sharing well ahead.

You forgot FTP, but I also concur. The reason I am following on this the
part of file sharing traffic: a recent entry in the list of "big" p2p
protocols is BitTorrent. Figures of some 30% of the total Internet
bandwidth being used by BT were discussed recently. Read ahead.


Contrary to the other big ones (eDonkey(eMule), DirectConnect(Kaaza),
WinMX(WinMX), Gnutella(Shareaza)), BT has been underestimated by many
people (beginning with me :-). It's not as easy as classify traffic as
it used to be.

BitTorrent is becoming _the_ choice of hardcore file swappers with a
darn good reason: download speed is basically unlimited. I did not
believe until I actually installed it at home, and short of a good
distributed probe not available yet, here it is on a $37/mo residential
3mbit down / 384kbit up aDSL:
http://home.pacbell.net/arn-py/photos/bt_bandwidth.jpg
Fast enough? (this requires downloading from anything between 50 and 100
peers simultaneously).
 

However, to get this kind of speed there's a catch, and this catch is
called private trackers and private torrents. Generic torrent sites such
as suprnova are not too shabby, but can't compare in terms of speed with
private trackers (a generic torrent will rarely yield more than
50kByte/s). Basically, what we have failed to see coming is a large and
growing number of private tracker sites (a lot being based on
ByteMonsoon) that both a) require a custom port number and b) enforce
UL/DL ratios. 

What's annoying with these:

a) custom port numbers (read rationale below)
Before, life was simple:
Direct Connect 411 - 413
Kazaa 1214
eDonkey 4662
Gnutella 6346 - 6347
WinMX 6699
(with some minor and obvious variations, such as 5662 for
some eDonkey variants)

Now, you can forget about port-based measuring. Even though BitTorrent
is supposedly 6881 - 6889, most private trackers have something like
this clearly stated:

Why do I get a "rejected by tracker - Port xxxx is blacklisted" error?
Your client is reporting to the tracker that it uses one of the
default
bittorrent ports (6881-6889) or any other common p2p port for incoming
connections.
joeblow's tracker (name altered to protect the guilty) does not allow
clients to use ports commonly associated with p2p protocols. The
reason
for this is that it is a common practice for ISPs to throttle those
ports (that is, limit the bandwidth, hence the speed).
The blocked ports list include, but is not necessarily limited to, the
following: Direct Connect 411 - 413 Kazaa 1214 eDonkey 4662  Gnutella
6346 - 6347  BitTorrent 6881 - 6889 
In order to use use our tracker you must configure your client to use
any port range that does not contain those ports (a range within the
region 49152 through 65535 is preferable, cf. IANA). Notice that some
clients, like Azureus 2.0.7.0 or higher, use a single port for all
torrents, while most others use one port per open torrent. The size
of the range you choose should take this into account (typically less
than 10 ports wide. There is no benefit whatsoever in choosing a wide
range, and there are possible security implications). 
These ports are used for connections between peers, not client to
tracker. Therefore this change will not interfere with your ability
to use other trackers (in fact it should increase your speed with
torrents from any tracker, not just ours). Your client will also still
be able to connect to peers that are using the standard ports. If your
client does not allow custom ports to be used, you will have to switch
to one that does.
Do not ask us, or in the forums, which ports you should choose. The
more random the choice is the harder it will be for ISPs to catch on
to us and start limiting speeds on the ports we use. If we simply
define another range ISPs will start throttling that range also.
Finally, remember to forward the chosen ports in your router and/or
open them in your firewall, should you have them. See the Why am I
listed as not connectable?  section and links therein for more
information on this.

In short: as of today, forget port-based stats. As of tomorrow, tough
luck with stats at all as the traffic not only being on a random port
will also be encrypted so you can't inspect the contents of the packet.


b) ratios:
Download Ratios Policies
With a download between:
5.0 GB and 7.5 GB you must have a ratio of at least 0.2
7.5 GB and 10 GB you must have a ratio of at least 0.3
Above 10 GB you must have a ratio of at least 0.4
Just remember: Who ever you are, What ever your status is,
How many millions you did donate .. IF YOUR RATIO DROPS
BELOW 0.4 (w/10gigs or more d/l'ed) AND WE SEE NO ACTIVITY
FROM YOU TO GET IT UP OR IF YOU THINK YOU HAVE A RIGHT TO
INSULT PEOPLE OR THIS SITE IN GENERAL.. WE WILL BAN YOU!!***

In short: you can download at 300kBytes per second, _IF_ you leave your
machine open so your buddies can download too. Simple mathematics: this
is unicast traffic so if you don't maintain a 1/1 overall ratio the
system does not work. Corollary: if your upstream is 10 times slower
than your downstream you need to leave your upstream open 10 times as
long as you download.

This is where it kills: non-hardcore file swappers that would otherwise
open their P2P clients only occasionally now leave them on 24/7 to pump
up their UL/DL ratio: "I maintain my ratio above 2.0 at all times, so no
problem if I decide to download that 55 GigaByte double-layer DVD
StarWars 6-episode set plus collateral material over the weekend".

<proverb>
Small torrents make big rivers.
</proverb>

Welcome to file sharing.

Michel.


_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf


<Prev in Thread] Current Thread [Next in Thread>