[Yum] Yum and Bittorrent

Jarret Raim jarito at lehigh.edu
Thu Oct 21 03:12:12 UTC 2004


Greg Knaddison wrote:
> I have copied some of icon's comments from
> http://www.redhat.com/archives/fedora-devel-list/2004-February/msg01146.html
> 
> +++++++++++++
> 1. Bittorrent is highly inefficient for a large collection of small
> files. You will have to start a separate tracker item for each rpm,
> and for some of them the amount of traffic generated just tracking the
> p2p clients will outweigh the savings of using bittorrent. I would
> imagine that several thousands of tracker items would also be quite
> processor-intensive.
> 2. You have to specifically punch holes in the firewall for bittorrent
> -- not one, but a range of ports, actually. Something most people will
> not do, so they will be constantly leeching.
> 3. Yum runs as root, so you suddenly have a very large amount of code
> (yum+bittorrent libs) listening as root for incoming connections.
> Yikes. Alternatively, you'd have to fork a downloader process and
> communicate with it using some methods. Either way is painful.
> +++++++++++++++++
> 
> For question 1, it sounds like Jarret's implementation is a fairly
> additive and modular one.  It sounds like  Bittorrent is just another
> way to get the file and not a total replacement for ftp/http/file
> based retrieval.  This way, the owner of the reference server can
> decide if a file warrants using bittorrent or not.  For example: new
> version of something big like OpenOfficeorg should be distributed via
> torrent.  New version of "cat", distribute as an RPM direct download. 
> I've also joined the bittorrent list so that I can broach this subject
> there and hopefully ask about the "overly popular file' problem.

Exactly. We envision a program that parses the HTTP logs of a current 
repository and decides which files would be best server via BT. Since 
most updates are small, we are limited to picking larger packages. 
However, updates like KDE, GNOME and Kernel sources are great examples 
of packages that could be served via BT. This would free up the HTTP 
server to do what it does best, serve small files quickly.

> 2 is clearly a problem.  However, it seems like a known drawback
> rather than a showstopper.  First, even with some leechers we would be
> better off than ftp/http only, right?  Second, many people actively
> work as seeds for files that they don't need simply to help other
> users.  I am one such person ;)  The Linux Mirroring Project is full
> of lots more such people and we could certainly get their help as
> well.

We hope to use UPnP (as Azureus does) to help alleviate this, but you 
are correct, swarm speeds will be hurt if most people do not forward 
their ports. We plan to include documentation on how to do this, but 
this is part of what our testing will be designed to gauge. We might 
even be able to convince Fedora to have it as an installation option 
although I haven't thought that one through yet.

> 3 Doesn't seem like a terrible issue to me.  AFAICT Bittorrent has yet
> to have a security problem identified in three years of the project
> running - not a bad record.  I'm not as sure of Yum's history, but I
> imagine it is good as well.  Clearly an audit of the two trees is a
> good idea - fortunately this is a rather active group!

Also, the client uploads do NOT run as root. For the security conscious, 
I would recommend creating a fake user with no rights to run the 
uploader. Right now this isn't possible as the uploader needs rights to 
read the package (not write) but that could be modified.

> How encouraging for me to return from dinner and read that smart
> people have already put good thoughts and effort into this idea!

Thanks. Don't get your hopes too high. Our internal testing is 
promising, but there are problems that will reveal themselves in a large 
public beta. This system is not proven to work....yet :).


Thanks,
Jarret Raim



More information about the Yum mailing list