[Yum] More on BitTorrent and YUM

[Harnish, Joseph]

>-----Original Message-----
>From: yum-bounces at lists.dulug.duke.edu
[mailto:yum->bounces at lists.dulug.duke.edu] On Behalf Of Michael Stenner
>Sent: Wednesday, June 29, 2005 3:29 PM
>To: Yellowdog Updater, Modified
>Subject: Re: [Yum] More on BitTorrent and YUM
>
>First, when you start a new thread, it would be great if you could
>start a thread :)  Please don't reply to an old message and simply
>change the subject line.  Threaded mail clients will put your message
>smack dab in the middle of some other thread.
>
>On Wed, Jun 29, 2005 at 03:14:01PM -0400, Bill Cox wrote:
>> Integrating BitTorrent into YUM seems like a good idea to me.  The
>> distribution of files with BitTorrent is quite secure and robust.
>
>Secure, robust and ideally suited to certain tasks.  I'm not yet
>convinced this is one of those tasks.  I assure you, we're not
>dismissing the idea out of hand.  We all had serious conversations
>(in person, so no links to point you to) about this a while back.
>
>> Instead of directly integrating BT into YUM, I'd propose creating a BT
>> client/server that acts like FTP.  It'd simply allow users to publish
>> directory trees with the server, and clients could download from the
>> directory tree in BitTorrent manner, sharing file pieces among
>> themselves.
>
>Let me see if I understand you.  You're suggesting some intermediate
>protocol?  Some FTPOBT (FTP over BitTorrent) that would look and feel
>like FTP at both ends, have some translation layer, and do bittorrent
>underneath?  FTP and BT are VERY different protocols and in practice
>it's difficult to make one protocol look like another.
>
>> I'm thinking that we could create scripts for mirroring popular RPM
>> sites by having a low-bandwidth server that downloads them periodically,
>> and making them available as torrents.  YUM would have to be modified to
>> use the FTP-like utility to download packages.
>
>I don't understand what you mean by "FTP-like utility" well enough to
>comment on that.
>
>> So, in summary, I suspect all we really need to integrate BT and YUM is
>> build an FTP-like utility based on BT for pubishing file systems and
>> downloading individual files from those file systems.
>> 
>> Should I work on such a beast?
>
>If you want to pursue this, I'd encourage you to look into how
>yum/urlgrabber currently do the network stuff.  For example, yum
>currently relies heavily on the ability to extract JUST THE HEADERS
>from an rpm on the server.  FTP/HTTP/File can do that, but BitTorrent
>cannot.
>
>					-Michael
>-- 
>  Michael D. Stenner                            mstenner at ece.arizona.edu
>  ECE Department, the University of Arizona                 520-626-1619
>  1230 E. Speedway Blvd., Tucson, AZ 85721-0104                 ECE 524G
>_______________________________________________
>Yum mailing list
>Yum at lists.dulug.duke.edu
>https://lists.dulug.duke.edu/mailman/listinfo/yum

Actually something that might make more sense if have yum able to pull down
multiple packages from multiple repos at the same time.  I think this is
where the "BitTorrent" idea comes from.  Because BitTorrent can download
from multiple sources at the same time.  But it only downloads a file or
three.  

What the focus of this should be (IMHO):

Your workstation has 25 updates.  10 from Extras, 10 from Updates and 5 from
Dag.  

Extras and Updates have X number of mirrors
Dag has a separate X number of mirrors

Why can't I pull one extra package from Extras mirror 1, one extra package
from Extra mirror 2, one Dag package from Dag Mirror 1 and so forth as long
as I have the available bandwidth.

The thing with BitTorrent I think that it manages it's bandwidth usage
(maybe not).  I know me personally thought of BT because of it's ability to
spread it's downloads across mirrors.  And I now understand that it BT
itself is not the answer but I think that thinking along the lines of
threaded downloads might benefit YUM most of all.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.baseurl.org/pipermail/yum/attachments/20050629/dfd3f5e6/attachment-0001.htm