[Yum-devel] External Downloader support in URLGrabber

Hedayat Vatankhah hedayat.fwd at gmail.com
Mon Jul 9 18:56:22 UTC 2012


> Hi,
>
>> Actually, I'd like to adopt
>> my yum-fast-downloader plugin with the recent changes and make it an
>> external downloader for URLGrabber. However, currently the external
>> downloader is hard-coded in grabber.py, so the only way to use a
>> different external downloader is to modify grabber.py, or replace
>> /usr/libexec/urlgrabber-ext-down, which are not good solutions since
>> they are part of the .rpm package. Would you please make the
>> external downloader to be configurable using a config file?
> I've looked at y-f-d, and probably understand why you want this..
> Making the external downloader pluggable seems to be generally good
> idea, and rewriting y-f-d from a plugin to a pluggable downloader
> could save/share a lot of code.
Yes, I really don't like "Monkey patching" which is currently used to 
handle metadata downloads. Also, It'll make supporting Presto much 
easier considering your resent changes to it to use async downloader.


>    But I also see some problems:
>
> 1) There's no urlgrabber.conf .. we'd have to make the downloader
> selection part of the urlgrabber API (add and document a new option),
> and add a new config option to yum.conf.  That looks way too hairy,
> considering most users will never touch this (directly, at least).
>
> 2) The urlgrabber-ext-down protocol is quite low level.  Yes, it's
> simple but was meant to be internal and is neither documented nor stable.
> Actually, I was thinking about changing the progress data format a bit,
> to support the 'unknown size' case.  The compat downloader handles
> this fine- size is parsed from headers, and when first data packet
> arrives, the progress object is started with already known size.
> The current ext-down progress API can't do that..
>
> I'd like to have some discussion on 1) first, esp if there are
> any other options (using /etc/alternatives symlinks, using
> environment variables, or whatever else..).
>
> I plan to implement 2) in urlgrabber-ext-down.  ATM it's quite
> low priority (1-2 weeks at best), but the wire protocol will
> change (the progress callback bit).
>
> I've CCd yum-devel.
Number 2 is OK (beside that I thought that supporting "external 
downloader" processes is not really meant to be internal), and for the 
first one, I'm not sure. Using alternatives looks to be good, but maybe 
a better option is also available?

Regards,
Hedayat



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.baseurl.org/pipermail/yum-devel/attachments/20120709/614737b4/attachment.html>


More information about the Yum-devel mailing list