[Yum-devel] URLGrabber Batch Mode

[Ryan Tomayko]

On Sat, 2004-02-28 at 08:23, David Farning wrote:
> To be truly useful, fail over logic and leach prevention would need to
> be handled within the batchedUrlGrabber. What a pain, how much gain?

Right. That's the question. It's going to be a pain regardless, right?
Is it something we do in URLGrabber so yum and others can take advantage
of it or do we think each app will want to roll it's own mechanisms
(like your thread pool suggestion) for doing this kind of thing? I'm
50/50 right now. Half of me thinks it could be really useful to apps
that don't have anything for forking or establishing concurrent
downloads; the other half thinks it won't be used at all because it will
be implemented at the application layer anyway.

> Also, several users have requested a download resume feature.
> Here is an example.  I found it in a python mailing list a few weeks ago
> can't remember where so I can't give the author credit.
> 
<snip>

Take a peek at byterange.py in urlgrabber cvs. This handles ranges for
http: ftp: and file: urls generically as an extension to urllib2. It can
be used outside of urlgrabber.

- Ryan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20040228/ec0ce4d0/attachment.pgp