[Yum-devel] Fastest mirror selection plugin

[Michael Stenner]

On Mon, Aug 22, 2005 at 06:14:17PM -0400, Luke Macken wrote:
> I hacked up a select()-based implementation of fastestmirror using the
> asynchronous socket handler from the 'asyncore' module.  This version
> supports both 'timeout' and 'stop_after' options as well.
> 
> If you think this implementation is sufficient, I'd be happy to whip up
> a patch for urlgrabber/yum as well.

My standards are pretty high for urlgrabber patch.  I'm working on one
right now, but here are a few things that I'd like to see done
differently in a urlgrabber version:

 a) this currently only does http, or rather, it tries to connect to
    port 80.  What about ftp, https and file?  how to deal with file?
 b) if a dns lookup returns multiple ip addresses (not common in the
    yum case) then this really just probes the first.  URLGrabber
    doesn't do that, though.  URLGrabber will keep trying until one
    works.  (I'm not referring to failover here - this code is in both
    httplib and ftplib - search on getaddrinfo)
 c) it'll be fooled by a long dns lookup, which probably shouldn't be
    counted as it'll only hit you once at the beginning.  It should
    probably do the dns lookups first, or at least start counting only
    after the lookups.
 d) it will choke on complex-ly formatted mirrors: 
       user at pass:foo.com:8080
 e) it'll choke on proxies

I'm not suggesting that all of these should be addressed in the most
thorough way.  It's probably reasonable (for the plugin, especially)
to have it just say "hey use this if you've got XXX" and then have it
throw a useful error if those requirements are violated.  I'll have to
decide how tough I want to be for urlgrabber, though.

Also, for urlgrabber, I have a slight preference not to pull in
asyncore, but I may be convinced on that score.

> http://people.redhat.com/lmacken/fastestmirror-asyncore.py

					-Michael
-- 
  Michael D. Stenner                            mstenner at ece.arizona.edu
  ECE Department, the University of Arizona                 520-626-1619
  1230 E. Speedway Blvd., Tucson, AZ 85721-0104                 ECE 524G