[Yum-devel] [PATCH] Sort mirrors using TH statistics.

Zdeněk Pavlas zpavlas at redhat.com
Wed Oct 10 09:30:06 UTC 2012


The main use case for this is when mirror #1 is down.  Parallel
downloader avoids such mirror, but legacy mirror group code does not.
This patch brings these in line.

Legacy users that don't set the 'timedhosts' option are not affected
(all mirrors evaluate to default speed and sorting is stable)

caveats: This runs after the fastestmirror plugin (if enabled) has
sorted the mirror list.  When TH file is used and contains enough
host stats, FM order is mostly ignored.
---
 urlgrabber/mirror.py |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/urlgrabber/mirror.py b/urlgrabber/mirror.py
index eeeb5d0..7975f1b 100644
--- a/urlgrabber/mirror.py
+++ b/urlgrabber/mirror.py
@@ -97,6 +97,7 @@ import thread  # needed for locking to make this threadsafe
 from grabber import URLGrabError, CallbackObject, DEBUG, _to_utf8
 from grabber import _run_callback, _do_raise
 from grabber import exception2msg
+from grabber import _TH
 
 def _(st): 
     return st
@@ -261,6 +262,17 @@ class MirrorGroup:
         self.default_action = None
         self._process_kwargs(kwargs)
 
+        # use the same algorithm as parallel downloader to initially sort
+        # the mirror list (sort by speed, but prefer live private mirrors)
+        def estimate(m):
+            speed, fail = _TH.estimate(m['mirror'])
+            private = not fail and m.get('kwargs', {}).get('private', False)
+            return private, speed
+
+        # update the initial order.  since sorting is stable, the relative
+        # order of unknown (not used yet) hosts is retained.
+        self.mirrors.sort(key=estimate, reverse=True)
+
     # if these values are found in **kwargs passed to one of the urlXXX
     # methods, they will be stripped before getting passed on to the
     # grabber
-- 
1.7.4.4



More information about the Yum-devel mailing list