[Yum-devel] Yum timeout when running against Roxen webserver

[Anders Blomdell]

Hi,

I just run across a yum (or Fedora-6 python-urlgrabber-2.9.9-2 to be more 
precise). The problem is that urlgrabber does this request:

   GET /path/to/some.rpm HTTP/1.1
   Host: www
   Accept-Encoding: identity
   Range: bytes=280-545669
   User-agent: urlgrabber/2.9.9

And this is the response from the server:

   HTTP/1.1 206 OK
   Accept-Ranges: bytes
   Server: Roxen/4 5 78-release1
   Last-Modified: Tue, 01 Nov 2005 14:38:04 GMT
   Vary: *
   Content-Range: bytes 280-545669/55570308
   Date: Mon, 04 Dec 2006 10:24:25 GMT
   Content-Type: audio/x-pn-realaudio-plugin
   Connection: keep-alive

The problem is that urlgrabber tries to read more than the requested amount of 
bytes, and therefore the connection times out. The following patch seems to work:

--- urlgrabber/grabber.py    2006-12-04 14:42:09.000000000 +0100
+++ urlgrabber/grabber.py~   2006-07-17 21:40:53.000000000 +0200
@@ -912,12 +912,7 @@
          def retryfunc(opts, url, filename):
              fo = URLGrabberFileObject(url, filename, opts)
              try:
-                try:
-                   (low, high) = opts.range
-                   amount = high - low
-                except:
-                    amount = None
-                fo._do_grab(amount)
+                fo._do_grab()
                  if not opts.checkfunc is None:
                      cb_func, cb_args, cb_kwargs = \
                               self._make_callback(opts.checkfunc)
@@ -1195,24 +1190,18 @@
          else:
              return (fo, hdr)

-    def _do_grab(self,amount=None):
+    def _do_grab(self):
          """dump the file to self.filename."""
          if self.append: new_fo = open(self.filename, 'ab')
          else: new_fo = open(self.filename, 'wb')
          bs = 1024*8
          size = 0

-       if amount:
-           block = self.read(min(bs,amount-size))
-        else:
-           block = self.read(bs)
+        block = self.read(bs)
          size = size + len(block)
          while block:
              new_fo.write(block)
-           if amount:
-               block = self.read(min(bs,amount-size))
-            else:
-               block = self.read(bs)
+            block = self.read(bs)
              size = size + len(block)

          new_fo.close()

-- 
Anders Blomdell                  Email: anders.blomdell at control.lth.se
Department of Automatic Control
Lund University                  Phone:    +46 46 222 4625
P.O. Box 118                     Fax:      +46 46 138118
SE-221 00 Lund, Sweden