[Yum-devel] [urlgrabber] keepalive broken

[Michael Stenner]

On Thu, Dec 14, 2006 at 02:49:29PM +0100, Florian La Roche wrote:
> We are currently at 3.1.0 release plus the keepalive patch from
> Dec. 5th. That level is leaking filedescriptors.
> 
> I've tried again with the above version from 1207 and that one kind of
> ignores keepalive and creates new connections, but also does not leak fds.
> So it opens new connections but then does not break apart. One way to
> still see leaking fds in this version is to reference a non-existing file.
> This will then still stop after running beyond the 1024 fd limit:
> 
> #!/usr/bin/python
> import urlgrabber
> for i in xrange(2000):
>     fd = urlgrabber.urlopen("http://127.0.0.1/mirror/K-does-not-exist",
>         timeout=10.0, retry=3, keepalive=1)
>     fd.read(10)
>     fd.close()

I can't seem to recreate what you're seeing.  I just get a 404
exception.  Lets table this for now.  I need to sort this out with
James.  I THINK the m2crypto stuff completely breaks keepalive and
probably doesn't work anyway.  I appreciate your help and I'd really
like to have you keep hammering on it, but there may be some serious
architectural changes in the near future and it doesn't make sense to
mess with these details right now.

If you need a solid working urlgrabber right now and don't need the
m2crypto stuff, then grab 3.0.0.

					-Michael
-- 
  Michael D. Stenner                            mstenner at ece.arizona.edu
  ECE Department and Optical Sciences Center                520-626-1619
  University of Arizona                                         ECE 524G