[Yum-devel] [UG] urlgrabber https doesn't work with rpm module when fork

[Panu Matilainen]

On 07/30/2013 09:14 PM, Zhigang Wang wrote:
> Hi,
>
> When importing rpm, this test (test_urlgrabber.py) doesn't work:
>
> #!/usr/bin/env python
>
> import os
> import rpm
>
> import urlgrabber
>
>
> def test():
>      option = {
>          'copy_local': 1,
>          'ssl_verify_peer': False,
>          'ssl_verify_host': False,
>      }
>      urlgrabber.urlgrab('https://zhigang.us.oracle.com/vm/vm.cfg', '/tmp/vm.cfg',
> **option)
>
>
> def main():
>      pid = os.fork()
>      if pid:
>          return
>      test()
>
>
> if __name__ == '__main__':
>      main()
>
>
> $ ./test_urlgrabber.py
> [zhigang at zhigang tmp]$ Traceback (most recent call last):
>    File "./test_urlgrabber.py", line 26, in <module>
>      main()
>    File "./test_urlgrabber.py", line 22, in main
>      test()
>    File "./test_urlgrabber.py", line 15, in test
>      urlgrabber.urlgrab('https://zhigang.us.oracle.com/vm/vm.cfg', '/tmp/vm.cfg',
> **option)
>    File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 694, in
> urlgrab
>      return default_grabber.urlgrab(url, filename, **kwargs)
>    File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 1130, in
> urlgrab
>      return _run_callback(opts.failfunc, opts)
>    File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 986, in
> _run_callback
>      return cb(obj)
>    File "/usr/lib/python2.7/site-packages/urlgrabber/grabber.py", line 980, in
> _do_raise
>      raise obj.exception
> urlgrabber.grabber.URLGrabError: [Errno 14] problem making ssl connection
>
> If we don't import rpm, everything works fine.
>
> It may related to this: http://article.gmane.org/gmane.comp.web.curl.library/38760
>
> Maybe we need to fix it in rpm.
>
> This bug is fatal when we use urlgrabber in a forking handler.
>
> This is tested on RHEL6.4.
>
> Please help to give some adverse on fixing it.

Probably urlgrabber and rpm fighting over the same global NSS context. 
Newer rpm versions use a private NSS context with NSS versions which 
support it:
http://rpm.org/gitweb?p=rpm.git;a=commitdiff;h=5ecfdcea4e80e58f133ffddd69130ee8df28b2ac

	- Panu -