[Yum-devel] [UG] HTTP POST with Urlgrabber?
William McVey
wam at cisco.com
Tue Oct 25 21:41:34 UTC 2005
I'm using urlgrabber to build a python API to a bugzilla's CGI scripts
(not the most elegant method of interfacing with a bugzilla instance,
but lacking a SOAP or XML-RPC backend to bugzilla, it's what is
available). Anyway, I'm having real difficulty getting urlgrabber to do
POST method http transactions. For example:
$ nc -l -p 9090 # Setup a fake web server to see what gets sent
In another window:
$ python
Python 2.4 (#2, Feb 12 2005, 00:29:46)
[GCC 3.4.3 (Mandrakelinux 10.2 3.4.3-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from urlgrabber.grabber import URLGrabber
>>> grabber = URLGrabber(timeout=10.0)
>>> grabber.urlopen("http://localhost:9090/foo", data="parameter=en&email=foo%40email.com")
And back in the other window:
GET /foo HTTP/1.1
Host: localhost:9090
Accept-Encoding: identity
User-agent: urlgrabber/2.9.6
Documentation at:
http://linux.duke.edu/projects/urlgrabber/help/urlgrabber.grabber.html
indicates that URLGrabber objects as well as urlopen() methods on
URLGrabber objects can take kwargs and that 'data' option:
Only relevant for the HTTP family (and ignored for other
protocols), this allows HTTP POSTs. When the data kwarg is
present (and not None), an HTTP request will automatically become
a POST rather than GET. This is done by direct passthrough to
urllib2. If you use this, you may also want to set the
'Content-length' and 'Content-type' headers with the http_headers
option. Note that python 2.2 handles the case of these
badly and if you do not use the proper case (shown here), your
values will be overridden with the defaults.
Note that *something* isn't being passed through properly to urllib2
since if I execute:
>>> import urllib2
>>> urllib2.urlopen("http://localhost:9090/foo", data="parameter=en&email=you%40yourmail.com")
My netcat shows:
$ nc -l -p 9090
POST /foo HTTP/1.1
Accept-Encoding: identity
Content-length: 37
Connection: close
Content-type: application/x-www-form-urlencoded
Host: localhost:9090
User-agent: Python-urllib/2.4
parameter=en&email=you%40yourmail.com
I tried going through the unittests that are distributed with
urlgrabber, and but I couldn't find any testcases that issue a 'POST'.
Any help would be very much appreciated.
-- William
More information about the Yum-devel
mailing list