[Yum-devel] urlgrabber: timestamp check

[Ryan Tomayko]

On Mar 9, 2005, at 12:28 PM, Michael Stenner wrote:
> OK, I'm attaching both a patch to grabber.py and a little test program
> to demonstrate how one might do this.  I'd really like input from you
> and Ryan about whether this makes sense.  For the newly defined Error
> number (which is for HTTPError), I've included special "code" and
> "exception" attributes, which seems a little kludgy.

I don't think it's kludgy at all.

> However, I do kinda like returning a bit more information.  Before, 
> the same
> exception got raised as IOError and the code/message were just encoded
> in the string.  That's OK to a user but it makes it hard to do things
> like this.

Yea. Not sure why we didn't do this in the first place.

> The only other way I thought of to solve the problem was to allow
> callers to include custom urllib2 handlers.  That would really allow
> a lot more flexibility, but I'd want to do it right, and that might
> take a little thought.

I added that a while back. Not sure I like the way we did it though:

   opener = None

     Overrides the default urllib2.OpenerDirector provided to urllib2
     when making requests.  This option exists so that the urllib2
     handler chain may be customized.  Note that the range, reget,
     proxy, and keepalive features require that custom handlers be
     provided to urllib2 in order to function properly.  If an opener
     option is provided, no attempt is made by urlgrabber to ensure
     chain integrity.  You are responsible for ensuring that any
     extension handlers are present if said features are required.

Not something for the feint of heart.

> Also note that this patch is currently causing some unit tests to
> fail, which I haven't looked into yet.  I haven't committed it because
> I want to get your opinions.

I'd say go ahead and commit it. I don't see any downside and you fixed 
a localization issue in there too :)

I haven't run the test yet. I'm at work now but I should be able to 
take a look at this tonight.

Ryan