[Yum-devel] urlgrabber: timestamp check
skvidal at phy.duke.edu
Fri Aug 5 08:17:51 UTC 2005
On Wed, 2005-03-09 at 10:28 -0700, Michael Stenner wrote:
> On Mon, Mar 07, 2005 at 12:44:45AM -0500, seth vidal wrote:
> > > The dealio is that if you have a 1000 byte remote file and a 1000 byte
> > > local file, urlgrabber will try and construct a zero-length byterange
> > > request which then barfs. We might, for reget only, consider
> > > special-casing the zero-length request to simply have it do nothing.
> > > Thoughts, Ryan?
> > >
> > > I'm open to considering if_newer_than options or somesuch, but I'd
> > > want to sort out some of the issues: urlgrab only? what if there's
> > > no local file?, etc.
> > Okay, I don't think I ever answered this. The short version of what I
> > want is an easy item to add to the top of:
> > yum/repos.Repository.getRepoXML() to see if we need to bother getting
> > the file.
> OK, I'm attaching both a patch to grabber.py and a little test program
> to demonstrate how one might do this. I'd really like input from you
> and Ryan about whether this makes sense. For the newly defined Error
> number (which is for HTTPError), I've included special "code" and
> "exception" attributes, which seems a little kludgy. However, I do
> kinda like returning a bit more information. Before, the same
> exception got raised as IOError and the code/message were just encoded
> in the string. That's OK to a user but it makes it hard to do things
> like this.
> The only other way I thought of to solve the problem was to allow
> callers to include custom urllib2 handlers. That would really allow a
> lot more flexibility, but I'd want to do it right, and that might take
> a little thought.
> Also note that this patch is currently causing some unit tests to
> fail, which I haven't looked into yet. I haven't committed it because
> I want to get your opinions.
> P.S. To use the test program, just touch a file named reference-comp
> and give it a new (today) timestamp or an old (say, 2000) timestamp.
This kinda languished for a while.
What do you think about the above looking back on it now. Should we go
ahead and do it or work on exposing the urllib2 custom handlers?
i would kinda like a quicker check than what we have currently.
More information about the Yum-devel