[Yum-devel] urlgrabber: timestamp check

[Ryan Tomayko]

Crap. Sorry, I just realized I didn't read your question right. I 
thought you were asking whether there was a way to only grab something 
if the remote resource is newer than the local resource.

Anyway, yea, I don't think it would be too hard to check the date of a 
remote resource.

HTTP: You can send an HEAD request to a resource and you get back the 
headers (including a date) without the resource.

FTP: Yuck. I've seen code in ftplib that does an `ls` and then parses 
the result to get information about the files. Kind of gross but should 
be possible.

Ryan

On Feb 23, 2005, at 9:44 AM, Ryan Tomayko wrote:
> On Feb 23, 2005, at 1:45 AM, seth vidal wrote:
>> Hey,
>>  is it possible right now in urlgrabber to check the timestamp on a
>> remote url?
>>
>> It'd be nice to remove the repomd.xml download if it is unchanged. It
>> would also give us a way to save the mirrorlists until they changed.
>
> I think I remember being really surprised that we weren't already 
> doing this a little while back. It should be trivial to implement 
> given a local last-modified date. I'll take a look. I may have already 
> implemented this. It was on my list at one point...
>
> HTTP has a 'If-Modified-Since' header that allow you to request a 
> resource only if it has been modified since a certain date:
>
>     GET /path/to/resource HTTP/1.1
>     If-Modified-Since: Wed, 23 Feb 2005 12:00:00 GMT
>     ...
>
> If the resource hasn't been modified since that date you get back a 
> 304:
>
>     HTTP/1.1 304 Not Modified
>     Last-Modified:
>
> And the resource is not sent. This is really clean because it lets you 
> negotiate whether you want to pull a resource in a single round trip.
>
> Ryan
>
> On Feb 23, 2005, at 1:45 AM, seth vidal wrote:
>> Hey,
>>  is it possible right now in urlgrabber to check the timestamp on a
>> remote url?
>>
>> It'd be nice to remove the repomd.xml download if it is unchanged. It
>> would also give us a way to save the mirrorlists until they changed.
>>
>> -sv
>>
>>
>> _______________________________________________
>> Yum-devel mailing list
>> Yum-devel at linux.duke.edu
>> https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
>>
>