[Yum-devel] Metalinks for Fedora 10 repomd.xml etc.

[James Antill]

On Sun, 2008-10-05 at 00:11 -0500, Matt Domsch wrote:
> On Sun, Oct 05, 2008 at 01:04:03AM -0400, James Antill wrote:
> > On Sat, 2008-10-04 at 19:24 -0500, Matt Domsch wrote:
> > 
> > > MM indexes the master mirror file system every 4-6 hours, so it
> > > shouldn't be out of date longer than this.
> > 
> >  Interesting, what does this mean? Can we do something more often that
> > just checks for new repomd.xml files and does the checksums?
> >  Also given the message on the infrastructure list, could that be
> > related? -- Although I only saw it for about 6-8 hours (I think I saw
> > the first problems around 16:00, not 100% on that though.
> >  Everything seems to be fine now.
> 
> I need to split up the master mirror directory crawl from the remote
> mirrors directory crawl.  The local crawl takes about 12 minutes right
> now.  I just committed code that lets me run that separately from the
> remote mirrors crawl, so I can schedule it to run almost constantly.

 *nods* ... cool :).

> > >  I need to implement a
> > > better way to know when directories change and index those that change
> > > more frequently more often.  Regardless, any index updates don't
> > > propogate to the mirrorlist cache until the top of the hour, so there
> > > will always be that much of a delay...
> > 
> >  *nods* ... do we know when rel-eng push new updates¹ and/or when
> > mirrors run rsync? My guess is even a constant 60 minutes won't be too
> > bad, but 6 hours could be annoying.
> > 
> >  Also while it won't help us right now, is it possible to get MM to
> > obey/use "Accept-Encoding: gzip"?
> 
> sure - tell me what I need to do.  if I see that header, gzip the data
> before returning it?  Any other headers or data types that need to
> change when doing so?

 Yeh, mostly ... you give out the data after passing through gzip, and
you add the response header "Content-Encoding: gzip".
 You can test it with:

curl -v --compressed --head \
'http://mirrors.fedoraproject.org/metalink?repo=updates-released-f9.newkey&arch=x86_64'

 I can't think of any other headers.

-- 
James Antill <james at fedoraproject.org>
Fedora