[Yum-devel] YumRepository._loadRepoXML proposed changes

[James Antill]

 This is the latest patch for the grouped MD policy loading, you can
either get the patch or the git repo here:

http://people.redhat.com/jantill/gits/yum/#_groupLoadRepoXML

http://people.redhat.com/jantill/yum/_groupLoadRepoXML.patch

...the only differences are that the git version includes the config.py
changes and a minor change to "yum makecache" (although, you can try it
out easier then, maybe :).
 I've integrated the simple changes, which didn't change/add any
functionality ... so this version should be much easier to read, to see
what is actually changing.

 The major things to look at are:

1. mdpolicy, is that a fine config. name?
2. Do we want to allow per. repo. configuration?
3. any comments on the group:main, group:small, group:primary policy
types? Bad names, you want different functionality?
4. Does anyone want to insist on the old behaviour being the default
instead of it always downloading the primary MD now too? Or someone want
to argue for something else, group:small seems sane to me as does
group:main (changelog data, not as much :).
5. Should we also change _instantLoadRepoXML() so it doesn't fail when
there is no network, or just wait for the urlgrabber change?
6. Does anyone care about the C-c behaviour, and should that be fixed
before the initial merge?
7. Anything else you want me to change before I merge?

...I've been testing it, with some yum-updatesd changes to have a
different mdpolicy for it, and it's been very nice. Basically doing
exactly what I'd want (alas. no comps destruction has happened in
Fedora, to give it a real world test ;).
 There is another change on top of this that I'm still working/testing
which re-tests the repomd.xml file when we change mirrors, but apart
from that ... this is it, I think.

 Pros.
 -----
   
 Should stop these metadata update problems:
    
  1. We get corrupted comps/etc. files on the master and everyone has
problems.
  2. We hit mirror(s) that have an updated repomd.xml but nothing else.
  3. We don't have a network but the cache does a timeout and urlgrabber
kills repomd.xml and we can't get a new one (makes yum stop working).
    
 Should stop "back in time updates". Ie. we hit an old mirror, and we
basically go back in time for updates.

 Helps stop yum cmd line usage hitting network. Basically yum-updatesd
can now download all of MD data we need and not just primary.xml so we
don't have a problem where user does "yum blah" which happens to need an
MD file we don't have so we hit the network.
 This actually has follow-on problems where the file isn't the same anymore
but we haven't updated repomd.xml yet (or the network is down, think yum
deplist /usr/bin/foo).
    
 Cons.
 -----
    
 Downloads more stuff at once. Basically the current yum model only ever
downloads what we need, when we need it, now we'll download more of the MD
files whenever repomd.xml gets updated and they need it. However we can
config. to do the old behaviour (and/or how much of the MDs we get).

 Currently I don't do anything special on C-c, or other weird
exceptions ... so it's possible that a C-c at the wrong time will leave
the repo in a partially updated state. But that's what it does all the
time now, so I don't think this is a big issue.
    
 _If_ we don't have a full set of MD currently, and we fail to get a new
batch of data then we'll revert back to the non-full set. So it's
_possible_ that we'll need one of the files in the set we don't
have, but that would be available if we had used the traditional code
path (Ie. the error was in one of the files we don't currently need).
 This is a very weird edge case, and if you care a good solution IMO
is to always use group:all ... then we always have a complete set.

 Minor CPU speed hit, due to parsing two sets of repomd.XML.

-- 
James Antill <james.antill at redhat.com>
Red Hat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20080110/1fc3ca47/attachment.pgp