[Yum-devel] API Changes: Was: plugings modifying the URLGrabber instance

[seth vidal]

On Sat, 2005-07-02 at 15:49 +1000, Menno Smits wrote:
> seth vidal wrote:
> 
> >>Okay, just note that I'm sort of poking at this from two directions.
> >>Making Yum speak RHN, and adding features to Yum so that RHN/Current
> >>could be modified to work with Yum.
> > 
> > 
> > So one thing I'm not interested in - I'm not interested in yum emulating
> > or talking over xml-rpc just like up2date/rhnsd does.
> > 
> > I wouldn't mind learning from problems they may have encountered but
> > making yum compatible with rhn's xml-rpc command stack is NOT terribly
> > interesting to me.
> > I can see how it might be valuable to be able to, eventually, but
> > targeting that first is a bad idea, imo.
> 
> I agree. We should aim for incremental refactoring steps that result in 
> a sufficiently clean abstraction of repositories and packages so that 
> talking to stuff like RHN is possible. I can envisage other repository 
> types as well.

I can think of a bunch of repository types that I'm not interested in
supporting, tbh. not interested in:
 red carpet's format
 apt's format
 urpmi's format
 yum 2.0.x headers/ format
 
 to be perfectly honest I'm not interested in RHN's format, I'm only
interested in making it more possible for other formats to be added by
other people. But I have ZERO inclination to maintain 57 different
format types just for fun. 


> To be clear, we are aiming for Yum to be able to support different 
> repository types simultaenously right?

yes. Though, again, I don't see a lot of a point to support formats
beyond the standard one created via createrepo. Considering how trivial
it is to make repomd repositories my sympathy for those folks who feel
they can't is pretty low.
But if we make it possible for other formats to be supported on the way
to cleaning up the repo/packageSack code in yum, that's fine.


Maybe someone should explain it to me but I don't really see much virtue
in supporting other repository formats. I've heard the 'compatibility'
argument but  if it cost something to make the createrepo format I'd
understand but it costs users nothing. The sole case I've heard so far
is Jack's argument: 
"I have an rhn satellite server and its my only mechanism for accessing
updates, how can I use it". That situation makes _some_ sense to me.
Supporting current doesn't really, though. If you're using current then
you already have all the packages in some directory on some server you
control anyway. Why would you bother setting up the extra complexity of
it when you could just setup a webserver and be done with it? What does
it buy yum to have this code in the tree? I guess that's my basic
question.


> I think different package sacks are still required but Jack's right in 
> that we do need some way for Yum to be able to use multiple packagesack 
> types at the same time. At the moment Yum either uses the the repomd or 
> the sqlite package sack. Some sort of "MetaSack" class is required to 
> hold different packagesacks and aggregrate operations on them.

The benefit of the single representation of packages is just searching.
Now if we wanted to do this an 'easier' way it would be to have the
metasack really just perform the same routines on the rest of the
packageSacks. One packageSack for each repository.

so MetaSack.searchProvides() would really do:
   for repo in repos:
      repo.sack.searchProvides()

   then return those results into a ListPackageSack or into whatever is
the expected return format.

> By "shared methods" I assume you mean "shared implementation". Shared 
> implementation isn't as important here as shared interface. As long as 
> PackageObjects look and act the same we can mix and match them 
> interchangably.

by shared methods I meant more like a base abstract class with methods
that all of the derived classes would have.


> > Something like:
> > YumPackageObject
> >     YumRepoMDPackageObject
> >     YumRHNPackageObject
> [snip]
> 
> These classes seem about right to me. The main thing I'd add is the 
> MetaSack as mentioned above and Repository classes to abstract the 
> mechanics of talking to the repository and populating package sacks or 
> whatever. Hell, repository classes could probably handle parsing their 
> own bits sections of yum.conf. Each repository class should have 
> particular PackageObject and PackageSack subclasses associated with them.

why would they parse their own sections in the config file? we would end
up needing to parse it once from the main config class and then parse it
again to pitch it off to each repository format to parse themselves.
That seems inelegant. 

I'd say the repository classes just take a simple dict of information
for their own information and we keep any configuration file parsing in
a single class.

So we end up with a base config class that stores global/main
configuration data. Then a configparser class which deals with the file
format, the RepositoryStorage class which holds and accesses all of the
repositories and finally repository classes which actually store the
repo information. That way the repository classes don't care if the
yum.conf file is in ConfigParser format or xml or some other format.

The other side of the argument is the need for some additional
repository type to neeed some wacky config variable in the config file.
Given the restrictions of ConfigParser as-is we might simply be able to
consider all other items in each repo section as key-value pairs, grab
them as simple strings and let the repository class deal with what it
gets in the config dict. That's not much cleaner, though. hmm.


> Ideas for things to do from here (maybe this should go on the wiki?):
> 
> - merge the repomd stuff into the rest of yum and clean it up. The whole
>    repomd directory can probably go unless there's still some need to
>    keep it separate.

agreed - it was originally separate for various organizational reasons
that have since gone away.


> - remove the older repo packagesack stuff so that we only have the
>    sqlite cache code left
well I want to make sure we can still read the xml completely into
memory, directly.

> - experiment with the MetaSack idea and see if its feasible to use
>    multiple packagesacks at the same time

I think if we treat it like a wrapper for  a sack for each repository
and just require that the repository sack object have certain methods we
should be in reasonably good shape.


> - develop a Repository base class and implement repo metadata repository
>    as a subclass. This may be just refactoring what's already in
>    repos.py.

<nod>

-sv