[Yum-devel] Solving the yum mirror and update reporting problem.

[Christopher Owen]

Hello you fantastic people you.

A couple of weeks ago I came in to the IRC channel talking about an idea for a better way to handle repos for yum.  After much feedback and discussion of how things are done now, the general consensus seemed to be that of: that's a good approach.

I feel it a good idea to bring this up here on the yum-devel mailing list for a couple reasons

1)      After thinking about it more and being not pressured by the in-the-moment nature of IRC, perhaps a different approach will present itself

2)      I'm having a bit of trouble bending the yum codebase to my will :).

-= The Problem Space =-


1)      I, like many people, use multiple third-party repos in my setups.  These repos are managed by yum-priorities in order to not walk over each other.  When one repo is unavailable the update process breaks.

2)      I have a respectable number of servers that use yum to manage packages.  It is a constant effort to keep them up to date.  This is because yum works incredibly well on a single server, but it is difficult to get insight into what's going on a network of servers.

-= The Way Things Are Done Now =-


1)      If you want your repos to be available, become a mirror!  This is a real downer for repos that you only require 2-3 packages from a giant repo.

2)      If you want your repos to be available, but don't want to mirror the whole thing, just rsync the packages you want on a schedule!  This would work, but only for the working set of packages I have now.  If I would want to install a new package or search for package ability, I would have to do it some other way than by the yum command.

3)      If you want your repos to be available, but don't want to mirror the whole thing, and still want the packages available for searching, just install a squid proxy!  This would work well, but is smells deeply of "enterprise" (and that's a foul odor), would require integration at the edge if it's to be transparent (unless yum supports proxies directly?) which could have impact on other aspects of the network.  These are all solvable problems, but it's a little stinky (for my uses.)  Although this is the best solution so far, it does not provide reporting.

4)      If you want reporting, use SpaceWalk or Satellite!  These projects are suffering very much from enterprise-itis.  The list of technologies used for spacewalk does not inspire confidence (oracle being a particular problem for me.)  In short, I don't like it :).

-= Well, What am I suggesting Then? =-


1)      Something simple, with as few dependencies (enterprisey or otherwise) as possible

2)      Something that makes use of as much existing yum code as possible

3)      Something that will be able to offer reports

4)      Something that will fetch updates in advance of their need

5)      Something that will be a single repo under the admin's control

6)      Something that will do its best to handle network outage or repo unavailability

I suggest a special repo server that takes as configuration repos, their priorities, and their enabled status.  It outputs a single repo based on this.  When a client requests a package, the server will check locally if it is available, otherwise will download the file to its cache and then serve it out.  When a client searches, it is presented with the total (flattened) metadata to search.

This alone will solve the repo problem I was experiencing before.

For reporting, yum-updated or some other application would need to check in with this repo periodically and supply the server with a list of installed packages.

It would then be simple logic to determine which machines require updates and produce reports from that (available via rss, email, a web interface, or whatever.)

-= Summary =-

Thank you for taking the time to read though this.  If you have suggestions on how I can improve the idea, I would appreciate hearing them.  If there is error in my logic (beyond "things are enterprisey for a reason ;) ) please point it out.

Additionally, I have had some trouble getting through the yum code base and bringing myself to a level where I can confidently bang this out.  And suggestions there would be much appreciated as well.

 Christopher Owen.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.baseurl.org/pipermail/yum-devel/attachments/20100302/51d5722b/attachment.htm>