[Yum] metadata compression

[Seth Vidal]

On Mon, 20 Apr 2009, Ville Skyttä wrote:

> On Sunday 19 April 2009, James Antill wrote:
>
> [sqlite bz2 vs lzma/xz]
>> ...which implies somewhere in the 25-35% savings range, but I doubt
>> that's enough (on it's own) given the CPU/code requirements.
>
> Regarding CPU requirements, xz/lzma should be much better on metadata consumer
> boxes than bzip2, and somewhat more memory intensive but I doubt this would
> matter much if any at all as long as lzma compression levels are kept at sane
> values.  It is however quite a bit heavier on the metadata producer boxes,
> both CPU and memory wise: http://tukaani.org/lzma/benchmarks .  Whether that's
> a problem depends on the scenario but I'm sure people wouldn't mind being
> given the choice; e.g. even if the CPU/memory requirements would be a problem
> for boxes composing something large like Fedora Rawhide all the time, at least
> for immutable final release repos it should be doable, ditto for many
> scenarios between these extremes.
>
> Regarding code requirements, if yum devs don't feel like implementing it, I'm
> sure the code will just magically appear somewhere if there's a clear green
> light given by the yum devs and when xz and its python bindings reaches a
> stable release.

The real problem is dealing with the backwards and somewhat forward 
compat w/o carting around 12 different compression formats in every 
repodata dir. It's not the end of the world but the code to open up
metadata is getting a bit bleah in places and if we're going to end
up with the flavor-of-the-week compression algorithm then I'd
prefer that the code change enough to accomodate next weeks flavor,
too.

-sv