[Yum-devel] Re: Reducing yum startup times and memory usage
skvidal at phy.duke.edu
Mon Jan 10 22:45:16 UTC 2005
On Mon, 2005-01-10 at 20:17 +0100, Gijs Hollestelle wrote:
> I've implemented a version of my fastcache that uses sqlite instead of
> pickles to store the fastcache.
> This further reduces startup times and memory usage by as much as a
> factor 2.5(!!) (compare to stock yum) and there is no need to split
> the primary.xml.gz into multiple files because sqlite only uses memory
> for the data that is requested.
> > - yum -C list updates: Original 8.9 seconds, 63MB memory usage. New
> > 2.6 seconds, 24MB
> Is now 2.2 seconds and 21MB
> > - yum -C install mono-complete: Original 10.4 seconds 68M. New 7.2 seconds 58M
> Is now 3.7 seconds and 24MB
> The attached patch is against a clean CVS checkout, all the new code
> is in fastcache.py and fastsack.py, only minimal (3 lines or so) on an
> existing file (repos.py). It requires you to have python-sqlite
> installed (available in dag's repository).
> Again this patch is just a proof of concept, currently it only handles
> primary.xml data, no file information at all and I've only tested
> list, list updates and install (of something that is not that hard to
> depsolve). Let me know what you all think, should we consider using
> sqlite (maybe as an optional add on) to speed up yum and help reduce
> its memory usage?
I pinged around to some folks involved in the pkg mgmt world in fedora
and elsewhere and it seems like it might not be a bad move to look at
this. It seems possible to get sqlite and python-sqlite into fedora
core, for example, if the benefit is significant.
I'm not near cvs at this moment but I'll take another look tonight to
see how it works. Thoughts I have:
1. how long does it take to do the read in from *.xml to the sqlite db?
2. what sort of memory does it look like reading that in?
More information about the Yum-devel