[Yum-devel] Re: Reducing yum startup times and memory usage

[Gijs Hollestelle]

I've implemented a version of my fastcache that uses sqlite instead of
pickles to store the fastcache.

This further reduces startup times and memory usage by as much as a
factor 2.5(!!) (compare to stock yum) and there is no need to split
the primary.xml.gz into multiple files because sqlite only uses memory
for the data that is requested.

> - yum -C list updates: Original 8.9 seconds, 63MB memory usage. New
> 2.6 seconds, 24MB
Is now 2.2 seconds and 21MB

> - yum -C install mono-complete: Original 10.4 seconds 68M. New 7.2 seconds 58M
Is now 3.7 seconds and 24MB

The attached patch is against a clean CVS checkout, all the new code
is in fastcache.py and fastsack.py, only minimal (3 lines or so) on an
existing file (repos.py). It requires you to have python-sqlite
installed (available in dag's repository).

Again this patch is just a proof of concept, currently it only handles
primary.xml data, no file information at all and I've only tested
list, list updates and install (of something that is not that hard to
depsolve). Let me know what you all think, should we consider using
sqlite (maybe as an optional add on) to speed up yum and help reduce
its memory usage?

Regards,
  Gijs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqlite.diff
Type: text/x-patch
Size: 9072 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20050110/c9352e04/attachment.bin