[Yum-devel] New yum plugin: RPM database caching

Ralf Ertzinger yum-devel at camperquake.de
Thu Oct 16 17:57:32 UTC 2008


Hi all.

I wrote a little plugin today which intends to speed up yum operations
which access the RPM database.

I noticed that doing install/update/remove operations tooks quite
some time seeking through the RPM database in order to gather
a picture of the installed packages. This was slow, due to internal
and external fragmentation of the database files and the need to
seek() around in the files.

So this plugin tries to remedy this by hijacking the kernel buffer
cache to help yum out.

When an install/upgrade/remove operation is intended it loops through
all the files in /var/lib/rpm and reads every file in that directory
(not keeping the data, of course), thus bringing the information into
the system cache. Subsequent operations on the database will most
likely be serviced from the cache, and work much faster.

Testing this on my system today brought a speedup from 73 seconds
(normal yum upgrade operation with cold caches) to 29 seconds
(yum upgrade operation, with plugin, with cold caches). Times
measured from yum start to the install confirmation prompt.

These times are, of course, heavily dependant on the exact system
state (fragmentation of the database files, amount of database queries
required and so on), but I think that in general a speedup can be
archived.

There are a few problems with this plugin:

a) which operations require RPM database access and would thus benefit
   from this caching? Currently install/upgrade/remove are hardcoded.
b) if we are running on a system with little RAM there is not much
   point in precaching all the database files, since there may not
   be enough RAM to cache all data, so we would gain nothing. How
   does one determine the amount of free RAM in python?
c) as Seth pointed out to me in IRC today the sqlite files may
   benefit from this, too.

Please find the plugin attached. The config file consists of the
minimal 'enabled=1' line and is not included.

-- 
"Ich forde Sie hiermit im Namen der Sozialdemokratischen Partei und der
Katholischen Kirche letztmalig auf, mir zur Ausübung meiner öffentlichen
Pflichten im Dienste der Umwelt und der Bundesrepublik Deutschland Ihr
Fahrrad zu überlassen!"                     -- Loriot, "Autofrei"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: warm-cache.py
Type: text/x-python
Size: 1647 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20081016/4e41cbc1/attachment.py 


More information about the Yum-devel mailing list