[Yum-devel] Simple speedup of 'yum install'

[David Lutterkort]

In poking around the yum code, I noticed that performance of 'yum
install foo' can be greatly sped up (assuming a warm cache, decent
network connection etc.)

Right now, the way in which the user's input of a pakcage spec is
translated into a package object in packages.parsePackages forces yum to
load all the data of the packages tables into memory, i.e. forces a
'select * from packages' for each enabled repo, and the ensuing creation
of a dict with seven entries for every package. That is slow for a
number of reasons.

I attach a patch that turns that around so that the database is queried
directly when the user runs 'yum install', avoiding sucking lots of data
into memory. If this looks like something that can be committed, I'll
look into doing similar things for the other uses of
packages.parsePackages.

I did some profiling without and with the patch on a fairly beefy FC6
machine with core, extras and updates enabled (a total of 8700 packages)
Profiling times are based on a warm cache, local repo mirrors and are
the average of a handful of runs. I used two simple tests:
      * Install of a nonexisting package; this is one use that should be
        fast because it's frustrating to have yum take a long time to
        tell the user about a type. Initially, running 'yum -d 0 -e 0
        install foobar' took 2.9s (user + system), with 1.3s spent in
        loading the package metadata and 0.5 s in packages.parsePackages
        (the patch eliminates the metadata loading step). With the patch
        applied, this now takes 0.8s, with 0.2s for parsing packages
      * Install of a single package, another operation that users expect
        to be fast; I ran 'yum -d 0 -e 0 -y install xpdf' repeatedly for
        this. Initially, this took 4.3s with 1.3s for loading the
        package metadata, and 0.5s for packages.parsePackages. With the
        patch, this now takes 2.7s with 0.2s spent in parsePackages

David

-------------- next part --------------
A non-text attachment was scrubbed...
Name: parsePackages.patch
Type: text/x-patch
Size: 6987 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20070115/d6d05f0c/attachment.bin