[Yum-devel] yum on an olpc machine (slooooooooooow)

seth vidal skvidal at linux.duke.edu
Sat Dec 16 15:42:52 UTC 2006


On Sat, 2006-12-16 at 13:48 +0200, Panu Matilainen wrote:
> On Sat, 16 Dec 2006, Paul Nasrat wrote:
> >
> >> The only way I can think of would be a different format so we don't have
> >> to parse the xml or pre-parsing the metadata into a sqlite db. This
> >> would make downloads of the metadata larger but maybe it would be faster
> >> for operations.
> >>
> >> For example - fedora extras:
> >> -rw-r--r--  1 root root 1.6M Dec 16 03:06 primary.xml.gz
> >> -rw-r--r--  1 root root 2.2M Dec 16 03:09 primary.xml.sqlite.bz2
> >>
> >> bzipped the primary xml sqlite db is 2.2M vs 1.6M for the xml itself.
> >
> > The reason I didn't go this route for  FC5 anaconda is that it's just
> > the same problem as having hdlist, etc.  Multiple versions of the same
> > metadata, the problem we were trying to avoid by moving to repodata.
> > I'd strongly argue this is the wrong approach.
> 
> +1
> 
> I suggest looking closer at where the time is *really* spent. Remember the
> libxml2 "slowness" which turned out to be something in the way things are
> copied between C and python? Is it really the xml parsing where most of
> the time is spent, or is it something else like sqlite interactions or...?
> Parsing those xml files sure isn't cheap, but it's not *that* slow in
> C/C++ - I'd look for other places first.
> 
> Here's one easy target for optimization (the time difference is
> consistent over successive runs):
> 
> [root at turre yum]# yum clean dbcache
> Loading "installonlyn" plugin
> 3 cache files removed
> [root at turre yum]# time ./yummain.py -C --disablerepo='*'
> --enablerepo='core' makecache
> Loading "installonlyn" plugin
> Setting up repositories
> ################################################## 2931/2931
> ################################################## 2931/2931
> ################################################## 2931/2931
> Metadata Cache Created
> 


Here's a few makecache runs of just one repository (extras) on the olpc
box:

-bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras'
makecache >> /dev/null

real    4m16.016s
user    3m14.860s
sys     0m14.140s

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
3 cache files removed
-bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras'
makecache >> /dev/null

real    4m3.399s
user    3m12.690s
sys     0m12.400s

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
3 cache files removed
-bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras'
makecache >> /dev/null

real    4m8.325s
user    3m13.450s
sys     0m12.300s


That's a time that cripples usability.

-sv





More information about the Yum-devel mailing list