[Yum-devel] yum on an olpc machine (slooooooooooow)

[seth vidal]

On Sat, 2006-12-16 at 13:48 +0200, Panu Matilainen wrote:
> > The reason I didn't go this route for  FC5 anaconda is that it's just
> > the same problem as having hdlist, etc.  Multiple versions of the same
> > metadata, the problem we were trying to avoid by moving to repodata.
> > I'd strongly argue this is the wrong approach.
> 
> +1
> 
> I suggest looking closer at where the time is *really* spent. Remember the
> libxml2 "slowness" which turned out to be something in the way things are
> copied between C and python? Is it really the xml parsing where most of
> the time is spent, or is it something else like sqlite interactions or...?
> Parsing those xml files sure isn't cheap, but it's not *that* slow in
> C/C++ - I'd look for other places first.
> 
> Here's one easy target for optimization (the time difference is
> consistent over successive runs):
> 



Here's the output of the same on the olpc box:
multiple runs here so read through them all:

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
8 cache files removed
-bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras' list
available >> /dev/null

real    2m10.291s
user    1m29.900s
sys     0m3.670s

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
1 cache files removed
-bash-3.1# time yum --disablerepo='*' --enablerepo='extras' list
available >> /dev/null

real    1m38.746s
user    1m29.620s
sys     0m3.540s

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
1 cache files removed
-bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras' list
available >> /dev/null

real    1m37.648s
user    1m29.810s
sys     0m3.560s

-bash-3.1# yum clean dbcache
Loading "installonlyn" plugin
1 cache files removed

-bash-3.1# time yum --disablerepo='*' --enablerepo='extras' list
available >> /dev/null

real    1m38.422s
user    1m29.790s
sys     0m3.350s



> 9.5 vs 8.0 seconds is one helluva big difference in percentage just to
> tell the user "something is happening". This is with a reasonably fast
> display adapter, I could imagine OLPC suffers even more from this. Didn't
> try it, but simply making the progress callbacks (well, writing to screen)
> less frequent should shave off quite an amount of time. The user doesn't
> *really* need to know we're now processing exactly 1654th of 2001 records,
> a rough idea of making progress (every 5/10 percent update for example) is
> quite enough.

looks like it is less important on the olpc machine. given the numbers.
It shaves a little less than a second. Keep in mind in that run of 1m38s
it is only parsing primary.xml.gz for extras.

-sv