[Yum-devel] yum on an olpc machine (slooooooooooow)

Panu Matilainen pmatilai at laiskiainen.org
Thu Dec 21 07:30:50 UTC 2006


On Thu, 21 Dec 2006, seth vidal wrote:

> On Wed, 2006-12-20 at 10:47 +0200, Panu Matilainen wrote:
>
>> I wrote a quick-n-dirty sqlite "backend" for apt to see how things would
>> look like for another implementation (with totally different usage
>> patterns) of repodata.
>>
>> The initial version performs so badly you wouldn't believe it :D
>>
>> On 2.4GHz AMD64 the creation of dependency cache (which involves walking
>> over all packages and recording the dependency data to it's the memory
>> mapped dependency cache) takes over 3 MINUTES (!) for just FC6 core
>> data. With xml repodata, that's ~6.5s operation on this system, fully
>> reading filelists.xml info as well, which the sqlite version doesn't do at
>> all at this point.
>>
>> After a bit of investigation, the major bottleneck here is that the
>> provides, requires, conflicts and obsoletes tables don't have index on
>> pkgKey. After creating those indexes, it's back to ~6.5s even with the
>> naive initial implementation.
>>
>> I haven't done any timings on how those indexes would affect yum's usage
>> patterns, probably not *that* dramatic but it might be something to look
>> at.
>
> I'm sure that better indexing would help some lookups rather a lot.

Curiously enough, the pkgKey indexes on prco data don't affect yum at all, 
even though one would think eg returnPrco() would benefit a great deal.

> out of curiosity - how much does the size of the sqlite file go up when
> the indexes are added?

For FC-development primary db:
-rw-r--r-- 1 pmatilai pmatilai 5074944 Dec 21 09:17 primary.xml.gz.sqlite.orig
-rw-r--r-- 1 pmatilai pmatilai 5650432 Dec 21 09:18 primary.xml.gz.sqlite

So it's certainly a non-trivial amount of metadata on metadata which yum 
doesn't even seem to care about :)

 	- Panu -




More information about the Yum-devel mailing list