[Yum-devel] Sqlite performance

[Florian Festi]

> So anyplace we retrieve pkgIds as a unique identifier it would seem to
> make sense to grab pkgKey, too and stuff that in the package object so
> we can get to it later for doing joins with it instead of pkgId. Is that
> what you were thinking?

I did that but it didn't help (see attachment). I tried to optimize 
returnPrco() but it kept using up a lot of time. I was wondering if there 
was a problem with the caching, but it turned out that it really has to load 
lots of Prcos.

Reason for this is that the sqlitesack.search*(name) returns a list of 
package objects. Because of that the surrounding code has to load the 
provides for all returned packages and has to search through them.

Easiest solution would be to return a list of (pkg, name, flag, (e,v,r)) 
tuples or a dict {pkg -> list of (n, f, (e,v,r))} (or use new Requires and 
Conflicts objects). Even better would be to also pass flag and (e,v,r) to 
the search methods. That way the sqlitesack could put as much intelligence 
into the SQL queries as it wants to further reduce the amount of data being 
processed.

The attached patch changes all pkgId to pkgKey lookups and loads every 
package only once. I renamed ._excluded to .excluded to avoid usage with 
pkgId insted of pkgKey.

Florian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: yum_sqlitesack_pkgKey.diff
Type: text/x-patch
Size: 17853 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20070322/aba3d7ee/attachment.bin