[Yum-devel] [PATCH][RFC] Optimizing SqliteSack

[Florian Festi]

James Antill wrote:
> On Fri, 2007-12-14 at 11:52 +0100, Florian Festi wrote:
>> +class YumAvailablePackageSqlite(YumAvailablePackage):
>> +
>> +    prco_cache = { 'obsoletes': SmallLRUCache(maxsize=10),
>> +                   'conflicts': SmallLRUCache(maxsize=10),
>> +                   'requires': SmallLRUCache(maxsize=10),
>> +                   'provides': SmallLRUCache(maxsize=10) }
>> +
> 
>  We don't need to make prco_cache part of the API do we?

True. I'll add an _

>  Where did the number 10 come from? That seems like a small number of
> packages to have prco info. for.
It is the number of packages we have PRCOs in memory. I choose 10 because it 
  doesn't hurt my test cases. May be we need to increase the number if we 
run into problem with some cases that are out of my attention right now.

>  Also it seems weird to have the LRU cache over each of prco as though
> they are different. It also seems like if we are trying to save memory
> (esp. for large transactions), then the obvious change is to have
> packages share prco data (Eg. zlib requires is a subset of glib
> requires, and I bet that's very common).

This sounds very expensive as you need to match the prco against each other. 
The current solution just loads them and discards them if the pkg is ok. Be 
aware that searching is done in the sqlite and we need the Prcos only when 
looping over them in Depsolve.check*().

>>      @catchSqliteException
>> -    def returnPrco(self, prcotype, printable=False):
>> -        if isinstance(self.prco[prcotype], tuple):
> 
>  Why are you removing part of the API?
  I am only removing the implementation. returnPrco stays part of the class 
API but the inherited implementation is used. This calls the _getPrco() 
method over the SqlitePrcoDict. This is a bit tricky, but I didn't see 
another way to obtain the Package.prco dict as part of the API.

Florian