[Yum-devel] [Patch] Resolver Performance and Correctness

[Florian Festi]

Hi!

I put together my patches in my own working dir (as proposed by James Bowes 
;)= ) to see if they have the expected performance gain and if they fit 
together. It turned out that the performance gain is within the expected 
range and that it is possible to achieve that with increasing the code base 
by only 20 lines. My test cases report the following timings:

real time in seconds
--------------------------------
Warmup: Full install    | 137.92
Resolve normal install  |  27.28
Resolve full install    | 170.79
Resolve install [a-k]*  | 191.09
Resolve empty update    |   7.92
Resolve update libgnome |  11.96
Resolve big update      |  19.13
Resolve erase glibc     |  33.67
Warmup: Dist upgrade    |  18.65
Resolve dist upgrade    |  20.30

I have the feeling that some tests could benefit even more by merging 
mytscheck into resolveDeps to reduce the number of ._checkConflicts and 
_checkFilerequirements calls. There are also some other optimizations I did 
not yet try out to keep the patch size below one MB. So I expect that 
further performance improvements are possible.

There is a short wrap up of what has been changed:
  * whatProvides/Requires - returns {po -> [matching Ps/Rs]}
   * added to PackageSackBase, MetaSack, PackageSack,
     YumAvailablePackageSqlite
   * Depsolver: .whatInNewProvides, .whatInTsProvides, .whatInTsRequires
    * rename/move to tsinfo?
   * rpmdb.whatProvides returns {po -> [hits]} -> code clean up
    * still needs param for compatibility -> 2 level deprecation needed!!!
  * Improved mytscheck
   * TransactionData.removedmembers - used in _mytscheck to treat
     removed INSTALL members as Removes and
     removed REMOVE members as installs
    * needs testing/test cases
   * TransactionData.relatedto and .depends_on converted to sets
   * new _checkInstall, _checkRemove,
    * dropped _provideToPkg and _requiredByPkg
   * _checkConflicts, _checkFileRequires - work globally
    * ignoring installed problems and problems in updates properly?
   * dropped Depsolve.deps cache
   * added local rpms to pkgSack via localSack (cli.py)
    * adjusted interface of PackageSackBase.matchPackageNames()
      to other Sacks
     * Does this break anything?
  * Prco helpers
   * Package.matchingPrcos
   * rpmUtils.miscutils.rangeCompare() - moved code from Package
  * SqliteSack
   * .prco uses empty tuple as default value
     to destinct from empty list and avoid multiple db queries
   * _search_cache
  * added better profiling output

Interesting points that need further discussion are:
  * rpmdb.whatProvides should change its return value. I currently use an
    additional parameter (as suggested by Jeremy) that could go a way be two
    deprecation cycles, but this is obviously very ugly.
  * PackageSackBase.matchPackageNames() is incompatible to the same
    methods in other Sack classes. Is changing/rewriting it ok?

Most other changes should not interfere with interfaces. There are some 
minor changes I am not sure if they touch anything that is considered as an 
interface (TransactionData.relatedto, .depends_on, Depsolve.deps).

Some aspects of the patch need (another) review. So I see it in a beta state 
currently. I am also aware that it needs to be broken into several pieces 
before it can be committed. But as large parts depend on each other the size 
of the singe patches can be discussed after it is sure what goes in and what 
doesn't.

Any feedback appreciated

Florian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yum-fix_depsolver.diff
Type: text/x-patch
Size: 56983 bytes
Desc: not available
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20070605/1a5039a1/attachment.bin