yum check takes two hours, 60% in 2 billion calls to str_eq()

[James Antill]

On Fri, 2014-05-30 at 16:46 +0200, Lars Ellenberg wrote:
> Hi there.
> (Cc me, I'm not subscribed)
> 
> I've got a "yum check" taking two hours.
> (really single core CPU time, actually, not much IO going on there, afaics)

 That is not normal. Whenever I've seen this kind of problem it's always
been a problem in the rpm indexes, rpm --rebuilddb might help.

> I cProfiled it.
> 
> 60% of the time it seems to spend in i18n.str_eq
> 
> I pastied an SVG (gprof2dot) here: http://paste.fedoraproject.org/105517/raw
> 
> It does 2001811124 calls to i18n.str_eq
> Yes, that's 2 billion.
> ... seems excessive ...

 Indeed.

> rpm -qa | wc -l : 716
> rpm -qa --provides | wc -l : 71774
> yum-3.2.29-40.el6.centos.noarch

 That doesn't seem super excessive.

> Seems like you have some O(2) or worse behaviour below check_provides(),

 You can see the code easily enough, it's generic and could be micro
optimised but it's basically:

 for each package pkg:
   for each provide in pkg prov:
       lookup prov in rpmdb and make sure pkg is in the results

...so it's roughly:

(provide lookup in rpmdb) * (provides in rpmdb)

...or in your case:

(time rpm -q --whatprovides blah) * 72000

...if the rebuild doesn't fix it, I'd try putting print statements in
rpmsack:check_provides ... to see which package/provide is causing the
problem.