[Yum-devel] fun little performance problem

[seth vidal]

> I've just looked at the code for repoclosure, and what it does it do a
> whatProvides on all requirements in all packages, for base these are
> 30.000 requires that means 30.000 calls to whatProvides resulting in
> about a lot of database queries.

<nod> Though if we were doing 4*N b/c for each time we were looking for
Provides we were also querying obsoletes, conflicts and requires. A
little expensive, that.


> A possible solution to this is to create a method in sqlitesack that
> can cache all provides to memory, so that we don't have to do a
> database query for each requires.

I had written that down in my notebook, too, I'm only worried about the
memory size if we do that. However for certain operations it may well be
worth it. Esp if we're only caching prco-info.

> Another way to speed things up, is something that I've implemented
> right now to repoclosure is to not check requirements that have
> already been checked (see the attached patch for that). This speeds up
> checking base by about a factor 10. But it is still slow.

I don't see an attached patch. :)


> An option to disable sqlite for some operations would be very useful,
> but that will prevent us from dropping the pickle stuff anytime soon.

What if we make the option to disable all caching - just read the xml
directly and immediately. For certain operations it would make sense to
just want to get the xml, have it in memory and leave it alone :)

That way we can dump the pickle but still provide a non-sqlite mechanism
for getting the data into memory.

-sv