[Yum-devel] [RFC] Questions about sqlitesack and list vs. generators

[seth vidal]

On Mon, 2007-12-10 at 14:12 -0500, James Antill wrote:
> A couple of questions about sqlite functions and usage:
> 
> 1. Does anyone know why returnPackages() re-runs the _excluded() method
> on cached content? AFAICS we just redo excluding for all pkgs for
> nothing.
> 

I think it is b/c we can update the exclude list whenever and from
wherever. It's not just a one time event, necessarily.


> 2. AFAICS simplePkgList would be better off written like:
> 
>     def simplePkgList(self):
>         """returns a list of pkg tuples (n, a, e, v, r) from the sack"""
> 
>         simplelist = []
>         for pkg in self.returnPackages():
>                 simplelist.append((pkg.name, pkg.arch, pkg.epoch, pkg.version, pkg.release))
>         return simplelist
> 
> ...this saves about half a second (20-25%) for pretty much all the
> commands, as the above loop is instant in comparison to the executeSQL()
> version[1].

is that before or after the pkglist has been made?



> 3. Why aren't we using generators more, I'm not really sure how to
> measure the memory requirements for the above simplePkgList() but given
> pretty much all usage is "for pkgtup in simplePkgList():" it seems
> mostly overhead. This isn't the only case either ... is it a back
> compat. issue?

In some cases it's as simple as because len(somegenerator) doesn't work
and that's nice to have.


> [1] The main Fedora repo. is the main problem here, due to all the
> items ... and given how little that changes I'm almost tempted to try
> putting another layer of caching in there just for that, almost.

The smarter move in fedora would be to set the metadata timeout for the
primary repository to something much higher so it is checked for much
less often. Since it doesn't change much (if ever) setting it to a 2 day
value probably wouldn't hurt anyone.

-sv