[Yum-devel] [RFC] Questions about sqlitesack and list vs. generators

[James Antill]

 A couple of questions about sqlite functions and usage:

1. Does anyone know why returnPackages() re-runs the _excluded() method
on cached content? AFAICS we just redo excluding for all pkgs for
nothing.

<code>
    def returnPackages(self, repoid=None):
        """Returns a list of packages, only containing nevra information """

        returnList = []
        if hasattr(self, 'pkgobjlist'):
            if self.pkgobjlist:
                for po in self.pkgobjlist:
                    if self._excluded(po.repo, po.pkgId):
                        continue
                    returnList.append(po)
            return returnList

        for (repo,cache) in self.primarydb.items():
            if (repoid == None or repoid == repo.id):
                cur = cache.cursor()

                executeSQL(cur, "select pkgId,name,epoch,version,release,arch from packages")
                for x in cur:
                    if self._excluded(repo,x['pkgId']):
                        continue

                    returnList.append(self.pc(repo,x))

        self.pkgobjlist = returnList

        return returnList
</code>

2. AFAICS simplePkgList would be better off written like:

    def simplePkgList(self):
        """returns a list of pkg tuples (n, a, e, v, r) from the sack"""

        simplelist = []
        for pkg in self.returnPackages():
                simplelist.append((pkg.name, pkg.arch, pkg.epoch, pkg.version, pkg.release))
        return simplelist

...this saves about half a second (20-25%) for pretty much all the
commands, as the above loop is instant in comparison to the executeSQL()
version[1].

3. Why aren't we using generators more, I'm not really sure how to
measure the memory requirements for the above simplePkgList() but given
pretty much all usage is "for pkgtup in simplePkgList():" it seems
mostly overhead. This isn't the only case either ... is it a back
compat. issue?


[1] The main Fedora repo. is the main problem here, due to all the
items ... and given how little that changes I'm almost tempted to try
putting another layer of caching in there just for that, almost.

-- 
James Antill <james.antill at redhat.com>
Red Hat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20071210/6d98d682/attachment.pgp