[Rpm-metadata] [PATCH] switch --update to use the sqlite dbs instead of the xml files. Should massively impact memory footprint and hopefully only marginally impact performance.

Dennis Gregorovic dgregor at redhat.com
Wed Sep 7 17:53:03 UTC 2011


On Wed, 2011-09-07 at 12:58 -0400, seth vidal wrote:
> On Wed, 2011-09-07 at 12:18 -0400, Dennis Gregorovic wrote:
> > On Fri, 2011-07-15 at 17:53 -0400, Seth Vidal wrote:
> > > -    def getNodes(self, relpath):
> > > -        """Return base, filelist, and other nodes for file, if they exist
> > >  
> > > -        Returns a tuple of nodes, or None if not found
> > > +    def getNodes(self, relpath):
> > > +        """return a package object based on relative path of pkg
> > >          """
> > > -        bnode = self.basenodes.get(relpath,None)
> > > -        if bnode is None:
> > > -            return None
> > > -        pkgid = self.pkg_ids.get(relpath,None)
> > > -        if pkgid is None:
> > > -            print _("No pkgid found for: %s") % relpath
> > > -            return None
> > > -        fnode = self.filesnodes.get(pkgid,None)
> > > -        if fnode is None:
> > > -            return None
> > > -        onode = self.othernodes.get(pkgid,None)
> > > -        if onode is None:
> > > -            return None
> > > -        return bnode, fnode, onode
> > > -
> > > -    def freeNodes(self,relpath):
> > > -        #causing problems
> > > -        """Free up nodes corresponding to file, if possible"""
> > > -        bnode = self.basenodes.get(relpath,None)
> > > -        if bnode is None:
> > > -            print "Missing node for %s" % relpath
> > > -            return
> > > -        bnode.unlinkNode()
> > > -        bnode.freeNode()
> > > -        del self.basenodes[relpath]
> > > -        pkgid = self.pkg_ids.get(relpath,None)
> > > -        if pkgid is None:
> > > -            print _("No pkgid found for: %s") % relpath
> > > +        if relpath in self.pkg_tups_by_path:
> > > +            pkgtup = self.pkg_tups_by_path[relpath]
> > > +            return self._repo.sack.searchPkgTuple(pkgtup)[0]
> > > +        else:
> > > +            print _("No pkg found for: %s") % relpath
> > >              return None
> > 
> > Seth,
> > 
> > Thank you very much for this patch.  I've locally backported it to RHEL
> > 6 and the memory savings are tremendous.  However, I just noticed one
> > small issue.  Before, the getNodes call would only print "No pkg found
> > for: %" when a package was found in the basenodes cache but not the
> > pkg_ids cache.  That was an edge condition that never actually happened
> > as far as I can tell, so the net effect was that "No pkg found" did not
> > get printed.  After the code change, "No pkg found" is now printed every
> > time getNodes is called with a package that isn't cached.  
> 
> Yah - I think someone else noticed that. Dgilmore, I think.
> 
> I was a bit torn about it - since it is quasi-useful to know which ones
> went walk-about and which ones did not. However, it isn't helpful for
> the general case of 'yes, I know I removed pkgs from the repo, just
> rebuild it'.
> 
> I suspect it is really only useful when debugging something ridiculous.
> 
> Anyone object to me just pulling it?
> -sv

Maybe only print out the message if run with --verbose?



More information about the Rpm-metadata mailing list