[Yum-devel] yum memory use question

seth vidal skvidal at phy.duke.edu
Wed Dec 8 15:05:12 UTC 2004


> > foo.setConfigOption('cache', 1)
> 
> This line doesn't work. Should be:
> foo.conf.setConfigOption('cache', 1)

of course it was, I was just testing you. ;) - if you believe that I
have a bridge to sell you. :)


> I see similar with those repos enabled. With the extra repos in my 
> config I hit around 67MB!
> 
> > That seems high to me, any ideas?
> 
> I've traced it through by using lots of prints and raw_input()'s. Memory 
> usage jumps up from around 15MB to 47MB in the self.repos.populateSack() 
> call in doSackSetup().

doSackSetup() is where all the primary.xml metadata gets read into
memory. There is something about either how it is stored or maybe about
freeing the cache information.

So the process for reading in the metadata is:

read in the xml file quickly. write it out to a pickle if possible.
translate the data from the xml memory dump to the structure for the
packageSack.

That's it.


> dobj = repo.cacheHandler.getPrimary(xml, csum)    (~28MB increase)

this reads in the file(s). decompress your primary.xml and see if the
size matches up fairly well to the memory size.

> self.pkgSack.addDict(repo.id, item, dobj, callback) (~10MB increase)

This adds the dobj to the packageSack. However, after that the dobj
should no longer be used and should expire. I've tried explicit deletes
and even tried invoking the garbage collector manually. No Joy.

> This large allocation of memory occurs when processing the metadata for 
> the first repository, which is "Base" (ie. the first loop of the outer 
> loop and the first loop of the inner loop, item == "metadata"). The 
> second repository "updates" doesn't seem to add too much more to yum's 
> memory usage.


Thanks for taking a look at it.

-sv





More information about the Yum-devel mailing list