[Rpm-metadata] createrepo: huge memory consumption?

[Ryan Lynch]

I've noticed that running 'createrepo' against large repositories
takes up an enormous amount of memory, on my machines, at least.

I'm currently using the following options:

 * --cachedir cache
 * --database
 * --update
 * --unique-md-filenames

I see createrepo's 'genpkgmetadata.py' script eating up ~1.5 GB of
memory, or 37% physical memory (runtime ~1 min).  I wouldn't have
noticed this, except that it usually triggers the OOM killer and
randomly whacks firefox, eclipse, etc.

So, is this normal/expected?  Are other users seeing this happen with
large repositories?  I've noticed that dropping the '--update' option
takes a lot longer to run (~8-10 min), but with a much lower peak
memory utilization.  Is it just a case of weighing memory usage vs.
runtime, and making my choice?

Thanks,
Ryan

-- 
Ryan B. Lynch
ryan.b.lynch at gmail.com