[Rpm-metadata] createrepo: huge memory consumption?
skvidal at fedoraproject.org
Mon Aug 24 14:54:19 UTC 2009
On Mon, 24 Aug 2009, Ryan Lynch wrote:
> My python is decent, and I have already been poking around in the
> code. I will see if I can come up with something and put some patches
> In the meantime, I suppose I can go without the '--update' option. Or,
> maybe I could split my repos up into smaller chunks, since the peak
> memory usage seems to be ~ proportional to the package count.
it is directly related to the size of the xml files - which is related to
the number of pkgs in the repo.
> BTW: I have a couple of patches, now, that I wrote while trying to
> log, parse, and handle 'cobbler reposync' with large numbers
> directories. One affects output formatting, to make logging and
> parsing easier, and the other helps to handle an annoying error:
> - adding a '--log-friendly' option: eliminates the '\r'
> terminal-control magic chars; prints the individual package names, one
> per line, instead;
> - adding a '--remove-on-error': if RPM cannot parse a package file
> (which normally raises an exception), delete the offending file before
> barfing, so subsequent 'reposync' runs will re-download it;
> I think they could be helpful in automated situations, like 'cobbler'.
> I had much frustration: My sync cronjobs were constantly breaking
> down, and I wan't getting useful logs, so I made these changes.
> Any interest in either of these functions?
possibly, but send them over on yum-devel list - since they aren't related
More information about the Rpm-metadata