[Yum-devel] yum: Parallel metadata download

Zdeněk Pavlas zpavlas at redhat.com
Thu Mar 22 15:03:37 UTC 2012


Hi,

This is an attempt to make (most of) MD downloads run in parallel.
It works well for the use cases I tested (repolist, makecache,
update, install).  filelist downloads, started from depsolver
run in parallel, too.

metalink/mirrorlist/repomd download is NOT changed.  These are
small in size and triggered in quite unexpected ways. Callers
would most likely break anyway.

[PATCH 1/3] self.repoXML: no eager downloading of metadata.

Patch #1 is something I worry about.  Currently, when anybody
touches repo.repomd attribute, yum downloads this repo's metadata
(usually just gzipped primary_db).  This has to be changed, as we
need to download from all repos in parallel.

I think the current behavior is broken, and #1 just fixes that,
but there might be users that rely on this.

[PATCH 2/3] Make _retrieveMD() async-able.

Just moved tail into checkfunc, and added kwargs.

[PATCH 3/3] populateSack: parallel metadata download

repo._retrieveMD already checks for duplicated invocations,
so adding an extra pre_populate() pass does not break it.

Error handling is simplistic- failures in 1st pass are ignored,
and (synchronous) retries in 2nd pass and handled as expected.


More information about the Yum-devel mailing list