[Yum-devel] yum: Parallel metadata download
Zdeněk Pavlas
zpavlas at redhat.com
Thu Mar 22 15:03:37 UTC 2012
Hi,
This is an attempt to make (most of) MD downloads run in parallel.
It works well for the use cases I tested (repolist, makecache,
update, install). filelist downloads, started from depsolver
run in parallel, too.
metalink/mirrorlist/repomd download is NOT changed. These are
small in size and triggered in quite unexpected ways. Callers
would most likely break anyway.
[PATCH 1/3] self.repoXML: no eager downloading of metadata.
Patch #1 is something I worry about. Currently, when anybody
touches repo.repomd attribute, yum downloads this repo's metadata
(usually just gzipped primary_db). This has to be changed, as we
need to download from all repos in parallel.
I think the current behavior is broken, and #1 just fixes that,
but there might be users that rely on this.
[PATCH 2/3] Make _retrieveMD() async-able.
Just moved tail into checkfunc, and added kwargs.
[PATCH 3/3] populateSack: parallel metadata download
repo._retrieveMD already checks for duplicated invocations,
so adding an extra pre_populate() pass does not break it.
Error handling is simplistic- failures in 1st pass are ignored,
and (synchronous) retries in 2nd pass and handled as expected.
More information about the Yum-devel
mailing list