[Yum-devel] [PATCH] Add queue to presto delta rebuilds
James Antill
james at fedoraproject.org
Thu Mar 7 14:43:31 UTC 2013
On Thu, 2013-03-07 at 08:35 -0500, Zdenek Pavlas wrote:
> Great, looks fine to me. Applied and used to DL today's f17 updates.
> 28M total, (17 drpms, 9 rpms).
>
> It certainly feels faster on the UI side (no blocking during DL),
> but the overall running time was about +10% (reproducible,
> 53s vs 58s, with less than 1s variation in multiple runs).
>
> moreover,
> with --setopt=deltarpm=2 it's 38s vs 44s.
> with --setopt=deltarpm=3 it's 32s vs 45s.
>
> I'm not sure what is the root cause here, but noticed that
> when all DLs are done and there's >1 rebuild queued,
> only one rebuild process was running.
Yeh, for the 2+ worker case I noticed that just after posting the patch
and I think I might have fixed it by turning the dequeue operation into
basically:
while presto.dequeue(block=False):
pass
...but I can't see that doing anything for the 1 worker case, my guess
is they'll have to live with a slight slowdown for being so stingy with
workers :)
I'm pretty sure the problem is that we only check for a new worker at
the end of each download, and with a half decent net connection we can
download more than one drpm in the time it takes to rebuild a drpm (so
the queue fills up, and with only 1 worker there's no way to empty it
unless you have a lot more rpms than you have drpms).
More information about the Yum-devel
mailing list