[Yum-devel] Some URLGrabber Performance Numbers

[Ryan Tomayko]

> More to come...

I grabbed the 0.2 sources from cvs (release-0_2 tag) and ran the same
tests against it. The results show some slowdown in the new stuff. Here
are two runs one right after the other:

urlgrabber-0.2 results:
-------------------------------
1 KB Results:
[full] mean: 0.928 ms, median: 0.911 ms, min: 0.894 ms, max: 1.931 ms
[raw]  mean: 0.650 ms, median: 0.635 ms, min: 0.625 ms, max: 1.520 ms
10 KB Results:
[full] mean: 1.375 ms, median: 1.361 ms, min: 1.289 ms, max: 2.846 ms
[raw]  mean: 1.026 ms, median: 1.014 ms, min: 0.947 ms, max: 2.226 ms
100 KB Results:
[full] mean: 2.472 ms, median: 2.387 ms, min: 2.105 ms, max: 23.158 ms
[raw]  mean: 1.666 ms, median: 1.634 ms, min: 1.369 ms, max: 3.473 ms
1000 KB Results:
[full] mean: 12.422 ms, median: 12.251 ms, min: 12.110 ms, max: 25.186 ms
[raw]  mean: 7.561 ms, median: 7.476 ms, min: 7.380 ms, max: 10.010 ms
10000 KB Results:
[full] mean: 113.695 ms, median: 113.236 ms, min: 111.376 ms, max: 205.268 ms
[raw]  mean: 70.196 ms, median: 67.108 ms, min: 65.778 ms, max: 277.880 ms


urlgrabber-0.3 results:
-------------------------------
1 KB Results:
[full] mean: 2.065 ms, median: 1.999 ms, min: 1.954 ms, max: 6.508 ms
[raw]  mean: 1.793 ms, median: 1.721 ms, min: 1.689 ms, max: 6.158 ms
10 KB Results:
[full] mean: 2.534 ms, median: 2.462 ms, min: 2.379 ms, max: 6.948 ms
[raw]  mean: 2.173 ms, median: 2.101 ms, min: 2.024 ms, max: 8.239 ms
100 KB Results:
[full] mean: 3.798 ms, median: 3.594 ms, min: 3.539 ms, max: 64.925 ms
[raw]  mean: 2.812 ms, median: 2.724 ms, min: 2.680 ms, max: 7.353 ms
1000 KB Results:
[full] mean: 14.782 ms, median: 14.576 ms, min: 14.325 ms, max: 29.981 ms
[raw]  mean: 8.776 ms, median: 8.637 ms, min: 8.489 ms, max: 13.151 ms
10000 KB Results:
[full] mean: 127.247 ms, median: 126.060 ms, min: 123.042 ms, max: 235.105 ms
[raw]  mean: 69.888 ms, median: 68.713 ms, min: 67.296 ms, max: 415.250 ms

The trend I notice right off is that the new stuff is somewhere between
1 and 2 milliseconds slower than the 0.2 code for both full and raw runs
across the board. The good thing is that this seems to be static time
and does not increase as file size increases. I'm assuming that these
extra couple of milliseconds are being spent dealing with
URLGrabberOptions. We're copying this object a few times for each
request and none of that was happening in 0.2.

I'm going to look into reducing the copying of URLGrabberOptions and
maybe a few other things but all in all I think these numbers look
pretty good. We've added a few new classes (URLGrabber,
URLGrabberOptions) and URLGrabberFileObject is always used now so
there's definitely some object overhead that wasn't present in 0.2. My
personal opinion is that 2 ms is a small price to pay for the
improvements in maintainability. My goal is to shave 0.3 ms to 0.5 ms
off the 2ms we're seeing now with small tweaks and changes. If we need
to shave more time, we will probably need to look at making more drastic
changes--like rethinking URLGrabberOptions or URLGrabberFileObject
usage.

- Ryan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.baseurl.org/pipermail/yum-devel/attachments/20040303/c9d32132/attachment.pgp