[Yum] Strange timeout issue...

[Cris Rhea]

On Fri, Aug 17, 2012 at 10:17:59AM -0400, Seth Vidal wrote:
> 
> 
> 
> On Fri, 17 Aug 2012, Cris Rhea wrote:
> 
> >
> >
> >As you suspected, urlgrabber has horrid performance compared to wget.
> >
> >What's the next step?
> 
> Figure out what it is that's taking all the time/timeout in urlgrabber - 
> strace is probably going to be your pal there.

Did show anything more interesting than the snip of strace I posted in
the first message.  Here's the interesting part of strac'ing urlgrabber:

     0.000111 gettimeofday({1345213462, 342323}, NULL) = 0
     0.000235 clock_gettime(CLOCK_MONOTONIC, {147703, 456218722}) = 0
     0.000105 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 5
     0.000106 fcntl(5, F_GETFL)         = 0x2 (flags O_RDWR)
     0.000091 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
     0.000090 connect(5, {sa_family=AF_INET, sin_port=htons(42898), sin_addr=inet_addr("129.176.212.87")}, 16) = -1 EINPROGRESS (Operation now in progress)
     0.000146 clock_gettime(CLOCK_MONOTONIC, {147703, 456753378}) = 0
     0.000099 poll([{fd=5, events=POLLOUT|POLLWRNORM}], 1, 299963) = 1 ([{fd=5, revents=POLLERR|POLLHUP}])
    62.999460 getsockopt(5, SOL_SOCKET, SO_ERROR, [8589934702], [4]) = 0
     0.000134 close(5)                  = 0
     0.000118 clock_gettime(CLOCK_MONOTONIC, {147766, 456564196}) = 0
     0.000120 sendto(4, "PASV\r\n", 6, MSG_NOSIGNAL, NULL, 0) = 6
     0.000160 clock_gettime(CLOCK_MONOTONIC, {147766, 456846415}) = 0
     0.000103 clock_gettime(CLOCK_MONOTONIC, {147766, 456945981}) = 0
     0.000117 clock_gettime(CLOCK_MONOTONIC, {147766, 457064697}) = 0
     0.000268 poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 3600000) = 1 ([{fd=4, revents=POLLIN|POLLRDNORM}])
     0.000287 recvfrom(4, "227 Entering Passive Mode (129,1"..., 16384, 0, NULL,


I was HOPEFUL that this group would see this and say "That's a known bug
XYZ in PDQ package. The update is out in patch 124.23."

Clearly not a yum specific issue, if yum just uses urlgrabber....


> >PS: Don't want to go off-topic for this list, but why the comments on ftp?
> 
> ftp is a pain to firewall for, most of the ftp daemons aren't actively 
> maintained anymore, in general it is just pain.
> 
> http servers are diverse, good and well-maintained. It is trivial to 
> firewall for them and they have much more flexibility than ftp servers do.
> 
> In general, I try to discourage folks from using ftp if they have a 
> choice.
> 
> -sv

I could see why some might prefer to use a web server for access to a 
SW repo as in this case. There are still some things I can do faster/easier
and with better granularity/control using one of the better FTP packages
(e.g., vsftpd) than with a web server.

Thanks for your help and comments. Unless someone on this list has
info on the "smoking gun", I'll try to track down issues with urlgrabber.

--- Cris

-- 
 Cristopher J. Rhea
 Mayo Clinic - Research Computing Facility
 200 First St SW, Rochester, MN 55905
 crhea at Mayo.EDU
 (507) 284-0587