[Yum-devel] Re: [Yum] Threading the IO

[Michael Stenner]

Moving this to yum-devel

On Thu, Feb 26, 2004 at 08:00:07AM -0600, David Farning wrote:
> I have been looking at adding the capability to perform IO intensive
> functions in parallel within thread pools.  
> 
> There are a number of areas within yum that could take advantage of
> these optimizations.   The things that jump out sections are file
> download and the header check sections.
> 
> Option one--Make urlgrabber threaded. 
> Advantages
> --all thread stuff and thread bugs would be in urlgrabber not spread
>   through out yum
> --yum coders (beyond Michel) would not need to be aware of threading

Ryan would share that burden at this point :)

> Disadvantages
> --threading would only be available to grabber
> --currently calls to urlGrabber are of the form

<snip>

I'm willing to entertain this sort of idea as a legitimate function of
URLGrabber, but I'll repeat here something that I said in a bugzilla
reply:  it's generally considered rude to "leech" from a single
server.  That is, I don't think I'd want to make yum run multiple
threads to download N files from the same server simultaneously.
Therefore, the only way this would really be useful is if it used
N threads for N mirrors.

> Option two--create a generic thread pool
> Advantage
> --Any parallelizeable functions could run in pool
> 
> Disadvantage
> --all coders will need to be aware of thread issues and resulting bug.  

I think this is the more proper way to deal with the problem.  That's
assuming it's considered worthwhile at all.

					-Michael
-- 
  Michael D. Stenner                            mstenner at ece.arizona.edu
  ECE Department, the University of Arizona                 520-626-1619
  1230 E. Speedway Blvd., Tucson, AZ 85721-0104                 ECE 524G