[Yum] PATCH simuleneous download for packages

[Tim Lauridsen]

On 04/26/2009 09:28 PM, pranjal pandit wrote:
> Hello,
> The patch attached performs simultaneous downloads of packages by
> creating two different urlgrab objects and passing them different
> package names to download.
>
> The download is fine but there is a problem in display in case of
> simultaneous downloads of  more than one packages. The display
> percentages and download speed is shown incorrect. The progress object
> used is of class TextMeter() from the module urlgrabber.progress.
>
> In case of a single package download, the code works perfectly fine.
>
> --Regards
> --Pranjal Pandit
> Computer Science Engineering Department
> NIT HAMIRPUR
>    
> ------------------------------------------------------------------------
>
> diff --git a/cli.py b/cli.py
> index 0374667..cb8507c 100644
> --- a/cli.py
> +++ b/cli.py
> @@ -25,9 +25,9 @@ import sys
>   import time
>   import random
>   import logging
> +import threading
>   from optparse import OptionParser
>   import rpm
> -
>   import output
>   import shell
>   import yum
> @@ -62,7 +62,7 @@ class CliError(yum.Errors.YumBaseError):
>           yum.Errors.YumBaseError.__init__(self)
>           self.args = args
>
> -class YumBaseCli(yum.YumBase, output.YumOutput):
> +class YumBaseCli(yum.YumBase, output.YumOutput,threading.Thread):
>       """This is the base class for yum cli.
>          Inherits from yum.YumBase and output.YumOutput """
>
> @@ -98,7 +98,11 @@ class YumBaseCli(yum.YumBase, output.YumOutput):
>           self.registerCommand(yumcommands.RepoListCommand())
>           self.registerCommand(yumcommands.HelpCommand())
>           self.registerCommand(yumcommands.ReInstallCommand())
> -
> +        self.helpDownloadPkgs=yum.HDP()
>    
yum.HDP() look very ugly to me, use something longer and more 
descriptive in mixed Case (MyDammUsefulClass)
> +        self.yB=yum.YumBase()
>    
Why do you define a yum.yumBase() class inside a class there inherit 
from yum.yumBase, you can just use self
> +        threading.Thread.__init__(self)
> +    def run(self):
> +        self.yB.downloadPkgs2(self.helpDownloadPkgs)
>       def registerCommand(self, command):
>           for name in command.getNames():
>               if self.yum_cli_commands.has_key(name):
> @@ -396,8 +400,22 @@ class YumBaseCli(yum.YumBase, output.YumOutput):
>
>           self.verbose_logger.log(yum.logginglevels.INFO_2,
>               _('Downloading Packages:'))
> -        problems = self.downloadPkgs(downloadpkgs, callback_total=self.download_callback_total_cb)
> -
> +
> +        self.yB.downloadPkgs1(downloadpkgs,self.helpDownloadPkgs)
> +
> +        if self.yB.numberOfPackages() is 1:
> +            self.yB.downloadPkgs2(self.helpDownloadPkgs)
> +        else:
> +            call1=YumBaseCli()
> +            call2=YumBaseCli()
>    
here you make 2 new instances if the class you already are inside, very 
confusing and a memory nightmare
now you have 3 YumBaseCli classes and 3 Extra YumBase clasess running.
> +            cl=[]
> +            cl.append(call1)
> +            cl.append(call2)
> +            call1.start()
> +            call2.start()
> +            for c in cl:
> +                c.join()
> +        problems = self.yB.downloadPkgs3()
>           if len(problems)>  0:
>               errstring = ''
>               errstring += _('Error Downloading Packages:\n')
> diff --git a/po/yum.pot b/po/yum.pot
> index 9e01e13..c868dc9 100644
> --- a/po/yum.pot
> +++ b/po/yum.pot
>    
yumex.pot is a genratend files, no need to send patches against that.

There is a lot of other issues, it makes the code very confusing. :(

I need to be convinced that it is worth the effort.

I think you are attacking the "problem" in the wrong way. you should try 
making a plugin instead there is downloading
the packages multi threaded in a "predownload" hook. The you need no 
changes to yumbase and yum will just skip the downloading because all 
the packages is in the cache already and continue with the transaction. 
You don't even need to
use urlgrapper, you can use something else like pycurl, i think that i 
already supports multiple downloads, so you dont need to use python 
threads :)
When it is rocking then it might be an idea to itegrate it into yumbase, 
but it in lower level than you current attempt.
So it can be useful to Yum API users also and not just yum cli users.

Hope that i don't sound to refusing, it was not my intention :)

Tim


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.baseurl.org/pipermail/yum/attachments/20090427/c47923d1/attachment.htm>